Document: ezcDocumentWikiCreoleTokenizer
[ ]
[ Conversion ] [ Styles ]
[ ]
[ ]
[ ]
[ ]
Class: ezcDocumentWikiCreoleTokenizer
|
Tokenizer for Creole wiki documents. [
source]
The Creole wiki syntax is a started effort to unify wiki markup languages. Its documentation can be found at:
http://www.wikicreole.org/
For the basic workings of the tokenizer see the class level documentation in the ezcDocumentWikiTokenizer class.
Parents
ezcDocumentWikiTokenizer
|
--ezcDocumentWikiCreoleTokenizer
Constants
SPECIAL_CHARS
= '/*^,#_~\\\\\\[\\]{}|=-'
|
Special characters, which do have some special meaaning and though may not have been matched otherwise. |
TEXT_END_CHARS
= '/*^,#_~\\\\\\[\\]{}|=\\r\\n\\t\\x20-'
|
Characters ending a pure text section. |
WHITESPACE_CHARS
= '[\\x20\\t]'
|
Common whitespace characters. The vertical tab is excluded, because it causes strange problems with PCRE. |
Inherited Member Variables
From
ezcDocumentWikiTokenizer:
Method Summary
Inherited Methods
From
ezcDocumentWikiTokenizer :
Methods
__construct
void __construct(
)
Construct tokenizer
Create token array with regular repression matching the respective token.
Redefinition of
filterTokens
array filterTokens(
$tokens )
Filter tokens
Method to filter tokens, after the input string ahs been tokenized. The filter should extract additional information from tokens, which are not generally available yet, like the depth of a title depending on the title markup.
Parameters
| Name |
Type |
Description |
$tokens |
array |
|
Redefinition of
parsePluginContents
Parse plugin contents
Plugins are totally different in each wiki component and its contents should not be passed through the normal wiki parser. So we fetch its contents completely and let each tokinzer extract names and parameters from the complete token itself.
Parameters
Last updated: Mon, 29 Jun 2009
