Path

ez components / documentation / api reference / trunk / document

eZ Components trunk

Document: ezcDocumentWikiDokuwikiTokenizer

[ Tutorial ] [ Conversion ] [ Styles ] [ Class tree ] [ Element index ] [ ChangeLog ] [ Credits ]

Class: ezcDocumentWikiDokuwikiTokenizer

Tokenizer for Dokuwiki wiki documents. [source]
The Dokuwiki wiki is a very popular wiki, which for example is currently used at http://wiki.php.net. The Dokuwiki syntax definition can be found at:
http://www.dokuwiki.org/syntax
For the basic workings of the tokenizer see the class level documentation in the ezcDocumentWikiTokenizer class.

Parents

ezcDocumentWikiTokenizer
   |
   --ezcDocumentWikiDokuwikiTokenizer

Constants

SPECIAL_CHARS = '/*^,\'_<>\\\\\\[\\]{}()|=' Special characters, which do have some special meaaning and though may not have been matched otherwise.
TEXT_END_CHARS = '/*^,\'_<>\\\\\\[\\]{}()|=\\r\\n\\t\\x20' Characters ending a pure text section.
WHITESPACE_CHARS = '[\\x20\\t]' Common whitespace characters. The vertical tab is excluded, because it causes strange problems with PCRE.

Inherited Member Variables

From ezcDocumentWikiTokenizer:
protected  ezcDocumentWikiTokenizer::$tokens

Method Summary

public void __construct( )
Construct tokenizer
protected array filterTokens( $tokens )
Filter tokens
protected void parsePluginContents( $plugin )
Parse plugin contents

Inherited Methods

From ezcDocumentWikiTokenizer :
public abstract void ezcDocumentWikiTokenizer::__construct()
Construct tokenizer
protected void ezcDocumentWikiTokenizer::convertTabs()
Convert tabs to spaces
protected abstract array ezcDocumentWikiTokenizer::filterTokens()
Filter tokens
public array ezcDocumentWikiTokenizer::tokenizeFile()
Tokenize the given file
public array ezcDocumentWikiTokenizer::tokenizeString()
Tokenize the given string

Methods

__construct

void __construct( )
Construct tokenizer
Create token array with regular repression matching the respective token.

Redefinition of

Method Description
ezcDocumentWikiTokenizer::__construct() Construct tokenizer

filterTokens

array filterTokens( $tokens )
Filter tokens
Method to filter tokens, after the input string ahs been tokenized. The filter should extract additional information from tokens, which are not generally available yet, like the depth of a title depending on the title markup.

Parameters

Name Type Description
$tokens array  

Redefinition of

Method Description
ezcDocumentWikiTokenizer::filterTokens() Filter tokens

parsePluginContents

void parsePluginContents( ezcDocumentWikiPluginToken $plugin )
Parse plugin contents
Plugins are totally different in each wiki component and its contents should not be passed through the normal wiki parser. So we fetch its contents completely and let each tokinzer extract names and parameters from the complete token itself.

Parameters

Name Type Description
$plugin ezcDocumentWikiPluginToken  

Last updated: Mon, 21 Dec 2009