|
|
Abstract class to define a source of tokens for the parser.
int Offset | Offset |
[protected]
int Line | Line |
[protected]
tokenizer (int offset=0,int line=0)
| tokenizer |
Create a tokenizer. Optionally specify which line and offset to start at.
~tokenizer ()
| ~tokenizer |
[virtual]
QString getToken (bool forward=true,bool comment=false)
| getToken |
[pure virtual]
Get a token from the string.
Parameters:
forward | Go forward or backwards to get next token. |
comment | Include comments as tokens. |
int line (void)
| line |
[virtual]
Get the current line of the tokenizer. A line is defined by a \n character
int offset (void)
| offset |
[virtual]
Current offset of the tokenizer (Should point to the character after the last token in the specified direction)
void setOffset (int offset)
| setOffset |
[virtual]
Set a new offset of tokenzer.
void setLine (int line)
| setLine |
[virtual]
Set new current line of tokenizer. This will not affect the current position of the tokenizer.
QString remaining (bool eol)
| remaining |
[pure virtual]
Get the data remaining after the current position.
Parameters:
eol | If true end of line, otherwise end of tokenizer. |