Provides some predefined tokens and Tokenizer implementations.
See: Description
Class Summary | |
---|---|
NamedToken | A token that has a name. |
Tokenizers | The facade class for all pre-built Tokenizer implementations. |
TokenQuoted | represents a string that is quoted by a open and close string. |
Tokens | This class create some basic tokens. |
TokenType | This represents all pre-built token types. |
TypedToken<Type> | Represents a token associated with a token type. |