public class Tokenizers
extends java.lang.Object
Constructor and Description |
---|
Tokenizers() |
Modifier and Type | Method and Description |
---|---|
static Tokenizer |
forChar()
Creates a tokenizer that's gonna tokenize a single quoted character literal possibly with escape character '\'
|
static Tokenizer |
forDecimal()
Get the Tokenizer object that creates a decimal number literal token.
|
static Tokenizer |
forDecLong()
Get the Tokenizer object that interprets the recognized character range
as a decimal integer and translate it to a long value.
|
static Tokenizer |
forHexLong()
Get the Tokenizer object that interprets the recognized character range
as a hex integer and translate it to a long value.
|
static Tokenizer |
forInteger()
Get the Tokenizer object that creates an integer literal token.
|
static Tokenizer |
forOctLong()
Get the Tokenizer object that interprets the recognized character range
as a oct integer and translate it to a long value.
|
static Tokenizer |
forQuotedString(char open,
char close)
Get the Tokenizer object that converts a string literal quoted by a pair of
opening and closing characters.
|
static Tokenizer |
forQuotedString(java.lang.String open,
java.lang.String close)
Get the Tokenizer object that converts a string literal quoted by a pair of
opening and closing strings.
|
static Tokenizer |
forReservedWord()
Get the Tokenizer object that creates a reserved word token.
|
static Tokenizer |
forSimpleStringLiteral()
Get the Tokenizer object that converts a string literal quoted by '"'
to a string object.
|
static Tokenizer |
forSqlStringLiteral()
Get the Tokenizer object that converts a sql string literal quoted by single quote
to a string object.
|
static Tokenizer |
forString()
Get the Tokenizer object that creates a string object.
|
static <T> Tokenizer |
forTypedToken(T type)
Create a tokenizer that tokenizes the recognized character range
to a TypedToken object.
|
static Tokenizer |
forWord()
Get the Tokenizer object that creates a word token.
|
public static <T> Tokenizer forTypedToken(T type)
type
- the token type.public static Tokenizer forReservedWord()
public static Tokenizer forWord()
public static Tokenizer forInteger()
public static Tokenizer forDecimal()
public static Tokenizer forChar()
public static Tokenizer forString()
public static Tokenizer forDecLong()
public static Tokenizer forHexLong()
public static Tokenizer forOctLong()
public static Tokenizer forSimpleStringLiteral()
public static Tokenizer forSqlStringLiteral()
public static Tokenizer forQuotedString(char open, char close)
TokenQuoted
object.public static Tokenizer forQuotedString(java.lang.String open, java.lang.String close)
TokenQuoted
object.