CharScanner Class Reference
Superclass of generated lexers. More...
#include <CharScanner.hpp>
Inheritance diagram for CharScanner:


Public Member Functions | |
CharScanner (InputBuffer &cb, bool case_sensitive) | |
CharScanner (InputBuffer *cb, bool case_sensitive) | |
CharScanner (const LexerSharedInputState &state, bool case_sensitive) | |
virtual | ~CharScanner () |
virtual int | LA (int i) |
virtual void | append (char c) |
virtual void | append (const std::string &s) |
virtual void | commit () |
virtual void | consume () |
virtual void | consumeUntil (int c) |
Consume chars until one matches the given char. | |
virtual void | consumeUntil (const BitSet &set) |
Consume chars until one matches the given set. | |
virtual int | mark () |
Mark the current position and return a id for it. | |
virtual void | rewind (int pos) |
Rewind the scanner to a previously marked position. | |
virtual void | match (int c) |
See if input contains character 'c' throw MismatchedCharException if not. | |
virtual void | match (const BitSet &b) |
See if input contains element from bitset b throw MismatchedCharException if not. | |
virtual void | match (const std::string &s) |
See if input contains string 's' throw MismatchedCharException if not. | |
virtual void | matchNot (int c) |
See if input does not contain character 'c' throw MismatchedCharException if not. | |
virtual void | matchRange (int c1, int c2) |
See if input contains character in range c1-c2 throw MismatchedCharException if not. | |
virtual bool | getCaseSensitive () const |
virtual void | setCaseSensitive (bool t) |
virtual bool | getCaseSensitiveLiterals () const =0 |
virtual int | getLine () const |
Get the line the scanner currently is in (starts at 1). | |
virtual void | setLine (int l) |
set the line number | |
virtual int | getColumn () const |
Get the column the scanner currently is in (starts at 1). | |
virtual void | setColumn (int c) |
set the column number | |
virtual const std::string & | getFilename () const |
get the filename for the file currently used | |
virtual void | setFilename (const std::string &f) |
Set the filename the scanner is using (used in error messages). | |
virtual bool | getCommitToPath () const |
virtual void | setCommitToPath (bool commit) |
virtual const std::string & | getText () const |
return a copy of the current text buffer | |
virtual void | setText (const std::string &s) |
virtual void | resetText () |
virtual RefToken | getTokenObject () const |
virtual void | newline () |
Used to keep track of line breaks, needs to be called from within generated lexers when a "\n \r" is encountered. | |
virtual void | tab () |
Advance the current column number by an appropriate amount according to the tabsize. | |
int | setTabsize (int size) |
set the tabsize. Returns the old tabsize | |
int | getTabSize () const |
Return the tabsize used by the scanner. | |
void | panic () |
Called when a unrecoverable error is encountered. | |
void | panic (const std::string &s) |
Called when a unrecoverable error is encountered. | |
virtual void | reportError (const RecognitionException &e) |
Report exception errors caught in nextToken(). | |
virtual void | reportError (const std::string &s) |
Parser error-reporting function can be overridden in subclass. | |
virtual void | reportWarning (const std::string &s) |
Parser warning-reporting function can be overridden in subclass. | |
virtual InputBuffer & | getInputBuffer () |
virtual LexerSharedInputState | getInputState () |
virtual void | setInputState (LexerSharedInputState state) |
set the input state for the lexer. | |
virtual void | setTokenObjectFactory (factory_type factory) |
Set the factory for created tokens. | |
virtual int | testLiteralsTable (int ttype) const |
Test the token text against the literals table Override this method to perform a different literals test. | |
virtual int | testLiteralsTable (const std::string &txt, int ttype) const |
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token. | |
virtual int | toLower (int c) const |
Override this method to get more specific case handling. | |
virtual void | uponEOF () |
This method is called by YourLexer::nextToken() when the lexer has hit EOF condition. | |
virtual void | traceIndent () |
Methods used to change tracing behavior. | |
virtual void | traceIn (const char *rname) |
virtual void | traceOut (const char *rname) |
Static Public Attributes | |
const int | EOF_CHAR = EOF |
Protected Types | |
typedef RefToken(* | factory_type )() |
Protected Member Functions | |
virtual RefToken | makeToken (int t) |
Create a new RefToken of type t. | |
Protected Attributes | |
std::string | text |
Text of current token. | |
bool | saveConsumedInput |
flag indicating wether consume saves characters | |
factory_type | tokenFactory |
Factory for tokens. | |
bool | caseSensitive |
Is this lexer case sensitive. | |
std::map< std::string, int, CharScannerLiteralsLess > | literals |
RefToken | _returnToken |
used to return tokens w/o using return val | |
LexerSharedInputState | inputState |
Input state, gives access to input stream, shared among different lexers. | |
bool | commitToPath |
Used during filter mode to indicate that path is desired. | |
int | tabsize |
tab size the scanner uses. | |
int | traceDepth |
Private Member Functions | |
CharScanner (const CharScanner &other) | |
CharScanner & | operator= (const CharScanner &other) |
Static Private Attributes | |
const int | NO_CHAR = 0 |
Detailed Description
Superclass of generated lexers.
Definition at line 79 of file CharScanner.hpp.
Member Typedef Documentation
|
Definition at line 81 of file CharScanner.hpp. |
Constructor & Destructor Documentation
|
Definition at line 26 of file CharScanner.cpp. |
|
Definition at line 38 of file CharScanner.cpp. References setTokenObjectFactory(). |
|
Definition at line 50 of file CharScanner.cpp. References LexerSharedInputState, and setTokenObjectFactory(). |
|
Definition at line 87 of file CharScanner.hpp. References CharScanner. |
|
|
Member Function Documentation
|
Definition at line 103 of file CharScanner.hpp. References text(). |
|
Definition at line 93 of file CharScanner.hpp. References text(). Referenced by consume(). |
|
Definition at line 109 of file CharScanner.hpp. |
|
Definition at line 62 of file CharScanner.cpp. References append(), inputState, LA(), and tab(). |
|
Consume chars until one matches the given set.
Definition at line 129 of file CharScanner.hpp. References BitSet::member(). |
|
Consume chars until one matches the given char.
Definition at line 117 of file CharScanner.hpp. |
|
Definition at line 211 of file CharScanner.hpp. |
|
Referenced by CharScannerLiteralsLess::operator()(). |
|
Get the column the scanner currently is in (starts at 1).
Definition at line 236 of file CharScanner.hpp. |
|
Definition at line 257 of file CharScanner.hpp. |
|
get the filename for the file currently used
Definition at line 247 of file CharScanner.hpp. Referenced by reportError(), and reportWarning(). |
|
Definition at line 335 of file CharScanner.hpp. |
|
Definition at line 340 of file CharScanner.hpp. References LexerSharedInputState. |
|
Get the line the scanner currently is in (starts at 1).
Definition at line 224 of file CharScanner.hpp. |
|
Return the tabsize used by the scanner.
Definition at line 316 of file CharScanner.hpp. |
|
return a copy of the current text buffer
Definition at line 268 of file CharScanner.hpp. |
|
Definition at line 285 of file CharScanner.hpp. References RefToken. |
|
Definition at line 488 of file CharScanner.hpp. References inputState, and toLower(). Referenced by consume(), traceIn(), and traceOut(). |
|
Create a new RefToken of type t.
Definition at line 444 of file CharScanner.hpp. References RefToken. |
|
Mark the current position and return a id for it.
Definition at line 141 of file CharScanner.hpp. |
|
See if input contains string 's' throw MismatchedCharException if not.
Definition at line 172 of file CharScanner.hpp. References len. |
|
See if input contains element from bitset b throw MismatchedCharException if not.
Definition at line 163 of file CharScanner.hpp. References BitSet::member(). |
|
See if input contains character 'c' throw MismatchedCharException if not.
Definition at line 152 of file CharScanner.hpp. |
|
See if input does not contain character 'c' throw MismatchedCharException if not.
Definition at line 189 of file CharScanner.hpp. |
|
See if input contains character in range c1-c2 throw MismatchedCharException if not.
Definition at line 201 of file CharScanner.hpp. |
|
Used to keep track of line breaks, needs to be called from within generated lexers when a "\n \r" is encountered.
Definition at line 293 of file CharScanner.hpp. |
|
|
|
Called when a unrecoverable error is encountered.
Definition at line 96 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE. |
|
Called when a unrecoverable error is encountered.
Definition at line 90 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE. |
|
Parser error-reporting function can be overridden in subclass.
Definition at line 109 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE, and getFilename(). |
|
Report exception errors caught in nextToken().
Definition at line 103 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE, and RecognitionException::toString(). |
|
Parser warning-reporting function can be overridden in subclass.
Definition at line 118 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE, and getFilename(). |
|
Definition at line 278 of file CharScanner.hpp. References text(). |
|
Rewind the scanner to a previously marked position.
Definition at line 146 of file CharScanner.hpp. |
|
Definition at line 216 of file CharScanner.hpp. |
|
set the column number
Definition at line 241 of file CharScanner.hpp. |
|
Definition at line 262 of file CharScanner.hpp. |
|
Set the filename the scanner is using (used in error messages).
Definition at line 252 of file CharScanner.hpp. |
|
set the input state for the lexer.
Definition at line 347 of file CharScanner.hpp. References LexerSharedInputState. |
|
set the line number
Definition at line 230 of file CharScanner.hpp. |
|
set the tabsize. Returns the old tabsize
Definition at line 309 of file CharScanner.hpp. |
|
Definition at line 273 of file CharScanner.hpp. References text(). |
|
Set the factory for created tokens.
Definition at line 353 of file CharScanner.hpp. Referenced by CharScanner(). |
|
Advance the current column number by an appropriate amount according to the tabsize. This methad is called automatically from consume() Definition at line 302 of file CharScanner.hpp. Referenced by consume(). |
|
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.
Definition at line 374 of file CharScanner.hpp. References ANTLR_USE_NAMESPACE. |
|
Test the token text against the literals table Override this method to perform a different literals test.
Definition at line 361 of file CharScanner.hpp. References ANTLR_USE_NAMESPACE, and text(). |
|
Override this method to get more specific case handling.
Definition at line 383 of file CharScanner.hpp. Referenced by LA(). |
|
Definition at line 132 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE, LA(), traceDepth, and traceIndent(). Referenced by CharScanner::Tracer::Tracer(). |
|
Methods used to change tracing behavior.
Definition at line 126 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE. Referenced by traceIn(), and traceOut(). |
|
Definition at line 140 of file CharScanner.cpp. References ANTLR_USE_NAMESPACE, LA(), traceDepth, and traceIndent(). Referenced by CharScanner::Tracer::~Tracer(). |
|
This method is called by YourLexer::nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition. uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token. You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file"). Definition at line 406 of file CharScanner.hpp. |
Member Data Documentation
|
used to return tokens w/o using return val
Definition at line 430 of file CharScanner.hpp. |
|
Is this lexer case sensitive.
Definition at line 427 of file CharScanner.hpp. |
|
Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true; Definition at line 439 of file CharScanner.hpp. |
|
Definition at line 150 of file CharScanner.cpp. |
|
Input state, gives access to input stream, shared among different lexers.
Definition at line 433 of file CharScanner.hpp. |
|
Definition at line 428 of file CharScanner.hpp. |
|
Definition at line 149 of file CharScanner.cpp. |
|
flag indicating wether consume saves characters
Definition at line 425 of file CharScanner.hpp. |
|
tab size the scanner uses.
Definition at line 441 of file CharScanner.hpp. |
|
Text of current token.
Definition at line 423 of file CharScanner.hpp. |
|
Factory for tokens.
Definition at line 426 of file CharScanner.hpp. |
|
Definition at line 474 of file CharScanner.hpp. Referenced by traceIn(), and traceOut(). |
The documentation for this class was generated from the following files:
- lib/antlr/antlr/CharScanner.hpp
- lib/antlr/src/CharScanner.cpp