KDevelop API Documentation

CharScanner Class Reference

Superclass of generated lexers. More...

#include <CharScanner.hpp>

Inheritance diagram for CharScanner:

TokenStream List of all members.

Public Member Functions

 CharScanner (InputBuffer &cb, bool case_sensitive)
 CharScanner (InputBuffer *cb, bool case_sensitive)
 CharScanner (const LexerSharedInputState &state, bool case_sensitive)
virtual ~CharScanner ()
virtual int LA (int i)
virtual void append (char c)
virtual void append (const std::string &s)
virtual void commit ()
virtual void consume ()
virtual void consumeUntil (int c)
 Consume chars until one matches the given char.

virtual void consumeUntil (const BitSet &set)
 Consume chars until one matches the given set.

virtual int mark ()
 Mark the current position and return a id for it.

virtual void rewind (int pos)
 Rewind the scanner to a previously marked position.

virtual void match (int c)
 See if input contains character 'c' throw MismatchedCharException if not.

virtual void match (const BitSet &b)
 See if input contains element from bitset b throw MismatchedCharException if not.

virtual void match (const std::string &s)
 See if input contains string 's' throw MismatchedCharException if not.

virtual void matchNot (int c)
 See if input does not contain character 'c' throw MismatchedCharException if not.

virtual void matchRange (int c1, int c2)
 See if input contains character in range c1-c2 throw MismatchedCharException if not.

virtual bool getCaseSensitive () const
virtual void setCaseSensitive (bool t)
virtual bool getCaseSensitiveLiterals () const =0
virtual int getLine () const
 Get the line the scanner currently is in (starts at 1).

virtual void setLine (int l)
 set the line number

virtual int getColumn () const
 Get the column the scanner currently is in (starts at 1).

virtual void setColumn (int c)
 set the column number

virtual const std::string & getFilename () const
 get the filename for the file currently used

virtual void setFilename (const std::string &f)
 Set the filename the scanner is using (used in error messages).

virtual bool getCommitToPath () const
virtual void setCommitToPath (bool commit)
virtual const std::string & getText () const
 return a copy of the current text buffer

virtual void setText (const std::string &s)
virtual void resetText ()
virtual RefToken getTokenObject () const
virtual void newline ()
 Used to keep track of line breaks, needs to be called from within generated lexers when a "\n \r" is encountered.

virtual void tab ()
 Advance the current column number by an appropriate amount according to the tabsize.

int setTabsize (int size)
 set the tabsize. Returns the old tabsize

int getTabSize () const
 Return the tabsize used by the scanner.

void panic ()
 Called when a unrecoverable error is encountered.

void panic (const std::string &s)
 Called when a unrecoverable error is encountered.

virtual void reportError (const RecognitionException &e)
 Report exception errors caught in nextToken().

virtual void reportError (const std::string &s)
 Parser error-reporting function can be overridden in subclass.

virtual void reportWarning (const std::string &s)
 Parser warning-reporting function can be overridden in subclass.

virtual InputBuffergetInputBuffer ()
virtual LexerSharedInputState getInputState ()
virtual void setInputState (LexerSharedInputState state)
 set the input state for the lexer.

virtual void setTokenObjectFactory (factory_type factory)
 Set the factory for created tokens.

virtual int testLiteralsTable (int ttype) const
 Test the token text against the literals table Override this method to perform a different literals test.

virtual int testLiteralsTable (const std::string &txt, int ttype) const
 Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.

virtual int toLower (int c) const
 Override this method to get more specific case handling.

virtual void uponEOF ()
 This method is called by YourLexer::nextToken() when the lexer has hit EOF condition.

virtual void traceIndent ()
 Methods used to change tracing behavior.

virtual void traceIn (const char *rname)
virtual void traceOut (const char *rname)

Static Public Attributes

const int EOF_CHAR = EOF

Protected Types

typedef RefToken(* factory_type )()

Protected Member Functions

virtual RefToken makeToken (int t)
 Create a new RefToken of type t.


Protected Attributes

std::string text
 Text of current token.

bool saveConsumedInput
 flag indicating wether consume saves characters

factory_type tokenFactory
 Factory for tokens.

bool caseSensitive
 Is this lexer case sensitive.

std::map< std::string, int,
CharScannerLiteralsLess
literals
RefToken _returnToken
 used to return tokens w/o using return val

LexerSharedInputState inputState
 Input state, gives access to input stream, shared among different lexers.

bool commitToPath
 Used during filter mode to indicate that path is desired.

int tabsize
 tab size the scanner uses.

int traceDepth

Private Member Functions

 CharScanner (const CharScanner &other)
CharScanneroperator= (const CharScanner &other)

Static Private Attributes

const int NO_CHAR = 0

Detailed Description

Superclass of generated lexers.

Definition at line 79 of file CharScanner.hpp.


Member Typedef Documentation

typedef RefToken(* CharScanner::factory_type)() [protected]
 

Definition at line 81 of file CharScanner.hpp.


Constructor & Destructor Documentation

CharScanner::CharScanner InputBuffer cb,
bool  case_sensitive
 

Definition at line 26 of file CharScanner.cpp.

CharScanner::CharScanner InputBuffer cb,
bool  case_sensitive
 

Definition at line 38 of file CharScanner.cpp.

References setTokenObjectFactory().

CharScanner::CharScanner const LexerSharedInputState state,
bool  case_sensitive
 

Definition at line 50 of file CharScanner.cpp.

References setTokenObjectFactory().

virtual CharScanner::~CharScanner  )  [inline, virtual]
 

Definition at line 87 of file CharScanner.hpp.

References CharScanner.

CharScanner::CharScanner const CharScanner other  )  [private]
 


Member Function Documentation

virtual void CharScanner::append const std::string &  s  )  [inline, virtual]
 

Definition at line 103 of file CharScanner.hpp.

References text().

virtual void CharScanner::append char  c  )  [inline, virtual]
 

Definition at line 93 of file CharScanner.hpp.

References text().

Referenced by consume().

virtual void CharScanner::commit  )  [inline, virtual]
 

Definition at line 109 of file CharScanner.hpp.

void CharScanner::consume  )  [virtual]
 

Definition at line 62 of file CharScanner.cpp.

References append(), caseSensitive, inputState, LA(), and tab().

virtual void CharScanner::consumeUntil const BitSet set  )  [inline, virtual]
 

Consume chars until one matches the given set.

Definition at line 129 of file CharScanner.hpp.

References BitSet::member().

virtual void CharScanner::consumeUntil int  c  )  [inline, virtual]
 

Consume chars until one matches the given char.

Definition at line 117 of file CharScanner.hpp.

virtual bool CharScanner::getCaseSensitive  )  const [inline, virtual]
 

Definition at line 211 of file CharScanner.hpp.

virtual bool CharScanner::getCaseSensitiveLiterals  )  const [pure virtual]
 

Referenced by CharScannerLiteralsLess::operator()().

virtual int CharScanner::getColumn  )  const [inline, virtual]
 

Get the column the scanner currently is in (starts at 1).

Definition at line 236 of file CharScanner.hpp.

virtual bool CharScanner::getCommitToPath  )  const [inline, virtual]
 

Definition at line 257 of file CharScanner.hpp.

virtual const std ::string& CharScanner::getFilename  )  const [inline, virtual]
 

get the filename for the file currently used

Definition at line 247 of file CharScanner.hpp.

Referenced by reportError(), and reportWarning().

virtual InputBuffer& CharScanner::getInputBuffer  )  [inline, virtual]
 

Definition at line 335 of file CharScanner.hpp.

virtual LexerSharedInputState CharScanner::getInputState  )  [inline, virtual]
 

Definition at line 340 of file CharScanner.hpp.

virtual int CharScanner::getLine  )  const [inline, virtual]
 

Get the line the scanner currently is in (starts at 1).

Definition at line 224 of file CharScanner.hpp.

int CharScanner::getTabSize  )  const [inline]
 

Return the tabsize used by the scanner.

Definition at line 316 of file CharScanner.hpp.

virtual const std ::string& CharScanner::getText  )  const [inline, virtual]
 

return a copy of the current text buffer

Definition at line 268 of file CharScanner.hpp.

References text().

virtual RefToken CharScanner::getTokenObject  )  const [inline, virtual]
 

Definition at line 285 of file CharScanner.hpp.

References RefToken.

int CharScanner::LA int  i  )  [inline, virtual]
 

Definition at line 488 of file CharScanner.hpp.

References caseSensitive, inputState, LA(), and toLower().

Referenced by consume(), LA(), traceIn(), and traceOut().

virtual RefToken CharScanner::makeToken int  t  )  [inline, protected, virtual]
 

Create a new RefToken of type t.

Definition at line 444 of file CharScanner.hpp.

virtual int CharScanner::mark  )  [inline, virtual]
 

Mark the current position and return a id for it.

Definition at line 141 of file CharScanner.hpp.

virtual void CharScanner::match const std::string &  s  )  [inline, virtual]
 

See if input contains string 's' throw MismatchedCharException if not.

Definition at line 172 of file CharScanner.hpp.

References len.

virtual void CharScanner::match const BitSet b  )  [inline, virtual]
 

See if input contains element from bitset b throw MismatchedCharException if not.

Definition at line 163 of file CharScanner.hpp.

References BitSet::member().

virtual void CharScanner::match int  c  )  [inline, virtual]
 

See if input contains character 'c' throw MismatchedCharException if not.

Definition at line 152 of file CharScanner.hpp.

virtual void CharScanner::matchNot int  c  )  [inline, virtual]
 

See if input does not contain character 'c' throw MismatchedCharException if not.

Definition at line 189 of file CharScanner.hpp.

virtual void CharScanner::matchRange int  c1,
int  c2
[inline, virtual]
 

See if input contains character in range c1-c2 throw MismatchedCharException if not.

Definition at line 201 of file CharScanner.hpp.

virtual void CharScanner::newline  )  [inline, virtual]
 

Used to keep track of line breaks, needs to be called from within generated lexers when a "\n \r" is encountered.

Definition at line 293 of file CharScanner.hpp.

CharScanner& CharScanner::operator= const CharScanner other  )  [private]
 

void CharScanner::panic const std::string &  s  ) 
 

Called when a unrecoverable error is encountered.

Definition at line 96 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, and endl().

void CharScanner::panic  ) 
 

Called when a unrecoverable error is encountered.

Definition at line 90 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, and endl().

void CharScanner::reportError const std::string &  s  )  [virtual]
 

Parser error-reporting function can be overridden in subclass.

Definition at line 109 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, endl(), getFilename(), and reportError().

void CharScanner::reportError const RecognitionException ex  )  [virtual]
 

Report exception errors caught in nextToken().

Definition at line 103 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, endl(), reportError(), and RecognitionException::toString().

Referenced by reportError().

void CharScanner::reportWarning const std::string &  s  )  [virtual]
 

Parser warning-reporting function can be overridden in subclass.

Definition at line 118 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, endl(), getFilename(), and reportWarning().

Referenced by reportWarning().

virtual void CharScanner::resetText  )  [inline, virtual]
 

Definition at line 278 of file CharScanner.hpp.

References text().

virtual void CharScanner::rewind int  pos  )  [inline, virtual]
 

Rewind the scanner to a previously marked position.

Definition at line 146 of file CharScanner.hpp.

virtual void CharScanner::setCaseSensitive bool  t  )  [inline, virtual]
 

Definition at line 216 of file CharScanner.hpp.

virtual void CharScanner::setColumn int  c  )  [inline, virtual]
 

set the column number

Definition at line 241 of file CharScanner.hpp.

virtual void CharScanner::setCommitToPath bool  commit  )  [inline, virtual]
 

Definition at line 262 of file CharScanner.hpp.

virtual void CharScanner::setFilename const std::string &  f  )  [inline, virtual]
 

Set the filename the scanner is using (used in error messages).

Definition at line 252 of file CharScanner.hpp.

virtual void CharScanner::setInputState LexerSharedInputState  state  )  [inline, virtual]
 

set the input state for the lexer.

Note:
state is a reference counted object, hence no reference

Definition at line 347 of file CharScanner.hpp.

virtual void CharScanner::setLine int  l  )  [inline, virtual]
 

set the line number

Definition at line 230 of file CharScanner.hpp.

int CharScanner::setTabsize int  size  )  [inline]
 

set the tabsize. Returns the old tabsize

Definition at line 309 of file CharScanner.hpp.

References size.

virtual void CharScanner::setText const std::string &  s  )  [inline, virtual]
 

Definition at line 273 of file CharScanner.hpp.

References text().

virtual void CharScanner::setTokenObjectFactory factory_type  factory  )  [inline, virtual]
 

Set the factory for created tokens.

Definition at line 353 of file CharScanner.hpp.

Referenced by CharScanner().

virtual void CharScanner::tab  )  [inline, virtual]
 

Advance the current column number by an appropriate amount according to the tabsize.

This methad is called automatically from consume()

Definition at line 302 of file CharScanner.hpp.

Referenced by consume().

virtual int CharScanner::testLiteralsTable const std::string &  txt,
int  ttype
const [inline, virtual]
 

Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.

Definition at line 374 of file CharScanner.hpp.

References ANTLR_USE_NAMESPACE.

virtual int CharScanner::testLiteralsTable int  ttype  )  const [inline, virtual]
 

Test the token text against the literals table Override this method to perform a different literals test.

Definition at line 361 of file CharScanner.hpp.

References ANTLR_USE_NAMESPACE, and text().

virtual int CharScanner::toLower int  c  )  const [inline, virtual]
 

Override this method to get more specific case handling.

Definition at line 383 of file CharScanner.hpp.

Referenced by LA().

void CharScanner::traceIn const char *  rname  )  [virtual]
 

Definition at line 132 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, endl(), LA(), traceDepth, traceIn(), and traceIndent().

Referenced by traceIn().

void CharScanner::traceIndent  )  [virtual]
 

Methods used to change tracing behavior.

Definition at line 126 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, and traceDepth.

Referenced by traceIn(), and traceOut().

void CharScanner::traceOut const char *  rname  )  [virtual]
 

Definition at line 140 of file CharScanner.cpp.

References ANTLR_USE_NAMESPACE, endl(), LA(), traceDepth, traceIndent(), and traceOut().

Referenced by traceOut().

virtual void CharScanner::uponEOF  )  [inline, virtual]
 

This method is called by YourLexer::nextToken() when the lexer has hit EOF condition.

EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition.

uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token.

You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").

Definition at line 406 of file CharScanner.hpp.


Member Data Documentation

RefToken CharScanner::_returnToken [protected]
 

used to return tokens w/o using return val

Definition at line 430 of file CharScanner.hpp.

bool CharScanner::caseSensitive [protected]
 

Is this lexer case sensitive.

Definition at line 427 of file CharScanner.hpp.

Referenced by consume(), and LA().

bool CharScanner::commitToPath [protected]
 

Used during filter mode to indicate that path is desired.

A subsequent scan error will report an error as usual if acceptPath=true;

Definition at line 439 of file CharScanner.hpp.

const int CharScanner::EOF_CHAR = EOF [static]
 

Definition at line 150 of file CharScanner.cpp.

LexerSharedInputState CharScanner::inputState [protected]
 

Input state, gives access to input stream, shared among different lexers.

Definition at line 433 of file CharScanner.hpp.

Referenced by consume(), and LA().

std ::map< std ::string,int,CharScannerLiteralsLess> CharScanner::literals [protected]
 

Definition at line 428 of file CharScanner.hpp.

const int CharScanner::NO_CHAR = 0 [static, private]
 

Definition at line 149 of file CharScanner.cpp.

bool CharScanner::saveConsumedInput [protected]
 

flag indicating wether consume saves characters

Definition at line 425 of file CharScanner.hpp.

int CharScanner::tabsize [protected]
 

tab size the scanner uses.

Definition at line 441 of file CharScanner.hpp.

std ::string CharScanner::text [protected]
 

Text of current token.

Definition at line 423 of file CharScanner.hpp.

factory_type CharScanner::tokenFactory [protected]
 

Factory for tokens.

Definition at line 426 of file CharScanner.hpp.

int CharScanner::traceDepth [protected]
 

Definition at line 474 of file CharScanner.hpp.

Referenced by traceIn(), traceIndent(), and traceOut().


The documentation for this class was generated from the following files:
KDE Logo
This file is part of the documentation for KDevelop Version 3.0.4.
Documentation copyright © 1996-2004 the KDE developers.
Generated on Wed Oct 6 17:39:20 2004 by doxygen 1.3.7 written by Dimitri van Heesch, © 1997-2003