A simple lexer.
The procedures in this module surprisingly often suffice to parse an input stream. They either skip, or build and return tokens, according to inclusion or delimiting semantics. The list of characters to expect, include, or to break at may vary from one invocation of a function to another. This allows the functions to easily parse even context-sensitive languages.
EOF is generally frowned on, and thrown up upon if encountered. Exceptions are
mentioned specifically. The list of expected characters (characters to skip
until, or break-characters) may include an EOF "character", which is to be coded
as the symbol, *eof*
.
The input stream to parse is specified as a port, which is usually the last (and optional) argument. It defaults to the current input port if omitted.
If the parser encounters an error, it will throw an exception to the key
parser-error
. The arguments will be of the form (
port
message specialising-msg*)
.
The first argument is a port, which typically points to the offending character
or its neighborhood. You can then use port-column
and port-line
to
query the current position. message is the description of the error. Other
arguments supply more details about the problem.