c_preproc¶
C/C++ preprocessor for finding dependencies
Reasons for using the Waf preprocessor by default
- Some c/c++ extensions (Qt) require a custom preprocessor for obtaining the dependencies (.moc files)
- Not all compilers provide .d files for obtaining the dependencies (portability)
- A naive file scanner will not catch the constructs such as “#include foo()”
- A naive file scanner will catch unnecessary dependencies (change an unused header -> recompile everything)
Regarding the speed concerns:
- the preprocessing is performed only when files must be compiled
- the macros are evaluated only for #if/#elif/#include
- system headers are not scanned by default
Now if you do not want the Waf preprocessor, the tool +gccdeps* uses the .d files produced during the compilation to track the dependencies (useful when used with the boost libraries). It only works with gcc >= 4.4 though.
A dumb preprocessor is also available in the tool c_dumbpreproc
-
waflib.Tools.c_preproc.
POPFILE
= '-'¶ Constant representing a special token used in
waflib.Tools.c_preproc.c_parser.start()
iteration to switch to a header read previously
-
waflib.Tools.c_preproc.
recursion_limit
= 150¶ Limit on the amount of files to read in the dependency scanner
-
waflib.Tools.c_preproc.
go_absolute
= False¶ Set to True to track headers on files in /usr/include, else absolute paths are ignored (but it becomes very slow)
-
waflib.Tools.c_preproc.
use_trigraphs
= 0¶ Apply trigraph rules (False by default)
-
waflib.Tools.c_preproc.
strict_quotes
= 0¶ Reserve the “#include <>” quotes for system includes (do not search for those includes). False by default.
-
waflib.Tools.c_preproc.
g_optrans
= {'and': '&&', 'and_eq': '&=', 'compl': '~', 'bitand': '&', 'not': '!', 'not_eq': '!', 'bitor': '|', 'xor': '^', 'xor_eq': '^=', 'or_eq': '|=', 'or': '||'}¶ Operators such as and/or/xor for c++. Set an empty dict to disable.
-
waflib.Tools.c_preproc.
re_lines
= <_sre.SRE_Pattern object at 0x93540c0>¶ Match #include lines
-
waflib.Tools.c_preproc.
re_mac
= <_sre.SRE_Pattern object>¶ Match macro definitions
-
waflib.Tools.c_preproc.
re_fun
= <_sre.SRE_Pattern object>¶ Match macro functions
-
waflib.Tools.c_preproc.
re_pragma_once
= <_sre.SRE_Pattern object>¶ Match #pragma once statements
-
waflib.Tools.c_preproc.
re_nl
= <_sre.SRE_Pattern object>¶ Match newlines
-
waflib.Tools.c_preproc.
re_cpp
= <_sre.SRE_Pattern object>¶ Filter C/C++ comments
-
waflib.Tools.c_preproc.
trig_def
= [('??=', '#'), ('??-', '~'), ('??/', '\\'), ('??!', '|'), ("??'", '^'), ('??(', '['), ('??)', ']'), ('??<', '{'), ('??>', '}')]¶ Trigraph definitions
-
waflib.Tools.c_preproc.
chr_esc
= {'a': 7, '0': 0, 'r': 13, 'b': 8, 't': 9, "'": 39, 'f': 11, 'v': 12, '\\': 92, 'n': 10}¶ Escape characters
-
waflib.Tools.c_preproc.
NUM
= 'i'¶ Number token
-
waflib.Tools.c_preproc.
OP
= 'O'¶ Operator token
-
waflib.Tools.c_preproc.
IDENT
= 'T'¶ Identifier token
-
waflib.Tools.c_preproc.
STR
= 's'¶ String token
-
waflib.Tools.c_preproc.
CHAR
= 'c'¶ Character token
-
waflib.Tools.c_preproc.
tok_types
= ['i', 's', 'T', 'O']¶ Token types
-
waflib.Tools.c_preproc.
exp_types
= ["0[xX](?P<hex>[a-fA-F0-9]+)(?P<qual1>[uUlL]*)|L*?'(?P<char>(\\\\.|[^\\\\'])+)'|(?P<n1>\\d+)[Ee](?P<exp0>[+-]*?\\d+)(?P<float0>[fFlL]*)|(?P<n2>\\d*\\.\\d+)([Ee](?P<exp1>[+-]*?\\d+))?(?P<float1>[fFlL]*)|(?P<n4>\\d+\\.\\d*)([Ee](?P<exp2>[+-]*?\\d+))?(?P<float2>[fFlL]*)|(?P<oct>0*)(?P<n0>\\d+)(?P<qual2>[uUlL]*)", 'L?"([^"\\\\]|\\\\.)*"', '[a-zA-Z_]\\w*', '%:%:|<<=|>>=|\\.\\.\\.|<<|<%|<:|<=|>>|>=|\\+\\+|\\+=|--|->|-=|\\*=|/=|%:|%=|%>|==|&&|&=|\\|\\||\\|=|\\^=|:>|!=|##|[\\(\\)\\{\\}\\[\\]<>\\?\\|\\^\\*\\+&=:!#;,%/\\-\\?\\~\\.]']¶ Expression types
-
waflib.Tools.c_preproc.
re_clexer
= <_sre.SRE_Pattern object at 0x93519f0>¶ Match expressions into tokens
-
waflib.Tools.c_preproc.
accepted
= 'a'¶ Parser state is accepted
-
waflib.Tools.c_preproc.
ignored
= 'i'¶ Parser state is ignored, for example preprocessor lines in an #if 0 block
-
waflib.Tools.c_preproc.
undefined
= 'u'¶ Parser state is undefined at the moment
-
waflib.Tools.c_preproc.
skipped
= 's'¶ Parser state is skipped, for example preprocessor lines in a #elif 0 block
-
waflib.Tools.c_preproc.
repl
(m)[source]¶ Replace function used with
waflib.Tools.c_preproc.re_cpp
-
waflib.Tools.c_preproc.
filter_comments
(filename)[source]¶ Filter the comments from a c/h file, and return the preprocessor lines. The regexps
waflib.Tools.c_preproc.re_cpp
,waflib.Tools.c_preproc.re_nl
andwaflib.Tools.c_preproc.re_lines
are used internally.Returns: the preprocessor directives as a list of (keyword, line) Return type: a list of string pairs
-
waflib.Tools.c_preproc.
prec
= {'||': 6, '<=': 3, '%': 0, '>=': 3, '>>': 2, '&': 5, '==': 4, '+': 1, '*': 0, '-': 1, ',': 7, '/': 0, '<': 3, '&&': 6, '|': 5, '^': 5, '!=': 4, '<<': 2, '>': 3}¶ Operator precendence rules required for parsing expressions of the form:
#if 1 && 2 != 0
-
waflib.Tools.c_preproc.
trimquotes
(s)[source]¶ Remove the single quotes around an expression:
trimquotes("'test'") == "test"
Parameters: s (string) – expression to transform Return type: string
-
waflib.Tools.c_preproc.
reduce_nums
(val_1, val_2, val_op)[source]¶ Apply arithmetic rules to compute a result
Parameters: - val1 (int or string) – input parameter
- val2 (int or string) – input parameter
- val_op (string) – C operator in +, /, -, etc
Return type: int
-
waflib.Tools.c_preproc.
get_num
(lst)[source]¶ Try to obtain a number from a list of tokens. The token types are defined in
waflib.Tools.ccroot.tok_types
.Parameters: lst (list of tuple (tokentype, value)) – list of preprocessor tokens Returns: a pair containing the number and the rest of the list Return type: tuple(value, list)
-
waflib.Tools.c_preproc.
get_term
(lst)[source]¶ Evaluate an expression recursively, for example:
1+1+1 -> 2+1 -> 3
Parameters: lst (list of tuple(token, value)) – list of tokens Returns: the value and the remaining tokens Return type: value, list
-
waflib.Tools.c_preproc.
reduce_eval
(lst)[source]¶ Take a list of tokens and output true or false for #if/#elif conditions.
Parameters: lst (list of tuple(token, value)) – a list of tokens Returns: a token Return type: tuple(NUM, int)
-
waflib.Tools.c_preproc.
stringize
(lst)[source]¶ Merge a list of tokens into a string
Parameters: lst (list of tuple(token, value)) – a list of tokens Return type: string
-
waflib.Tools.c_preproc.
error
(*k, **kw)[source]¶ Wrap logging.errors, display the origin of the message when ‘-vv’ is set
-
waflib.Tools.c_preproc.
debug
(*k, **kw)[source]¶ Wrap logging.debug, the output is filtered for performance reasons
-
waflib.Tools.c_preproc.
paste_tokens
(t1, t2)[source]¶ Token pasting works between identifiers, particular operators, and identifiers and numbers:
a ## b -> ab > ## = -> >= a ## 2 -> a2
Parameters: - t1 (tuple(type, value)) – token
- t2 (tuple(type, value)) – token
-
waflib.Tools.c_preproc.
reduce_tokens
(lst, defs, ban=[])[source]¶ Replace the tokens in lst, using the macros provided in defs, and a list of macros that cannot be re-applied
Parameters: - lst (list of tuple(token, value)) – list of tokens
- defs (dict) – macro definitions
- ban (list of string) – macros that cannot be substituted (recursion is not allowed)
Returns: the new list of tokens
Return type: value, list
-
waflib.Tools.c_preproc.
eval_macro
(lst, defs)[source]¶ Reduce the tokens by
waflib.Tools.c_preproc.reduce_tokens()
and try to return a 0/1 result bywaflib.Tools.c_preproc.reduce_eval()
.Parameters: - lst (list of tuple(token, value)) – list of tokens
- defs (dict) – macro definitions
Return type: int
-
waflib.Tools.c_preproc.
extract_macro
(txt)[source]¶ - Process a macro definition of the form::
- #define f(x, y) x * y
into a function or a simple macro without arguments
Parameters: txt (string) – expression to exact a macro definition from Returns: a tuple containing the name, the list of arguments and the replacement Return type: tuple(string, [list, list])
-
waflib.Tools.c_preproc.
extract_include
(txt, defs)[source]¶ Process a line in the form:
#include foo
Parameters: - txt (string) – include line to process
- defs (dict) – macro definitions
Returns: the file name
Return type: string
-
waflib.Tools.c_preproc.
parse_char
(txt)[source]¶ Parse a c character
Parameters: txt (string) – character to parse Returns: a character literal Return type: string
-
waflib.Tools.c_preproc.
tokenize
(s)[source]¶ Convert a string into a list of tokens (shlex.split does not apply to c/c++/d)
Parameters: s (string) – input to tokenize Returns: a list of tokens Return type: list of tuple(token, value)
-
class
waflib.Tools.c_preproc.
c_parser
(nodepaths=None, defines=None)[source]¶ Bases:
object
Used by
waflib.Tools.c_preproc.scan()
to parse c/h files. Note that by default, only project headers are parsed.-
__doc__
= '\n\tUsed by :py:func:`waflib.Tools.c_preproc.scan` to parse c/h files. Note that by default,\n\tonly project headers are parsed.\n\t'¶
-
__module__
= 'waflib.Tools.c_preproc'¶
-
lines
= None¶ list of lines read
-
nodepaths
= None¶ Include paths
-
nodes
= None¶ List of
waflib.Node.Node
found so far
-
names
= None¶ List of file names that could not be matched by any file
-
curfile
= None¶ Current file
-
ban_includes
= None¶ Includes that must not be read (#pragma once)
-
cached_find_resource
(node, filename)[source]¶ Find a file from the input directory
Parameters: - node (
waflib.Node.Node
) – directory - filename (string) – header to find
Returns: the node if found, or None
Return type: - node (
-
tryfind
(filename)[source]¶ Try to obtain a node from the filename based from the include paths. Will add the node found to
waflib.Tools.c_preproc.c_parser.nodes
or the file name towaflib.Tools.c_preproc.c_parser.names
if no corresponding file is found. Called bywaflib.Tools.c_preproc.c_parser.start
.Parameters: filename (string) – header to find Returns: the node if found Return type: waflib.Node.Node
-
addlines
(node)[source]¶ Add the lines from a header in the list of preprocessor lines to parse
Parameters: node ( waflib.Node.Node
) – header
-
start
(node, env)[source]¶ Preprocess a source file to obtain the dependencies, which are accumulated to
waflib.Tools.c_preproc.c_parser.nodes
andwaflib.Tools.c_preproc.c_parser.names
.Parameters: - node (
waflib.Node.Node
) – source file - env (
waflib.ConfigSet.ConfigSet
) – config set containing additional defines to take into account
- node (
-
-
waflib.Tools.c_preproc.
scan
(task)[source]¶ Get the dependencies using a c/c++ preprocessor, this is required for finding dependencies of the kind:
#include some_macro()
This function is bound as a task method on
waflib.Tools.c.c
andwaflib.Tools.cxx.cxx
for example