Class: YARD::Parser::SourceParser
- Inherits:
-
Object
- Object
- YARD::Parser::SourceParser
- Defined in:
- lib/yard/parser/source_parser.rb
Overview
Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.
Custom Parsers
SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call register_parser_type
Constant Summary
- SHEBANG_LINE =
/\A\s*#!\S+/
- ENCODING_LINE =
/\A(?:\s*#*!.*\r?\n)?\s*(?:#+|\/\*+|\/\/+).*coding\s*[:=]{1,2}\s*([a-z\d_\-]+)/i
- ENCODING_BYTE_ORDER_MARKS =
Byte order marks for various encodings
{ 'utf-8' => "\xEF\xBB\xBF", # Not yet supported #'utf-16be' => "\xFE\xFF", #'utf-16le' => "\xFF\xFE", #'utf-32be' => "\x00\x00\xFF\xFE", #'utf-32le' => "\xFF\xFE", }
Class Attribute Summary (collapse)
-
+ (Symbol) parser_type
The default parser type (defaults to :ruby).
Instance Attribute Summary (collapse)
-
- (String) contents
readonly
The contents of the file to be parsed.
-
- (String) file
readonly
The filename being parsed by the parser.
-
- (OpenStruct) globals
readonly
An open struct containing arbitrary global state shared between files and handlers.
-
- (Symbol) parser_type
The parser type associated with the parser instance.
Parser Callbacks (collapse)
-
+ (Proc) after_parse_file {|parser| ... }
Registers a callback to be called after an individual file is parsed.
-
+ (Array<Proc>) after_parse_file_callbacks
The list of callbacks to be called after parsing a file.
-
+ (Proc) after_parse_list {|files, globals| ... }
Registers a callback to be called after a list of files is parsed via parse.
-
+ (Array<Proc>) after_parse_list_callbacks
The list of callbacks to be called after parsing a list of files.
-
+ (Proc) before_parse_file {|parser| ... }
Registers a callback to be called before an individual file is parsed.
-
+ (Array<Proc>) before_parse_file_callbacks
The list of callbacks to be called before parsing a file.
-
+ (Proc) before_parse_list {|files, globals| ... }
Registers a callback to be called before a list of files is parsed via parse.
-
+ (Array<Proc>) before_parse_list_callbacks
The list of callbacks to be called before parsing a list of files.
Class Method Summary (collapse)
-
+ (void) parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
Parses a path or set of paths.
-
+ (Object) parse_string(content, ptype = parser_type)
Parses a string content.
-
+ (Symbol) parser_type_for_extension(extension)
Finds a parser type that is registered for the extension.
-
+ (void) register_parser_type(type, parser_klass, extensions = nil)
Registers a new parser type.
-
+ (Array) tokenize(content, ptype = parser_type)
Tokenizes but does not parse the block of code.
Instance Method Summary (collapse)
-
- (SourceParser) initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil)
constructor
Creates a new parser object for code parsing with a specific parser type.
-
- (Object?) parse(content = __FILE__)
The main parser method.
-
- (Array) tokenize(content)
Tokenizes but does not parse the block of code using the current #parser_type.
Constructor Details
- (SourceParser) initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil)
Creates a new parser object for code parsing with a specific parser type.
402 403 404 405 406 407 |
# File 'lib/yard/parser/source_parser.rb', line 402 def initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) @load_order_errors = load_order_errors @file = '(stdin)' @globals = globals || OpenStruct.new self.parser_type = parser_type end |
Class Attribute Details
+ (Symbol) parser_type
The default parser type (defaults to :ruby)
61 62 63 |
# File 'lib/yard/parser/source_parser.rb', line 61 def parser_type @parser_type end |
Instance Attribute Details
- (String) contents (readonly)
The contents of the file to be parsed
395 396 397 |
# File 'lib/yard/parser/source_parser.rb', line 395 def contents @contents end |
- (String) file (readonly)
The filename being parsed by the parser.
382 383 384 |
# File 'lib/yard/parser/source_parser.rb', line 382 def file @file end |
- (OpenStruct) globals (readonly)
An open struct containing arbitrary global state shared between files and handlers.
391 392 393 |
# File 'lib/yard/parser/source_parser.rb', line 391 def globals @globals end |
- (Symbol) parser_type
The parser type associated with the parser instance. This should be set by the constructor.
386 387 388 |
# File 'lib/yard/parser/source_parser.rb', line 386 def parser_type @parser_type end |
Class Method Details
+ (Proc) after_parse_file {|parser| ... }
Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called after the entire list of files is processed, see after_parse_list.
303 304 305 |
# File 'lib/yard/parser/source_parser.rb', line 303 def after_parse_file(&block) after_parse_file_callbacks << block end |
+ (Array<Proc>) after_parse_file_callbacks
The list of callbacks to be called after parsing a file. Should only be used for testing.
331 332 333 |
# File 'lib/yard/parser/source_parser.rb', line 331 def after_parse_file_callbacks @after_parse_file_callbacks ||= [] end |
+ (Proc) after_parse_list {|files, globals| ... }
Registers a callback to be called after a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
237 238 239 |
# File 'lib/yard/parser/source_parser.rb', line 237 def after_parse_list(&block) after_parse_list_callbacks << block end |
+ (Array<Proc>) after_parse_list_callbacks
The list of callbacks to be called after parsing a list of files. Should only be used for testing.
317 318 319 |
# File 'lib/yard/parser/source_parser.rb', line 317 def after_parse_list_callbacks @after_parse_list_callbacks ||= [] end |
+ (Proc) before_parse_file {|parser| ... }
Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called before the entire list of files is processed, see before_parse_list.
274 275 276 |
# File 'lib/yard/parser/source_parser.rb', line 274 def before_parse_file(&block) before_parse_file_callbacks << block end |
+ (Array<Proc>) before_parse_file_callbacks
The list of callbacks to be called before parsing a file. Should only be used for testing.
324 325 326 |
# File 'lib/yard/parser/source_parser.rb', line 324 def before_parse_file_callbacks @before_parse_file_callbacks ||= [] end |
+ (Proc) before_parse_list {|files, globals| ... }
Registers a callback to be called before a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
213 214 215 |
# File 'lib/yard/parser/source_parser.rb', line 213 def before_parse_list(&block) before_parse_list_callbacks << block end |
+ (Array<Proc>) before_parse_list_callbacks
The list of callbacks to be called before parsing a list of files. Should only be used for testing.
310 311 312 |
# File 'lib/yard/parser/source_parser.rb', line 310 def before_parse_list_callbacks @before_parse_list_callbacks ||= [] end |
+ (void) parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
This method returns an undefined value.
Parses a path or set of paths
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
# File 'lib/yard/parser/source_parser.rb', line 75 def parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser") excluded = excluded.map do |path| case path when Regexp; path else Regexp.new(path.to_s, Regexp::IGNORECASE) end end files = [paths].flatten. map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }. map {|p| p.include?("*") ? Dir[p] : p }.flatten. reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } } log.enter_level(level) do parse_in_order(*files.uniq) end end |
+ (Object) parse_string(content, ptype = parser_type)
Parses a string content
98 99 100 |
# File 'lib/yard/parser/source_parser.rb', line 98 def parse_string(content, ptype = parser_type) new(ptype).parse(StringIO.new(content)) end |
+ (Symbol) parser_type_for_extension(extension)
Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.
150 151 152 153 154 155 |
# File 'lib/yard/parser/source_parser.rb', line 150 def parser_type_for_extension(extension) type = parser_type_extensions.find do |t, exts| [exts].flatten.any? {|ext| ext === extension } end validated_parser_type(type ? type.first : :ruby) end |
+ (void) register_parser_type(type, parser_klass, extensions = nil)
This method returns an undefined value.
Registers a new parser type.
121 122 123 124 125 126 127 |
# File 'lib/yard/parser/source_parser.rb', line 121 def register_parser_type(type, parser_klass, extensions = nil) unless Base > parser_klass raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base" end parser_type_extensions[type.to_sym] = extensions if extensions parser_types[type.to_sym] = parser_klass end |
+ (Array) tokenize(content, ptype = parser_type)
Tokenizes but does not parse the block of code
107 108 109 |
# File 'lib/yard/parser/source_parser.rb', line 107 def tokenize(content, ptype = parser_type) new(ptype).tokenize(content) end |
Instance Method Details
- (Object?) parse(content = __FILE__)
The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.
414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 |
# File 'lib/yard/parser/source_parser.rb', line 414 def parse(content = __FILE__) case content when String @file = File.cleanpath(content) content = convert_encoding(File.read_binary(file)) checksum = Registry.checksum_for(content) return if Registry.checksums[file] == checksum if Registry.checksums.has_key?(file) log.info "File '#{file}' was modified, re-processing..." end Registry.checksums[@file] = checksum self.parser_type = parser_type_for_filename(file) else content = content.read if content.respond_to? :read end @contents = content @parser = parser_class.new(content, file) self.class.before_parse_file_callbacks.each do |cb| return @parser if cb.call(self) == false end @parser.parse post_process self.class.after_parse_file_callbacks.each do |cb| cb.call(self) end @parser rescue ArgumentError, NotImplementedError => e log.warn("Cannot parse `#{file}': #{e.}") log.backtrace(e) if log.level >= Logger::DEBUG rescue ParserSyntaxError => e log.warn(e..capitalize) log.backtrace(e) if log.level >= Logger::DEBUG end |
- (Array) tokenize(content)
Tokenizes but does not parse the block of code using the current #parser_type
458 459 460 461 |
# File 'lib/yard/parser/source_parser.rb', line 458 def tokenize(content) @parser = parser_class.new(content, file) @parser.tokenize end |