Class: YARD::Parser::Ruby::TokenResolver

Inherits:
Object
  • Object
show all
Includes:
Enumerable, CodeObjects::NamespaceMapper
Defined in:
lib/yard/parser/ruby/token_resolver.rb

Overview

Supports #each enumeration over a source’s tokens, yielding the token and a possible CodeObjects::Base associated with the constant or identifier token.

Class Method Summary collapse

Instance Method Summary collapse

Methods included from CodeObjects::NamespaceMapper

#clear_separators, #default_separator, on_invalidate, #register_separator, #separators, #separators_for_type, #separators_match, #types_for_separator, #unregister_separator_by_type

Constructor Details

#initialize(source, namespace = Registry.root) ⇒ TokenResolver

Creates a token resolver for given source.

Parameters:

  • source (String) —

    the source code to tokenize

  • namespace (CodeObjects::Base) (defaults to: Registry.root) —

    the object/namespace to resolve from

Raises:

[View source]

16
17
18
19
20
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 16

def initialize(source, namespace = Registry.root)
  @tokens = RubyParser.parse(source, '(tokenize)').tokens
  raise ParserSyntaxError if @tokens.empty? && !source.empty?
  @default_namespace = namespace
end

Class Method Details

.state_attr(*attrs) ⇒ Object

[View source]

92
93
94
95
96
97
98
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 92

def self.state_attr(*attrs)
  attrs.each do |attr|
    define_method(attr) { @states.last[attr.to_sym] }
    define_method("#{attr}=") {|v| @states.last[attr.to_sym] = v }
    protected attr, :"#{attr}="
  end
end

Instance Method Details

#each {|token, object| ... } ⇒ Object

Iterates over each token, yielding the token and a possible code object that is associated with the token.

Examples:

Yielding code objects

r = TokenResolver.new("A::B::C")
r.each do |tok, obj|
  if obj
    puts "#{tok[0]} -> #{obj.path.inspect}"
  else
    puts "No object: #{tok.inspect}"
  end
end

# Prints:
# :const -> "A"
# No object: [:op, "::"]
# :const -> "A::B"
# No object: [:op, "::"]
# :const -> "A::B::C"

Yield Parameters:

  • token (Array(Symbol,String,Array(Integer,Integer))) —

    the current token object being iterated

  • object (CodeObjects::Base, nil) —

    the fully qualified code object associated with the current token, or nil if there is no object for the yielded token.

[View source]

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 46

def each
  @states = []
  push_state
  @tokens.each do |token|
    yield_obj = false

    if skip_group && [:const, :ident, :op, :period].include?(token[0])
      yield token, nil
      next
    else
      self.skip_group = false
    end

    case token[0]
    when :const
      lookup(token[0], token[1])
      yield_obj = true
      self.last_sep = nil
    when :ident
      lookup(token[0], token[1])
      yield_obj = true
      self.last_sep = nil
    when :op, :period
      self.last_sep = token[1]
      unless CodeObjects.types_for_separator(token[1])
        self.object = nil
        self.last_sep = nil
      end
    when :lparen
      push_state
    when :rparen
      pop_state
    else
      self.object = nil
    end

    yield token, (yield_obj ? object : nil)

    if next_object
      self.object = next_object
      self.next_object = nil
    end
    self.skip_group = true if yield_obj && object.nil?
  end
end