Class: Spacy::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby-spacy.rb

Overview

See also spaCy Python API document for [‘Token`](spacy.io/api/token).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(py_token) ⇒ Token

It is recommended to use Doc#tokens or Span#tokens methods to create tokens. There is no way to generate a token from scratch but relying on a pre-exising Python ‘Token` object.

Parameters:

  • py_token (Object) —

    Python ‘Token` object



812
813
814
815
 
# File 'lib/ruby-spacy.rb', line 812

def initialize(py_token)
  @py_token = py_token
  @text = @py_token.text
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(name, *args) ⇒ Object

Methods defined in Python but not wrapped in ruby-spacy can be called by this dynamic method handling mechanism.



958
959
960
 
# File 'lib/ruby-spacy.rb', line 958

def method_missing(name, *args)
  @py_token.send(name, *args)
end

Instance Attribute Details

#py_token ⇒ Object (readonly)

Returns a Python ‘Token` instance accessible via `PyCall`.

Returns:

  • (Object) —

    a Python ‘Token` instance accessible via `PyCall`



804
805
806
 
# File 'lib/ruby-spacy.rb', line 804

def py_token
  @py_token
end

#text ⇒ String (readonly)

Returns a string representing the token.

Returns:

  • (String) —

    a string representing the token



807
808
809
 
# File 'lib/ruby-spacy.rb', line 807

def text
  @text
end

Instance Method Details

#ancestors ⇒ Array<Token>

Returns the token’s ancestors.

Returns:

  • (Array<Token>) —

    an array of tokens



835
836
837
838
839
840
841
 
# File 'lib/ruby-spacy.rb', line 835

def ancestors
  ancestor_array = []
  PyCall::List.call(@py_token.ancestors).each do |ancestor|
    ancestor_array << Token.new(ancestor)
  end
  ancestor_array
end

#children ⇒ Array<Token>

Returns a sequence of the token’s immediate syntactic children.

Returns:

  • (Array<Token>) —

    an array of tokens



845
846
847
848
849
850
851
 
# File 'lib/ruby-spacy.rb', line 845

def children
  child_array = []
  PyCall::List.call(@py_token.children).each do |child|
    child_array << Token.new(child)
  end
  child_array
end

#dep ⇒ String

Returns the dependency relation by calling ‘dep_’ of ‘@py_token` object

Returns:

  • (String) 


929
930
931
 
# File 'lib/ruby-spacy.rb', line 929

def dep
  @py_token.dep_
end

#ent_type ⇒ String

Returns the named entity type by calling ‘ent_type_’ of ‘@py_token` object

Returns:

  • (String) 


947
948
949
 
# File 'lib/ruby-spacy.rb', line 947

def ent_type
  @py_token.ent_type_
end

#head ⇒ Token

Returns the head token

Returns:



819
820
821
 
# File 'lib/ruby-spacy.rb', line 819

def head
  Token.new(@py_token.head)
end

#lang ⇒ String

Returns the language by calling ‘lang_’ of ‘@py_token` object

Returns:

  • (String) 


935
936
937
 
# File 'lib/ruby-spacy.rb', line 935

def lang
  @py_token.lang_
end

#lefts ⇒ Array<Token>

The leftward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



855
856
857
858
859
860
861
 
# File 'lib/ruby-spacy.rb', line 855

def lefts
  token_array = []
  PyCall::List.call(@py_token.lefts).each do |token|
    token_array << Token.new(token)
  end
  token_array
end

#lemma ⇒ String

Returns the lemma by calling ‘lemma_’ of ‘@py_token` object

Returns:

  • (String) 


899
900
901
 
# File 'lib/ruby-spacy.rb', line 899

def lemma
  @py_token.lemma_
end

#lexeme ⇒ Lexeme

Returns a lexeme object

Returns:



953
954
955
 
# File 'lib/ruby-spacy.rb', line 953

def lexeme
  Lexeme.new(@py_token.lex)
end

#lower ⇒ String

Returns the lowercase form by calling ‘lower_’ of ‘@py_token` object

Returns:

  • (String) 


905
906
907
 
# File 'lib/ruby-spacy.rb', line 905

def lower
  @py_token.lower_
end

#morphology(hash: true) ⇒ Hash, String

Returns a hash or string of morphological information

Parameters:

  • hash (Boolean) (defaults to: true) —

    if true, a hash will be returned instead of a string

Returns:

  • (Hash, String) 


882
883
884
885
886
887
888
889
890
891
892
893
894
895
 
# File 'lib/ruby-spacy.rb', line 882

def morphology(hash: true)
  if @py_token.has_morph
    morph_analysis = @py_token.morph
    if hash
      morph_analysis.to_dict
    else
      morph_analysis.to_s
    end
  elsif hash
    {}
  else
    ""
  end
end

#pos ⇒ String

Returns the pos by calling ‘pos_’ of ‘@py_token` object

Returns:

  • (String) 


917
918
919
 
# File 'lib/ruby-spacy.rb', line 917

def pos
  @py_token.pos_
end

#respond_to_missing?(sym, *args) ⇒ Boolean

Returns:

  • (Boolean) 


962
963
964
 
# File 'lib/ruby-spacy.rb', line 962

def respond_to_missing?(sym, *args)
  sym ? true : super
end

#rights ⇒ Array<Token>

The rightward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



865
866
867
868
869
870
871
 
# File 'lib/ruby-spacy.rb', line 865

def rights
  token_array = []
  PyCall::List.call(@py_token.rights).each do |token|
    token_array << Token.new(token)
  end
  token_array
end

#shape ⇒ String

Returns the shape (e.g. “Xxxxx”) by calling ‘shape_’ of ‘@py_token` object

Returns:

  • (String) 


911
912
913
 
# File 'lib/ruby-spacy.rb', line 911

def shape
  @py_token.shape_
end

#subtree ⇒ Array<Token>

Returns the token in question and the tokens that descend from it.

Returns:

  • (Array<Token>) —

    an array of tokens



825
826
827
828
829
830
831
 
# File 'lib/ruby-spacy.rb', line 825

def subtree
  descendant_array = []
  PyCall::List.call(@py_token.subtree).each do |descendant|
    descendant_array << Token.new(descendant)
  end
  descendant_array
end

#tag ⇒ String

Returns the fine-grained pos by calling ‘tag_’ of ‘@py_token` object

Returns:

  • (String) 


923
924
925
 
# File 'lib/ruby-spacy.rb', line 923

def tag
  @py_token.tag_
end

#to_s ⇒ String

String representation of the token.

Returns:

  • (String) 


875
876
877
 
# File 'lib/ruby-spacy.rb', line 875

def to_s
  @text
end

#whitespace ⇒ String

Returns the trailing space character if present by calling ‘whitespace_’ of ‘@py_token` object

Returns:

  • (String) 


941
942
943
 
# File 'lib/ruby-spacy.rb', line 941

def whitespace
  @py_token.whitespace_
end