Class: Spacy::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby-spacy.rb

Overview

See also spaCy Python API document for Token.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(py_token) ⇒ Token

It is recommended to use Doc#tokens or Span#tokens methods to create tokens. There is no way to generate a token from scratch but relying on a pre-exising Python Token object.

Parameters:

  • py_token (Object) —

    Python Token object



1141
1142
1143
1144
 
# File 'lib/ruby-spacy.rb', line 1141

def initialize(py_token)
  @py_token = py_token
  @text = @py_token.text
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(name, *args) ⇒ Object

Methods defined in Python but not wrapped in ruby-spacy can be called by this dynamic method handling mechanism.



1273
1274
1275
 
# File 'lib/ruby-spacy.rb', line 1273

def method_missing(name, *args)
  Spacy.safe_py_send(@py_token, name, args)
end

Instance Attribute Details

#py_token ⇒ Object (readonly)

Returns a Python Token instance accessible via PyCall.

Returns:

  • (Object) —

    a Python Token instance accessible via PyCall



1133
1134
1135
 
# File 'lib/ruby-spacy.rb', line 1133

def py_token
  @py_token
end

#text ⇒ String (readonly)

Returns a string representing the token.

Returns:

  • (String) —

    a string representing the token



1136
1137
1138
 
# File 'lib/ruby-spacy.rb', line 1136

def text
  @text
end

Instance Method Details

#ancestors ⇒ Array<Token>

Returns the token's ancestors.

Returns:

  • (Array<Token>) —

    an array of tokens



1166
1167
1168
 
# File 'lib/ruby-spacy.rb', line 1166

def ancestors
  PyCall::List.call(@py_token.ancestors).map { |ancestor| Token.new(ancestor) }
end

#children ⇒ Array<Token>

Returns a sequence of the token's immediate syntactic children.

Returns:

  • (Array<Token>) —

    an array of tokens



1172
1173
1174
 
# File 'lib/ruby-spacy.rb', line 1172

def children
  PyCall::List.call(@py_token.children).map { |child| Token.new(child) }
end

#dep ⇒ String

Returns the dependency relation by calling dep_' of @py_token` object

Returns:

  • (String) 


1244
1245
1246
 
# File 'lib/ruby-spacy.rb', line 1244

def dep
  @py_token.dep_
end

#ent_type ⇒ String

Returns the named entity type by calling ent_type_' of @py_token` object

Returns:

  • (String) 


1262
1263
1264
 
# File 'lib/ruby-spacy.rb', line 1262

def ent_type
  @py_token.ent_type_
end

#head ⇒ Token

Returns the head token

Returns:



1154
1155
1156
 
# File 'lib/ruby-spacy.rb', line 1154

def head
  Token.new(@py_token.head)
end

#idx ⇒ Integer

Returns the character offset of the token within the parent document.

Returns:

  • (Integer) 


1148
1149
1150
 
# File 'lib/ruby-spacy.rb', line 1148

def idx
  @py_token.idx
end

#instance_variables_to_inspect ⇒ Object 



1281
1282
1283
 
# File 'lib/ruby-spacy.rb', line 1281

def instance_variables_to_inspect
  [:@text]
end

#lang ⇒ String

Returns the language by calling lang_' of @py_token` object

Returns:

  • (String) 


1250
1251
1252
 
# File 'lib/ruby-spacy.rb', line 1250

def lang
  @py_token.lang_
end

#lefts ⇒ Array<Token>

The leftward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



1178
1179
1180
 
# File 'lib/ruby-spacy.rb', line 1178

def lefts
  PyCall::List.call(@py_token.lefts).map { |token| Token.new(token) }
end

#lemma ⇒ String

Returns the lemma by calling lemma_' of @py_token` object

Returns:

  • (String) 


1214
1215
1216
 
# File 'lib/ruby-spacy.rb', line 1214

def lemma
  @py_token.lemma_
end

#lexeme ⇒ Lexeme

Returns a lexeme object

Returns:



1268
1269
1270
 
# File 'lib/ruby-spacy.rb', line 1268

def lexeme
  Lexeme.new(@py_token.lex)
end

#lower ⇒ String

Returns the lowercase form by calling lower_' of @py_token` object

Returns:

  • (String) 


1220
1221
1222
 
# File 'lib/ruby-spacy.rb', line 1220

def lower
  @py_token.lower_
end

#morphology(hash: true) ⇒ Hash, String

Returns a hash or string of morphological information

Parameters:

  • hash (Boolean) (defaults to: true) —

    if true, a hash will be returned instead of a string

Returns:

  • (Hash, String) 


1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
 
# File 'lib/ruby-spacy.rb', line 1197

def morphology(hash: true)
  if @py_token.has_morph
    morph_analysis = @py_token.morph
    if hash
      morph_analysis.to_dict
    else
      morph_analysis.to_s
    end
  elsif hash
    {}
  else
    ""
  end
end

#pos ⇒ String

Returns the pos by calling pos_' of @py_token` object

Returns:

  • (String) 


1232
1233
1234
 
# File 'lib/ruby-spacy.rb', line 1232

def pos
  @py_token.pos_
end

#respond_to_missing?(sym, include_private = false) ⇒ Boolean

Returns:

  • (Boolean) 


1277
1278
1279
 
# File 'lib/ruby-spacy.rb', line 1277

def respond_to_missing?(sym, include_private = false)
  Spacy.py_hasattr?(@py_token, sym) || super
end

#rights ⇒ Array<Token>

The rightward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



1184
1185
1186
 
# File 'lib/ruby-spacy.rb', line 1184

def rights
  PyCall::List.call(@py_token.rights).map { |token| Token.new(token) }
end

#shape ⇒ String

Returns the shape (e.g. "Xxxxx") by calling shape_' of @py_token` object

Returns:

  • (String) 


1226
1227
1228
 
# File 'lib/ruby-spacy.rb', line 1226

def shape
  @py_token.shape_
end

#subtree ⇒ Array<Token>

Returns the token in question and the tokens that descend from it.

Returns:

  • (Array<Token>) —

    an array of tokens



1160
1161
1162
 
# File 'lib/ruby-spacy.rb', line 1160

def subtree
  PyCall::List.call(@py_token.subtree).map { |descendant| Token.new(descendant) }
end

#tag ⇒ String

Returns the fine-grained pos by calling tag_' of @py_token` object

Returns:

  • (String) 


1238
1239
1240
 
# File 'lib/ruby-spacy.rb', line 1238

def tag
  @py_token.tag_
end

#to_s ⇒ String

String representation of the token.

Returns:

  • (String) 


1190
1191
1192
 
# File 'lib/ruby-spacy.rb', line 1190

def to_s
  @text
end

#whitespace ⇒ String

Returns the trailing space character if present by calling whitespace_' of @py_token` object

Returns:

  • (String) 


1256
1257
1258
 
# File 'lib/ruby-spacy.rb', line 1256

def whitespace
  @py_token.whitespace_
end