Class: Chainer::FunctionNode

Inherits:
Object
  • Object
show all
Defined in:
lib/chainer/function_node.rb

Direct Known Subclasses

FunctionAdapter, Chainer::Functions::Activation::LeakyReLU, Chainer::Functions::Activation::LeakyReLUGrad, Chainer::Functions::Activation::LogSoftmax, Chainer::Functions::Activation::LogSoftmaxGrad, Chainer::Functions::Activation::ReLUGrad2, Chainer::Functions::Activation::Relu, Chainer::Functions::Activation::Sigmoid, Chainer::Functions::Activation::SigmoidGrad, Chainer::Functions::Activation::Tanh, Chainer::Functions::Activation::TanhGrad, Chainer::Functions::Array::Assign, Chainer::Functions::Array::BroadcastTo, Chainer::Functions::Array::Cast, Chainer::Functions::Array::Reshape, Chainer::Functions::Array::Rollaxis, Chainer::Functions::Array::SelectItem, Chainer::Functions::Array::Squeeze, Chainer::Functions::Array::Transpose, Chainer::Functions::Connection::Convolution2DFunction, Chainer::Functions::Connection::Convolution2DGradW, Chainer::Functions::Connection::Deconvolution2DFunction, Chainer::Functions::Connection::LinearFunction, Chainer::Functions::Loss::MeanSquaredError, Chainer::Functions::Math::Add, Chainer::Functions::Math::AddConstant, Chainer::Functions::Math::Div, Chainer::Functions::Math::Exp, Chainer::Functions::Math::Identity, Chainer::Functions::Math::Mul, Chainer::Functions::Math::MulConstant, Chainer::Functions::Math::Neg, Chainer::Functions::Math::PowVarConst, Chainer::Functions::Math::PowVarVar, Chainer::Functions::Math::Sub, Chainer::Functions::Math::Sum, Chainer::Functions::Noise::Dropout, Chainer::Functions::Noise::DropoutGrad, Chainer::Functions::Normalization::BatchNormalization, Chainer::Functions::Normalization::FixedBatchNormalization, Chainer::Functions::Pooling::AveragePooling2DGrad, Chainer::Functions::Pooling::MaxPooling2DGrad, Chainer::Functions::Pooling::MaxPooling2DWithIndexes, Chainer::Functions::Pooling::Pooling2D

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ FunctionNode

Returns a new instance of FunctionNode.



17
18
19
20
21
22
23
24
25
 
# File 'lib/chainer/function_node.rb', line 17

def initialize
  @rank = 0
  @inputs = nil
  @outputs = nil

  @retained_output_data = nil
  @input_indexes_to_retain = nil
  @output_indexes_to_retain = nil
end

Instance Attribute Details

#inputs ⇒ Object

Returns the value of attribute inputs.



15
16
17
 
# File 'lib/chainer/function_node.rb', line 15

def inputs
  @inputs
end

#outputs ⇒ Object

Returns the value of attribute outputs.



15
16
17
 
# File 'lib/chainer/function_node.rb', line 15

def outputs
  @outputs
end

#rank ⇒ Object

Returns the value of attribute rank.



15
16
17
 
# File 'lib/chainer/function_node.rb', line 15

def rank
  @rank
end

Instance Method Details

#apply(inputs) ⇒ Array<Chainer::Variable>

Computes output variables and grows the computational graph.

Basic behavior is expressed in the documentation of ‘FunctionNode`.

Parameters:

  • inputs (Chainer::Variable, Numo::NArray) —

    If the element is an Numo::NArray, it is automatically wrapped with ‘Chainer::Variable`.

Returns:

Raises:

  • (TypeError) 


53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/chainer/function_node.rb', line 53

def apply(inputs)
  input_vars = inputs.map { |x| Chainer::Variable.as_variable(x) }
  in_data = input_vars.map(&:data)
  requires_grad = input_vars.map(&:requires_grad).any?

  # Forward propagation
  @input_indexes_to_retain = nil
  @output_indexes_to_retain = nil
  outputs = forward(in_data)
  raise TypeError, "#{outputs.class} not Array" unless outputs.is_a?(Array)

  ret = outputs.map { |y| Chainer::Variable.new(y, requires_grad: requires_grad) }

  if Chainer.configuration.enable_backprop
    # Topological ordering
    @rank = input_vars.size > 0 ? input_vars.map(&:rank).max : 0

    # Add backward edges
    ret.each { |y| y.creator_node = self }
    @inputs = input_vars.map(&:node)
    # Add forward edges (must be weak references)
    @outputs = ret.map { |y| WeakRef.new(y.node) }

    unless @input_indexes_to_retain.nil?
      @input_indexes_to_retain.each do |index|
        input_vars[index].retain_data
      end
    end

    unless @output_indexes_to_retain.nil?
      retained_data = []
      @output_indexes_to_retain.each do |index|
        ret[index].retain_data
        retained_data << outputs[index]
      end
      @retained_output_data = Array(retained_data)
    end
  end

  ret
end

#backward(target_indexes, grad_outputs) ⇒ Array<Chainer::Variable>

Computes gradients w.r.t. specified inputs given output gradients.

This method is used to compute one step of the backpropagation corresponding to the forward computation of this function node. Given the gradients w.r.t. output variables, this method computes the gradients w.r.t. specified input variables. Note that this method does not need to compute any input gradients not specified by ‘target_input_indexes` It enables the function node to return the input gradients with the full computational history,

in which case it supports *differentiable backpropagation* or *higher-order differentiation*.

Parameters:

  • target_indexes (Array<Integer>) —

    Indices of the input variables w.r.t. which the gradients are required. It is guaranteed that this tuple contains at least one element.

  • grad_outputs (Array<Chainer::Variable>) —

    Gradients w.r.t. the output variables. If the gradient w.r.t. an output variable is not given, the corresponding element is ‘None`.

Returns:

  • (Array<Chainer::Variable>) —

    Array of Chainer::Variable that represent the gradients.



148
149
150
 
# File 'lib/chainer/function_node.rb', line 148

def backward(target_indexes, grad_outputs)
  [nil] * target_indexes.size
end

#backward_accumulate(target_input_indexes, grad_outputs, grad_inputs) ⇒ Array<Chainer::Variable>

Computes gradients w.r.t. specified inputs and accumulates them.

This method provides a way to fuse the backward computation and the gradient accumulations

in the case that the multiple functions are applied to the same variable.

Users have to override either of this method or ‘backward`. It is often simpler to implement `backward` and is recommended if you do not need to provide efficient gradient accumulation.

Parameters:

  • target_input_indexes (Array<Integer>) —

    Indices of the input variables w.r.t. which the gradients are required. It is guaranteed that this tuple contains at least one element.

  • grad_outputs (Array<Chainer::Variable>) —

    Gradients w.r.t. the output variables. If the gradient w.r.t. an output variable is not given, the corresponding element is ‘None`.

  • grad_inputs (Array<Chainer::Variable>) —

    Gradients w.r.t. the input variables specified by ‘target_input_indexes`. These values are computed by other computation paths. If there is no gradient value existing for the variable, the corresponding element is “None“.

Returns:

  • (Array<Chainer::Variable>) —

    Array of variables that represent the gradients w.r.t. specified input variables.



167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
 
# File 'lib/chainer/function_node.rb', line 167

def backward_accumulate(target_input_indexes, grad_outputs, grad_inputs)
  gxs = backward(target_input_indexes, grad_outputs)

  len_gxs = gxs.size
  if len_gxs == @inputs.size
    gxs = target_input_indexes.map { |i| gxs[i] }
  elsif len_gxs != target_input_indexes.size
    raise ArgumentError, "number of gradients returned by #{impl_name} (#{label}) is incorrect."
  end

  gxs.zip(grad_inputs).map do |gx, g_input|
    if g_input.nil?
      gx
    elsif gx.nil?
      g_input
    else
      gx + g_input
    end
  end
end

#forward(inputs) ⇒ Array

Computes the output arrays from the input arrays.

Parameters:

  • inputs (Array) —

    input array(s)

Returns:

  • (Array) —

    output array(s)

Raises:

  • (TypeError) 


99
100
101
102
103
 
# File 'lib/chainer/function_node.rb', line 99

def forward(inputs)
  raise TypeError, "mustt inputs > 0, inputs size is #{inputs.size}" if inputs.size.zero?
  # TODO GPU
  forward_cpu(inputs)
end

#forward_cpu(inputs) ⇒ Array<Numo::NArray>

Computes the output arrays from the input Numo::NArray.

Parameters:

  • inputs (Array<Numo::NArray>) —

    Numo::NArray objects.

Returns:

  • (Array<Numo::NArray>) —

    Array of output arrays.

Raises:

  • (NotImplementedError) 


109
110
111
 
# File 'lib/chainer/function_node.rb', line 109

def forward_cpu(inputs)
  raise NotImplementedError
end

#get_retained_inputs ⇒ Array

Returns a Array of retained input variables.

This method is used to retrieve the input variables retained in ‘forward`.

Returns:

  • (Array) —

    a Array of retained input variables.



193
194
195
 
# File 'lib/chainer/function_node.rb', line 193

def get_retained_inputs
  @input_indexes_to_retain.map { |index| @inputs[index].get_variable }
end

#get_retained_outputs ⇒ Array

Returns a Array of retained output variables.

This method is used to retrieve the input variables retained in ‘forward`.

Returns:

  • (Array) —

    a Array of retained input variables.



202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/chainer/function_node.rb', line 202

def get_retained_outputs
  ret = []
  outputs = @outputs

  new_outputs = outputs.dup
  outputs_modified = false

  @output_indexes_to_retain.zip(@retained_output_data) do |index, data|
    output =  outputs[index].__getobj__
    if output.nil?
      output_var = Chainer::Variable.new(data)
      output_var.creator_node = self
      new_outputs[index] = WeakRef.new(output_var)
      outputs_modified = true
    else
      output_var = output.get_variable
    end

    ret << output_var
  end

  if outputs_modified
    @outputs = Array(new_outputs)
  end

  ret
end

#label ⇒ Object

Short text that represents the function.

The default implementation returns its type name. Each function should override it to give more information.



31
32
33
 
# File 'lib/chainer/function_node.rb', line 31

def label
  self.class.name
end

#output_data ⇒ Object

A tuple of the retained output arrays. This property is mainly used by $Function$. Users basically do not have to use this property; use $get_retained_outputs$ instead.

Raises:

  • (RuntimeError) 


38
39
40
41
42
43
44
45
 
# File 'lib/chainer/function_node.rb', line 38

def output_data
  raise RuntimeError, 'retained output data is gone' if @retained_output_data.nil?
  out_data = [nil] * @outputs.size
  @output_indexes_to_retain.zip(@retained_output_data).each do |index, data|
    out_data[index] = data
  end
  out_data
end

#retain_inputs(indexes) ⇒ Object

Lets specified input variable nodes keep data arrays.

By calling this method from ‘forward`, the function node can specify which inputs are required for backprop. The input variables with retained arrays can be obtained by `get_retained_inputs` from `backward`.

Note that **this method must not be called from the outside of forward method.**

Parameters:

  • indexes (Integer, Array) —

    Indexes of input variables that the function does not require for backprop.



120
121
122
 
# File 'lib/chainer/function_node.rb', line 120

def retain_inputs(indexes)
  @input_indexes_to_retain = indexes
end

#retain_outputs(indexes) ⇒ Object

Lets specified output variable nodes keep data arrays.

By calling this method from ‘forward`, the function node can specify which outputs are required for backprop. If this method is not called, any output variables are not marked to keep the data array at the point of returning from `apply`. The output variables with retained arrays can be obtained by `get_retained_outputs` from `backward`. Note that **this method must not be called from the outside of forward method.**

Parameters:

  • indexes (Integer, Array) —

    Indexes of input variables that the function does not require for backprop.



131
132
133
 
# File 'lib/chainer/function_node.rb', line 131

def retain_outputs(indexes)
  @output_indexes_to_retain = indexes
end

#unchain ⇒ Object

Purges in/out nodes and this function node itself from the graph.



231
232
233
234
235
236
237
238
239
 
# File 'lib/chainer/function_node.rb', line 231

def unchain
  @outputs.each do |y|
    y_ref = y.()
    unless y_ref.nil?
      y_ref.unchain
    end
  end
  @inputs = nil
end