Class: Chainer::Functions::Activation::LogSoftmax

Inherits:
Chainer::Function show all
Defined in:
lib/chainer/functions/activation/log_softmax.rb

Overview

Log-softmax activation function.

Instance Attribute Summary

Attributes inherited from Chainer::Function

#inputs, #output_data, #outputs, #rank, #retain_after_backward

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Chainer::Function

#call, #forward_cpu, #initialize, #retain_inputs, #retain_outputs

Constructor Details

This class inherits a constructor from Chainer::Function

Class Method Details

.log_softmax(x) ⇒ Chainer::Variable

Note:

log(softmax(x)) may cause underflow when x is too small, because softmax(x) may returns 0. log_softmax method is more stable.

Channel-wise log-softmax function.

This function computes its logarithm of softmax along the second axis. Let $c = (c_1, c_2, \dots, c_D)$ be the slice of x along with the second axis. For each slice $c$, it computes the logarithm of the function $f(c)$ defined as

$$ f(c) = { \exp(c) \over \sum_{ d } \exp(c_d) }. $$

This method is theoretically equivalent to log(softmax(x)) but is more stable.

Examples:

> x = Numo::SFloat[[0, 1, 2], [0, 2, 4]]
=> Numo::SFloat#shape=[2,3]
[[0, 1, 2],
 [0, 2, 4]]
> F = Chainer::Functions::Activation::LogSoftmax
> F.log_softmax(x).data
=> Numo::SFloat#shape=[2,3]
[[-2.40761, -1.40761, -0.407606],
 [-4.14293, -2.14293, -0.142932]]

(T.B.I : F.log, F.softmax)

> F.log_softmax(x).data.nearly_eq(F.log(F.softmax(x)).data).all?)
=> true

Parameters:

  • x (Chainer::Variable or Numo::NArray) —

    Input variable. A $n$-dimensional ($n \geq 2$) float array.

Returns:

  • (Chainer::Variable) —

    Output variable. A $n$-dimensional ($n \geq 2$) float array, which is the same shape with x.

See Also:

  • Softmax


58
59
60
 
# File 'lib/chainer/functions/activation/log_softmax.rb', line 58

def self.log_softmax(x)
  self.new.(x)
end

Instance Method Details

#backward(x, gy) ⇒ Object 



71
72
73
74
75
 
# File 'lib/chainer/functions/activation/log_softmax.rb', line 71

def backward(x, gy)
  y = @output_data[0]
  gx = gy[0] - Numo::NMath.exp(y) * gy[0].sum(axis: 1, keepdims: true)
  [gx]
end

#forward(xs) ⇒ Object 



62
63
64
65
66
67
68
69
 
# File 'lib/chainer/functions/activation/log_softmax.rb', line 62

def forward(xs)
  y = Chainer::Functions::Activation._log_softmax(xs[0])
  @x_shape = xs[0].shape
  @x_dtype = xs[0].class
  retain_inputs([])
  retain_outputs([0])
  [y]
end