Class: Chainer::Functions::Activation::LeakyReLU

Inherits:
Chainer::Function show all
Defined in:
lib/chainer/functions/activation/leaky_relu.rb

Overview

Leaky rectifier unit.

Instance Attribute Summary

Attributes inherited from Chainer::Function

#inputs, #output_data, #outputs, #rank, #retain_after_backward

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Chainer::Function

#backward, #call, #forward, #retain_inputs, #retain_outputs

Constructor Details

#initialize(slope: 0.2) ⇒ LeakyReLU

Returns a new instance of LeakyReLU.



37
38
39
 
# File 'lib/chainer/functions/activation/leaky_relu.rb', line 37

def initialize(slope:0.2)
  @slope = slope
end

Class Method Details

.leaky_relu(x, slope: 0.2) ⇒ Chainer::Variable

Leaky Rectified Linear Unit function.

This function is expressed as

$$ f(x)=\max(x, ax), $$

where $a$ is a configurable slope value.

Examples:

> x = Numo::SFloat[[-1, 0], [2, -3], [-2, 1]]
> x
=> Numo::SFloat#shape=[3,2]
[[-1, 0], 
 [2, -3], 
 [-2, 1]]
> F = Chainer::Functions::Activation::LeakyReLU
> F.leaky_relu(x, slope:0.2).data
=> Numo::SFloat#shape=[3,2]
[[-0.2, 0], 
 [2, -0.6], 
 [-0.4, 1]]

Parameters:

  • x (Chainer::Variable or Numo::NArray) —

    Input variable. A $(s_1, s_2, …, s_N)$-shaped float array.

  • slope (float) (defaults to: 0.2) —

    Slope value $a$.

Returns:

  • (Chainer::Variable) —

    Output variable. A $(s_1, s_2, …, s_N)$-shaped float array.



33
34
35
 
# File 'lib/chainer/functions/activation/leaky_relu.rb', line 33

def self.leaky_relu(x, slope: 0.2)
  self.new(slope: slope).(x)
end

Instance Method Details

#backward_cpu(x, gy) ⇒ Object 



51
52
53
54
55
56
57
58
59
60
 
# File 'lib/chainer/functions/activation/leaky_relu.rb', line 51

def backward_cpu(x, gy)
  gx = gy[0].dup()
  if @slope >= 0
    y = @output_data
    gx[y[0] < 0] *= @slope
  else
    gx[x[0] < 0] *= @slope
  end
  [gx]
end

#forward_cpu(x) ⇒ Object 



41
42
43
44
45
46
47
48
49
 
# File 'lib/chainer/functions/activation/leaky_relu.rb', line 41

def forward_cpu(x)
  y = x[0].dup()
  y[x[0] < 0] *= @slope
  if @slope >= 0
    retain_inputs([])
    retain_outputs([0])
  end
  [y]
end