Class: TransitionCache

Inherits:
Object
  • Object
show all
Defined in:
lib/repositories/transition_cache.rb

Overview

This can follow a transition graph around and keeps a cache of its path. Reinforcement learning and evolutionary algorithms can use this data to guide the next transitions attempted as well as apply a value function on the data.

The current_state needs an Enumerable interface to work well with others. The default is set to a MonitoredArray with a SimpleArrayEviction policy.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(opts = {}, &block) ⇒ TransitionCache

Returns a new instance of TransitionCache.



20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/repositories/transition_cache.rb', line 20

def initialize(opts={}, &block)

  @graph = opts[:graph]
  @graph ||= opts[:transition_graph]
  
  self.eviction_policy = opts[:eviction_policy]
  self.repository = opts[:repository]
  self.current_state = opts[:current_state]

  self.instance_eval(&block) if block

  # Notice, this is a useful graph, it will need to be worked on before
  # anything useful can happen here. 
  self.graph ||= TransitionGraph.new

  self.current_state ||= self.graph.states.first
  self.eviction_policy ||= SimpleArrayEviction.new(10)
  self.repository ||= MonitoredArray
end

Instance Attribute Details

#current_state ⇒ Object

The state we're exploring



41
42
43
 
# File 'lib/repositories/transition_cache.rb', line 41

def current_state
  @current_state
end

#eviction_policy ⇒ Object

The policy for limiting the size of the state history repository. The deault is a SimpleArrayEviction set at 10 records.



86
87
88
 
# File 'lib/repositories/transition_cache.rb', line 86

def eviction_policy
  @eviction_policy
end

#graph ⇒ Object

The transition graph, or state machine, that constrains the actions available.



18
19
20
 
# File 'lib/repositories/transition_cache.rb', line 18

def graph
  @graph
end

#repository ⇒ Object

Returns a cache of past states. The default repository uses an ArrayCache.



94
95
96
 
# File 'lib/repositories/transition_cache.rb', line 94

def repository
  @repository
end

#reward ⇒ Object Also known as: reward!

A cummulative count of all rewards encountered.



74
75
76
 
# File 'lib/repositories/transition_cache.rb', line 74

def reward
  @reward ||= 0
end

Instance Method Details

#advance(name = nil, &block) ⇒ Object Also known as: advance!

Makes a choice and advances the state through a transition. This allows a block to evaluate, passing the block all available actions. The block should return a name or a transition to advance. If a block is given, the block MUST return a valid transition to avance. I.e., if you go to the trouble of choosing between the available actions and you don't choose one, then you can't advance anyway. However, this method also advances automatically when there is only one choice available. So, if there is no block, and there is no name given, and there is only one choice, it avances. The default usage of this method is to either give it a name of the choice made, or give it a block to make a choice.



61
62
63
64
65
66
67
68
69
70
 
# File 'lib/repositories/transition_cache.rb', line 61

def advance(name=nil, &block)
  name = infer_available_transition_name(block.call(self.available_actions)) if block
  if can_advance_with(name)
    make_transition(find_transition(name))
  elsif name.nil? and self.available_actions.size == 1 and not block
    make_transition(self.available_actions.first)
  else
    nil
  end
end

#available_actions ⇒ Object

Any actions available from this state. The cached value works, because the only way to transition on a state is through make_transition, which removes this cache.



46
47
48
 
# File 'lib/repositories/transition_cache.rb', line 46

def available_actions
  @available_actions ||= self.graph.from(self.current_state)
end