Method: MapReduce::Mapper#map

Defined in:
lib/map_reduce/mapper.rb

#map(*args, **kwargs) ⇒ Object

Passes the received key to your map-reduce implementation and adds yielded key-value pair to a buffer. When the memory limit is reached, the chunk is sorted and written to a tempfile.

Examples:

mapper.map("some_key")
mapper.map("other_key")

Parameters:

  • The key to pass to the map-reduce implementation.



43
44
45
46
47
48
49
50
51
52
53
54
55
 
# File 'lib/map_reduce/mapper.rb', line 43

def map(*args, **kwargs)
  @implementation.map(*args, **kwargs) do |new_key, new_value|
    synchronize do
      partition = @partitioner.call(new_key)
      item = [[partition, new_key], new_value]

      @buffer.push(item)
      @buffer_size += JSON.generate(item).bytesize

      write_chunk if @buffer_size >= @memory_limit
    end
  end
end