Class: ActiveStorageEncryption::ResumableGCSUpload::ByteChunker

Inherits:
Object
  • Object
show all
Defined in:
lib/active_storage_encryption/resumable_gcs_upload.rb

Overview

When doing GCP uploads the chunks need to be sized to 256KB increments, and the output that we generate is not guaranteed to be chopped up this way. Also the upload for the last chunk is done slightly different than the preceding chunks. It is convenient to have a way to “chop up” an arbitrary streaming output into evenly sized chunks.

Instance Method Summary collapse

Constructor Details

#initialize(chunk_size: 256 * 1024, &delivery_proc) ⇒ ByteChunker

Returns a new instance of ByteChunker.

Parameters:

  • chunk_size (Integer) (defaults to: 256 * 1024) —

    the chunk size that all the chunks except the last one must have



31
32
33
34
35
36
37
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 31

def initialize(chunk_size: 256 * 1024, &delivery_proc)
  @chunk_size = chunk_size.to_i
  # Use a fixed-capacity String instead of a StringIO since there are some advantages
  # to mutable strings, if a string can be reused this saves memory
  @buf_str = String.new(encoding: Encoding::BINARY, capacity: @chunk_size * 2)
  @delivery_proc = delivery_proc.to_proc
end

Instance Method Details

#<<(bin_str) ⇒ Object

Appends data to the buffer. Once the size of the chunk has been exceeded, a precisely-sized chunk will be passed to the delivery_proc

Parameters:

  • bin_str (String) —

    string in binary encoding

Returns:

  • self



44
45
46
47
48
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 44

def <<(bin_str)
  @buf_str << bin_str.b
  deliver_buf_in_chunks
  self
end

#finish ⇒ Object

Sends the last chunk to the delivery_proc even if there is nothing output - the last request will usually be needed to close the file

Returns:

  • void



64
65
66
67
68
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 64

def finish
  deliver_buf_in_chunks
  @delivery_proc.call(@buf_str, _is_last_chunk = true)
  nil
end

#write(bin_str) ⇒ Integer

Appends data to the buffer. Once the size of the chunk has been exceeded, a precisely-sized chunk will be passed to the delivery_proc

Parameters:

  • bin_str (String) —

    string in binary encoding

Returns:

  • (Integer) —

    number of bytes appended to the buffer



55
56
57
58
 
# File 'lib/active_storage_encryption/resumable_gcs_upload.rb', line 55

def write(bin_str)
  self << bin_str
  bin_str.bytesize
end