Module: Hasta

Extended by:
Forwardable, Hasta
Included in:
Hasta
Defined in:
lib/hasta.rb,
lib/hasta/env.rb,
lib/hasta/tasks.rb,
lib/hasta/filter.rb,
lib/hasta/mapper.rb,
lib/hasta/runner.rb,
lib/hasta/s3_uri.rb,
lib/hasta/reducer.rb,
lib/hasta/s3_file.rb,
lib/hasta/storage.rb,
lib/hasta/version.rb,
lib/hasta/emr_node.rb,
lib/hasta/s3_storage.rb,
lib/hasta/s3_data_sink.rb,
lib/hasta/tasks/runner.rb,
lib/hasta/configuration.rb,
lib/hasta/local_storage.rb,
lib/hasta/s3_file_cache.rb,
lib/hasta/cached_s3_file.rb,
lib/hasta/s3_data_source.rb,
lib/hasta/identity_mapper.rb,
lib/hasta/local_file_path.rb,
lib/hasta/combined_storage.rb,
lib/hasta/filtered_s3_file.rb,
lib/hasta/identity_reducer.rb,
lib/hasta/execution_context.rb,
lib/hasta/emr_job_definition.rb,
lib/hasta/interpolate_string.rb,
lib/hasta/sorted_data_source.rb,
lib/hasta/in_memory_data_sink.rb,
lib/hasta/combined_data_source.rb,
lib/hasta/in_memory_data_source.rb,
lib/hasta/resolve_cached_s3_file.rb,
lib/hasta/resolve_filtered_s3_file.rb

Overview

Copyright Swipely, Inc. All rights reserved.

Defined Under Namespace

Modules: IdentityMapper, IdentityReducer, LocalFilePath, Storage, Tasks Classes: CachedS3File, CombinedDataSource, CombinedStorage, Configuration, EmrJobDefinition, EmrNode, Env, ExecutionContext, Filter, FilteredS3File, InMemoryDataSink, InMemoryDataSource, InterpolateString, LocalStorage, Mapper, Reducer, ResolveCachedS3File, ResolveFilteredS3File, Runner, S3DataSink, S3DataSource, S3File, S3FileCache, S3Storage, S3URI, SortedDataSource

Constant Summary collapse

Error = 
Class.new(StandardError)
NonExistentPath = 
Class.new(Error)
ClassLoadError = 
Class.new(Error)
ExecutionError = 
Class.new(Error)
ConfigurationError = 
Class.new(Error)
DELEGATED_ATTRS = 
[
  :combined_storage,
  :local_storage_root,
  :logger,
  :project_root,
  :project_steps,
]
VERSION = 
"0.1.0"

Instance Method Summary collapse

Instance Method Details

#configure {|config| ... } ⇒ Object

Yields:

  • (config) 


29
30
31
 
# File 'lib/hasta.rb', line 29

def configure
  yield config
end

#tab_separated_line(line) ⇒ Object 



33
34
35
36
37
38
39
 
# File 'lib/hasta.rb', line 33

def tab_separated_line(line)
  if line.include?("\t")
    line
  else
    "#{line}\t"
  end
end