Class: ScoutRailsProxy::Agent

Inherits:
Object
  • Object
show all
Defined in:
lib/scout_rails_proxy/agent.rb

Overview

The agent gathers performance data from a Ruby application. One Agent instance is created per-Ruby process.

Each Agent object creates a worker thread (unless monitoring is disabled or we’re forking). The worker thread wakes up every Agent#period, merges in-memory metrics w/those saved to disk, saves the merged data to disk, and sends it to the Scout server.

Constant Summary collapse

HTTP_HEADERS =

Headers passed up with all API requests.

{ "Agent-Hostname" => Socket.gethostname }
DEFAULT_HOST = 
'scoutapp.com'
@@instance =

see self.instance

nil

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Agent

Note - this doesn’t start instruments or the worker thread. This is handled via #start as we don’t want to start the worker thread or install instrumentation if (1) disabled for this environment (2) a worker thread shouldn’t be started (when forking).



33
34
35
36
37
38
39
40
41
42
 
# File 'lib/scout_rails_proxy/agent.rb', line 33

def initialize(options = {})
  @started = false
  @options ||= options
  @store = ScoutRailsProxy::Store.new
  @layaway = ScoutRailsProxy::Layaway.new
  @config = ScoutRailsProxy::Config.new(options[:config_path])
  @metric_lookup = Hash.new
  @process_cpu=ScoutRailsProxy::Instruments::Process::ProcessCpu.new(environment.processors)
  @process_memory=ScoutRailsProxy::Instruments::Process::ProcessMemory.new
end

Instance Attribute Details

#config ⇒ Object

Returns the value of attribute config.



17
18
19
 
# File 'lib/scout_rails_proxy/agent.rb', line 17

def config
  @config
end

#environment ⇒ Object

Returns the value of attribute environment.



18
19
20
 
# File 'lib/scout_rails_proxy/agent.rb', line 18

def environment
  @environment
end

#layaway ⇒ Object

Returns the value of attribute layaway.



16
17
18
 
# File 'lib/scout_rails_proxy/agent.rb', line 16

def layaway
  @layaway
end

#log_file ⇒ Object

path to the log file



21
22
23
 
# File 'lib/scout_rails_proxy/agent.rb', line 21

def log_file
  @log_file
end

#logger ⇒ Object

Returns the value of attribute logger.



20
21
22
 
# File 'lib/scout_rails_proxy/agent.rb', line 20

def logger
  @logger
end

#metric_lookup ⇒ Object

Hash used to lookup metric ids based on their name and scope



23
24
25
 
# File 'lib/scout_rails_proxy/agent.rb', line 23

def metric_lookup
  @metric_lookup
end

#options ⇒ Object

options passed to the agent when #start is called.



22
23
24
 
# File 'lib/scout_rails_proxy/agent.rb', line 22

def options
  @options
end

#store ⇒ Object

Accessors below are for associated classes



15
16
17
 
# File 'lib/scout_rails_proxy/agent.rb', line 15

def store
  @store
end

Class Method Details

.instance(options = {}) ⇒ Object

All access to the agent is thru this class method to ensure multiple Agent instances are not initialized per-Ruby process.



26
27
28
 
# File 'lib/scout_rails_proxy/agent.rb', line 26

def self.instance(options = {})
  @@instance ||= self.new(options)
end

Instance Method Details

#add_metric_ids(metrics) ⇒ Object

Before reporting, lookup metric_id for each MetricMeta. This speeds up reporting on the server-side.



182
183
184
185
186
187
188
 
# File 'lib/scout_rails_proxy/agent.rb', line 182

def add_metric_ids(metrics)
  metrics.each do |meta,stats|
    if metric_id = metric_lookup[meta]
      meta.metric_id = metric_id
    end
  end
end

#checkin_uri ⇒ Object 



247
248
249
 
# File 'lib/scout_rails_proxy/agent.rb', line 247

def checkin_uri
  URI.parse("http://#{config.settings['host'] || DEFAULT_HOST}/app/#{config.settings['key']}/checkin.scout?name=#{CGI.escape(config.settings['name'])}")
end

#gem_root ⇒ Object 



92
93
94
 
# File 'lib/scout_rails_proxy/agent.rb', line 92

def gem_root
  File.expand_path(File.join("..","..",".."), __FILE__)
end

#handle_exit ⇒ Object

Placeholder: store metrics locally on exit so those in memory aren’t lost. Need to decide whether we’ll report these immediately or just store locally and risk having stale data.



80
81
82
83
84
85
86
 
# File 'lib/scout_rails_proxy/agent.rb', line 80

def handle_exit
  if environment.sinatra? || environment.jruby? || environment.rubinius?
    logger.debug "Exit handler not supported"
  else
    at_exit { at_exit { logger.debug "Shutdown!" } }
  end
end

#init_logger ⇒ Object 



96
97
98
99
100
101
102
103
104
 
# File 'lib/scout_rails_proxy/agent.rb', line 96

def init_logger
  @log_file = "#{log_path}/scout_rails_proxy.log"
  @logger = Logger.new(@log_file)
  @logger.level = Logger::DEBUG
  def logger.format_message(severity, timestamp, progname, msg)
    prefix = "[#{timestamp.strftime("%m/%d/%y %H:%M:%S %z")} #{Socket.gethostname} (#{$$})] #{severity} : #{msg}\n"
  end
  @logger
end

#install_passenger_worker_process_event ⇒ Object 



114
115
116
117
118
119
 
# File 'lib/scout_rails_proxy/agent.rb', line 114

def install_passenger_worker_process_event
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    logger.debug "Passenger is starting a worker process. Starting worker thread."
    self.class.instance.start_worker_thread
  end
end

#install_unicorn_worker_loop ⇒ Object 



121
122
123
124
125
126
127
128
129
130
 
# File 'lib/scout_rails_proxy/agent.rb', line 121

def install_unicorn_worker_loop
  logger.debug "Installing Unicorn worker loop."
  Unicorn::HttpServer.class_eval do
    old = instance_method(:worker_loop)
    define_method(:worker_loop) do |worker|
      ScoutRailsProxy::Agent.instance.start_worker_thread
      old.bind(self).call(worker)
    end
  end
end

#load_instruments ⇒ Object

Loads the instrumention logic.



295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
 
# File 'lib/scout_rails_proxy/agent.rb', line 295

def load_instruments
  case environment.framework
  when :rails
    require File.expand_path(File.join(File.dirname(__FILE__),'instruments/rails/action_controller_instruments.rb'))
  when :rails3
    require File.expand_path(File.join(File.dirname(__FILE__),'instruments/rails3/action_controller_instruments.rb'))
  when :sinatra
    require File.expand_path(File.join(File.dirname(__FILE__),'instruments/sinatra_instruments.rb'))
  end
  require File.expand_path(File.join(File.dirname(__FILE__),'instruments/active_record_instruments.rb'))
  require File.expand_path(File.join(File.dirname(__FILE__),'instruments/net_http.rb'))
rescue
  logger.warn "Exception loading instruments:"
  logger.warn $!.message
  logger.warn $!.backtrace
end

#log_path ⇒ Object 



132
133
134
 
# File 'lib/scout_rails_proxy/agent.rb', line 132

def log_path
  "#{environment.root}/log"
end

#period ⇒ Object

in seconds, time between when the worker thread wakes up and runs.



137
138
139
 
# File 'lib/scout_rails_proxy/agent.rb', line 137

def period
  60
end

#post(url, body, headers = Hash.new) ⇒ Object 



251
252
253
254
255
256
257
258
259
260
261
 
# File 'lib/scout_rails_proxy/agent.rb', line 251

def post(url, body, headers = Hash.new)
  response = nil
  request(url) do |connection|
    post = Net::HTTP::Post.new( url.path +
                                (url.query ? ('?' + url.query) : ''),
                                HTTP_HEADERS.merge(headers) )
    post.body = body
    response=connection.request(post)
  end
  response
end

#process_metrics ⇒ Object

Called in the worker thread. Merges in-memory metrics w/those on disk and reports metrics to the server.



215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
 
# File 'lib/scout_rails_proxy/agent.rb', line 215

def process_metrics
  logger.debug "Processing metrics"
  run_samplers
  metrics = layaway.deposit_and_deliver
  if metrics.any?
    add_metric_ids(metrics)  
    # for debugging, count the total number of requests    
    controller_count = 0
    metrics.each do |meta,stats|
      if meta.metric_name =~ /\AController/
        controller_count += stats.call_count
      end
    end        
    logger.debug "#{config.settings['name']} Delivering metrics for #{controller_count} requests."        
    response =  post( checkin_uri,
                       Marshal.dump(:metrics => metrics, :sample => store.sample),
                       "Content-Type"     => "application/json" )
    if response and response.is_a?(Net::HTTPSuccess)
      directives = Marshal.load(response.body)
      self.metric_lookup.merge!(directives[:metric_lookup])
      store.transaction_sample_lock.synchronize do
        store.sample = nil
      end
      logger.debug "Metric Cache Size: #{metric_lookup.size}"
    end
  end
rescue
  logger.info "Error on checkin to #{checkin_uri.to_s}"
  logger.info $!.message
  logger.debug $!.backtrace
end

#request(url, &connector) ⇒ Object 



263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
 
# File 'lib/scout_rails_proxy/agent.rb', line 263

def request(url, &connector)
  response = nil
  
  http_class = if config.settings['proxy']
    Net::HTTP::Proxy(
      config.settings['proxy']['host'], 
      config.settings['proxy']['port'], 
      config.settings['proxy']['user'], 
      config.settings['proxy']['password']
    )
  else
    Net::HTTP
  end
  
  http = http_class.new url.host, url.port
  response = http.start(&connector)
  logger.debug "got response: #{response.inspect}"
  case response
  when Net::HTTPSuccess, Net::HTTPNotModified
    logger.debug "/checkin OK"
  when Net::HTTPBadRequest
    logger.warn "/checkin FAILED: The Account Key [#{config.settings['key']}] is invalid."
  else
    logger.debug "/checkin FAILED: #{response.inspect}"
  end
rescue Exception
  logger.debug "Exception sending request to server: #{$!.message}"
ensure
  response
end

#run_samplers ⇒ Object

Called from #process_metrics, which is run via the worker thread.



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'lib/scout_rails_proxy/agent.rb', line 191

def run_samplers
  begin
    cpu_util=@process_cpu.run # returns a hash
    logger.debug "Process CPU: #{cpu_util.inspect} [#{environment.processors} CPU(s)]"
    store.track!("CPU/Utilization",cpu_util) if cpu_util
  rescue => e
    logger.info "Error reading ProcessCpu"
    logger.debug e.message
    logger.debug e.backtrace.join("\n")
  end

  begin
    mem_usage=@process_memory.run # returns a single number, in MB
    logger.debug "Process Memory: #{mem_usage}MB"
    store.track!("Memory/Physical",mem_usage) if mem_usage
  rescue => e
    logger.info "Error reading ProcessMemory"
    logger.debug e.message
    logger.debug e.backtrace.join("\n")
  end
end

#start(options = {}) ⇒ Object

This is called via ScoutRailsProxy::Agent.instance.start when ScoutRailsProxy is required in a Ruby application. It initializes the agent and starts the worker thread (if appropiate).



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/scout_rails_proxy/agent.rb', line 50

def start(options = {})
  @options.merge!(options)
  init_logger
  logger.info "Attempting to start Scout Agent [#{ScoutRailsProxy::VERSION}] on [#{Socket.gethostname}]"
  if !config.settings['monitor'] and !@options[:force]
    logger.warn "Monitoring isn't enabled for the [#{environment.env}] environment."
    return false
  elsif !environment.app_server
    logger.warn "Couldn't find a supported app server. Not starting agent."
    return false
  elsif started?
    logger.warn "Already started agent."
    return false
  end
  @started = true
  logger.info "Starting monitoring. Framework [#{environment.framework}] App Server [#{environment.app_server}]."
  start_instruments
  if !start_worker_thread?
    logger.debug "Not starting worker thread"
    install_passenger_worker_process_event if environment.app_server == :passenger
    install_unicorn_worker_loop if environment.app_server == :unicorn
    return
  end
  start_worker_thread
  handle_exit
  logger.info "Scout Agent [#{ScoutRailsProxy::VERSION}] Initialized"
end

#start_instruments ⇒ Object

Injects instruments into the Ruby application.



313
314
315
316
 
# File 'lib/scout_rails_proxy/agent.rb', line 313

def start_instruments
  logger.debug "Installing instrumentation"
  load_instruments
end

#start_worker_thread ⇒ Object

Creates the worker thread. The worker thread is a loop that runs continuously. It sleeps for Agent#period and when it wakes, processes data, either saving it to disk or reporting to Scout.



143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
# File 'lib/scout_rails_proxy/agent.rb', line 143

def start_worker_thread
  logger.debug "Creating worker thread."
  @worker_thread = Thread.new do
    begin
      logger.debug "Starting worker thread, running every #{period} seconds"
      next_time = Time.now + period
      while true do
        now = Time.now
        while now < next_time
          sleep_time = next_time - now
          sleep(sleep_time) if sleep_time > 0
          now = Time.now
        end
        process_metrics
        while next_time <= now
          next_time += period
        end
      end
    rescue
      logger.debug "Worker Thread Exception!!!!!!!"
      logger.debug $!.message
      logger.debug $!.backtrace
    end
  end # thread new
  logger.debug "Done creating worker thread."
end

#start_worker_thread? ⇒ Boolean

The worker thread will automatically start UNLESS:

  • A supported application server isn’t detected (example: running via Rails console)

  • A supported application server is detected, but it forks (Passenger). In this case, the agent is started in the forked process.

Returns:

  • (Boolean) 


110
111
112
 
# File 'lib/scout_rails_proxy/agent.rb', line 110

def start_worker_thread?
  !environment.forking? or environment.app_server == :thin
end

#started? ⇒ Boolean

Returns:

  • (Boolean) 


88
89
90
 
# File 'lib/scout_rails_proxy/agent.rb', line 88

def started?
  @started
end

#write_to_file(object) ⇒ Object

Writes each payload to a file for auditing.



171
172
173
174
175
176
177
178
 
# File 'lib/scout_rails_proxy/agent.rb', line 171

def write_to_file(object)
  logger.debug "Writing to file"
  full_path = Pathname.new(RAILS_ROOT+'/log/audit/scout')
  ( full_path +
    "#{Time.now.strftime('%Y-%m-%d_%H:%M:%S')}.json" ).open("w") do |f|
    f.puts object.to_json
  end
end