Module: ScraperUtils::DebugUtils
- Defined in:
- lib/scraper_utils/debug_utils.rb
Overview
Utilities for debugging web scraping processes
Constant Summary collapse
- DEBUG_ENV_VAR =
"DEBUG"
- MORPH_DEBUG_ENV_VAR =
"MORPH_DEBUG"
- DISABLED_LEVEL =
Debug level constants
0
- BASIC_LEVEL =
1
- VERBOSE_LEVEL =
2
- TRACE_LEVEL =
3
Class Method Summary collapse
-
.basic? ⇒ Boolean
Check if basic debug output or higher is enabled.
-
.debug?(level = BASIC_LEVEL) ⇒ Boolean
Check if debug is enabled at specified level or higher.
-
.debug_level ⇒ Integer
Get current debug level (0 = disabled, 1 = basic, 2 = verbose, 3 = trace) Checks DEBUG and MORPH_DEBUG env variables.
-
.debug_page(page, message) ⇒ void
Logs details of a web page when debug mode is enabled.
-
.debug_request(http_method, url, parameters: nil, headers: nil, body: nil) ⇒ void
Logs details of an HTTP request when debug mode is enabled.
-
.debug_selector(page, selector, message) ⇒ void
Logs details about a specific page selector when debug mode is enabled.
-
.trace? ⇒ Boolean
Check if debug tracing or higher is enabled.
-
.verbose? ⇒ Boolean
Check if verbose debug output or higher is enabled.
Class Method Details
.basic? ⇒ Boolean
Check if basic debug output or higher is enabled
35 36 37 |
# File 'lib/scraper_utils/debug_utils.rb', line 35 def self.basic? debug?(BASIC_LEVEL) end |
.debug?(level = BASIC_LEVEL) ⇒ Boolean
Check if debug is enabled at specified level or higher
29 30 31 |
# File 'lib/scraper_utils/debug_utils.rb', line 29 def self.debug?(level = BASIC_LEVEL) debug_level >= level end |
.debug_level ⇒ Integer
Get current debug level (0 = disabled, 1 = basic, 2 = verbose, 3 = trace) Checks DEBUG and MORPH_DEBUG env variables
20 21 22 23 |
# File 'lib/scraper_utils/debug_utils.rb', line 20 def self.debug_level debug = ENV.fetch(DEBUG_ENV_VAR, ENV.fetch(MORPH_DEBUG_ENV_VAR, '0')) debug =~ /^\d/ ? debug.to_i : BASIC_LEVEL end |
.debug_page(page, message) ⇒ void
This method returns an undefined value.
Logs details of a web page when debug mode is enabled
76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
# File 'lib/scraper_utils/debug_utils.rb', line 76 def self.debug_page(page, ) return unless trace? puts LogUtils.log "🔍 DEBUG: #{}" puts "Current URL: #{page.uri}" puts "Page title: #{page.at('title').text.strip}" if page.at("title") puts "", "Page content:", "-" * 40, page.body, "-" * 40 $stdout.flush end |
.debug_request(http_method, url, parameters: nil, headers: nil, body: nil) ⇒ void
This method returns an undefined value.
Logs details of an HTTP request when debug mode is enabled
60 61 62 63 64 65 66 67 68 69 |
# File 'lib/scraper_utils/debug_utils.rb', line 60 def self.debug_request(http_method, url, parameters: nil, headers: nil, body: nil) return unless basic? puts LogUtils.log "🔍 #{http_method.upcase} #{url}" puts "Parameters:", JSON.pretty_generate(parameters) if parameters puts "Headers:", JSON.pretty_generate(headers) if headers puts "Body:", JSON.pretty_generate(body) if body $stdout.flush end |
.debug_selector(page, selector, message) ⇒ void
This method returns an undefined value.
Logs details about a specific page selector when debug mode is enabled
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 |
# File 'lib/scraper_utils/debug_utils.rb', line 97 def self.debug_selector(page, selector, ) return unless trace? puts LogUtils.log "🔍 DEBUG: #{}" puts "Looking for selector: #{selector}" element = page.at(selector) if element puts "Found element:" puts element.to_html else puts "Element not found in:" puts "-" * 40 puts page.body puts "-" * 40 end $stdout.flush end |
.trace? ⇒ Boolean
Check if debug tracing or higher is enabled
47 48 49 |
# File 'lib/scraper_utils/debug_utils.rb', line 47 def self.trace? debug?(TRACE_LEVEL) end |
.verbose? ⇒ Boolean
Check if verbose debug output or higher is enabled
41 42 43 |
# File 'lib/scraper_utils/debug_utils.rb', line 41 def self.verbose? debug?(VERBOSE_LEVEL) end |