Backtrace Limiting Guide

This guide explains how backtrace limiting works in RailsErrorDashboard and how to configure it for optimal storage efficiency.

What is Backtrace Limiting?

Backtrace limiting automatically truncates error backtraces to a configurable number of lines, reducing database storage requirements while preserving the most important debugging information.

Why Limit Backtraces?

The Problem

Full exception backtraces can be extremely large:

• Typical backtrace: 50-100 lines
• Deep call stacks: 200-500 lines
• Framework-heavy apps: 500+ lines

Example backtrace sizes:

• 50 lines × 100 chars/line = ~5KB per error
• 500 lines × 100 chars/line = ~50KB per error

Impact on 1 million errors:

• 50-line limit: ~5GB storage
• No limit (avg 200 lines): ~20GB storage
• Savings: 75% reduction

The Solution

Limit backtraces to the most useful lines (typically the first 20-50):

• First lines contain your application code (most relevant)
• Middle/end lines are usually framework internals (less useful)
• Storage savings of 60-90%
• No meaningful loss of debugging information

Configuration

Default Configuration

# config/initializers/rails_error_dashboard.rb
RailsErrorDashboard.configure do |config|
  config.max_backtrace_lines = 50  # Default
end

Recommended Values

Custom Configuration Examples

Conservative (Maximum Info)

config.max_backtrace_lines = 100

• Stores first 100 lines
• Good for low-volume apps
• Maximum debugging context

Balanced (Recommended)

config.max_backtrace_lines = 50

• Stores first 50 lines
• Good for most apps
• 70-80% storage reduction
• Retains all relevant info

Aggressive (Maximum Savings)

config.max_backtrace_lines = 20

• Stores first 20 lines
• Good for high-volume apps
• 85-90% storage reduction
• Still captures app-level errors

Minimal

config.max_backtrace_lines = 10

• Stores first 10 lines
• For extremely high-volume
• 90-95% storage reduction
• May miss some context

How It Works

Truncation Logic

# Original backtrace (100 lines)
[
  "app/controllers/users_controller.rb:42:in `create'",
  "app/services/user_service.rb:15:in `register'",
  # ... 48 more app lines ...
  "activesupport/.../callbacks.rb:123:in `run'",
  # ... 50 framework lines ...
]

# After limiting to 50 lines:
[
  "app/controllers/users_controller.rb:42:in `create'",  # Line 1
  "app/services/user_service.rb:15:in `register'",       # Line 2
  # ... 48 more lines ...
  "activesupport/.../callbacks.rb:123:in `run'",          # Line 50
  "... (50 more lines truncated)"                         # Truncation notice
]

Storage Format

Stored in database as text:

app/controllers/users_controller.rb:42:in `create'
app/services/user_service.rb:15:in `register'
app/models/user.rb:89:in `validate_email'
...
... (50 more lines truncated)

The truncation notice tells you how many lines were omitted.

Performance Impact

Storage Savings

Test with 100,000 errors (PostgreSQL):

Query Performance

Smaller backtraces = faster queries:

Write Performance

Smaller backtraces = faster inserts:

Practical Examples

Example 1: Production App

Scenario: E-commerce app, 50K errors/day

Before limiting:

• Avg backtrace: 200 lines
• Daily storage: ~1GB
• Monthly storage: ~30GB
• Annual storage: ~365GB

After limiting (50 lines):

• Avg backtrace: 50 lines
• Daily storage: ~250MB
• Monthly storage: ~7.5GB
• Annual storage: ~91GB
• Savings: 75% (274GB/year)

Example 2: High-Volume SaaS

Scenario: Multi-tenant app, 500K errors/day

Before limiting:

• Daily storage: ~10GB
• Monthly storage: ~300GB
• Cloud storage cost: ~$600/month

After limiting (30 lines):

• Daily storage: ~1.5GB
• Monthly storage: ~45GB
• Cloud storage cost: ~$90/month
• Savings: $510/month ($6,120/year)

Example 3: Startup

Scenario: Growing app, 5K errors/day

Before limiting:

• Database size: Growing 100MB/day
• Performance: Queries slowing down

After limiting (50 lines):

• Database size: Growing 25MB/day
• Performance: 2x faster queries
• Benefit: Extended runway before optimization needed

Best Practices

1. Start Conservative

Begin with higher limits and reduce based on data:

# Week 1: Baseline
config.max_backtrace_lines = 100

# Week 2: After reviewing actual needs
config.max_backtrace_lines = 50

# Week 3: Optimization
config.max_backtrace_lines = 30

2. Environment-Specific Configuration

Use different limits per environment:

# config/initializers/rails_error_dashboard.rb
RailsErrorDashboard.configure do |config|
  config.max_backtrace_lines = case Rails.env
  when 'development'
    100  # More context for local debugging
  when 'staging'
    50   # Balance between dev and prod
  when 'production'
    30   # Optimize for storage
  else
    50   # Safe default
  end
end

3. Monitor Truncation Impact

Check how many errors are being truncated:

# Rails console
truncated_count = RailsErrorDashboard::ErrorLog
  .where("backtrace LIKE ?", "%truncated%")
  .count

total_count = RailsErrorDashboard::ErrorLog.count
percentage = (truncated_count.to_f / total_count * 100).round(1)

puts "#{percentage}% of errors have truncated backtraces"

4. Review Truncated Errors

Periodically check if truncation is removing valuable info:

# Find errors with longest original backtraces
RailsErrorDashboard::ErrorLog
  .where("backtrace LIKE ?", "%truncated%")
  .order(Arel.sql("CAST(SUBSTRING(backtrace FROM '\\((\\d+) more') AS INTEGER) DESC"))
  .limit(10)
  .each do |error|
    # Review these to see if you need more lines
    puts "#{error.error_type}: #{error.backtrace.match(/\((\d+) more/)[1]} lines truncated"
  end

Troubleshooting

“I need more context!”

If truncated backtraces don’t have enough info:

Solution 1: Increase the limit

config.max_backtrace_lines = 100

Solution 2: Enable full backtraces in development

config.max_backtrace_lines = Rails.env.production? ? 30 : 1000

Solution 3: Conditional limiting by error type

# Custom wrapper (in your app)
module CustomErrorLogging
  def self.log_error(exception, context = {})
    # Override limit for critical errors
    if exception.is_a?(SecurityError)
      original_limit = RailsErrorDashboard.configuration.max_backtrace_lines
      RailsErrorDashboard.configure { |c| c.max_backtrace_lines = 200 }
      result = RailsErrorDashboard::Commands::LogError.call(exception, context)
      RailsErrorDashboard.configure { |c| c.max_backtrace_lines = original_limit }
      result
    else
      RailsErrorDashboard::Commands::LogError.call(exception, context)
    end
  end
end

Database Still Growing Too Fast

If storage is still an issue:

Check error volume:

RailsErrorDashboard::ErrorLog
  .group(:error_type)
  .count
  .sort_by { |_, v| -v }
  .first(10)

Solutions:

1. Reduce backtrace lines further (try 20 or 15)
2. Enable sampling for high-volume errors
3. Ignore certain exceptions (e.g., routing errors)
4. Implement retention policy (delete old errors)

Truncation Notice Not Showing

If you don’t see truncation notices:

Possible causes:

1. Backtraces are naturally short (< max lines)
2. Configuration not applied (check initializer)
3. Old errors (truncation only applies to new errors)

Verification:

# Create test error with long backtrace
error = StandardError.new("Test")
error.set_backtrace(200.times.map { |i| "line_#{i}.rb:#{i}" })

log = RailsErrorDashboard::Commands::LogError.call(error, {})
puts log.backtrace.include?("truncated")  # Should be true

Migration Guide

Updating Existing Errors

Backtrace limiting only applies to NEW errors. To truncate existing errors:

# One-time migration (run in console)
RailsErrorDashboard::ErrorLog.find_each do |error_log|
  next if error_log.backtrace.nil?

  lines = error_log.backtrace.lines
  next if lines.count <= 50  # Skip if already short

  truncated = lines.first(50).join
  remaining = lines.count - 50
  truncated += "\n... (#{remaining} more lines truncated)"

  error_log.update_column(:backtrace, truncated)
end

WARNING: This is irreversible - original backtraces are lost!

Gradual Rollout

For safety, enable gradually:

Week 1: Monitor only

# Add logging to see what would be truncated
config.max_backtrace_lines = 1000  # Effectively unlimited

Week 2: Staging deployment

# Enable in staging first
config.max_backtrace_lines = 50

Week 3: Production deployment

# Roll out to production
config.max_backtrace_lines = 30

Advanced Configuration

Dynamic Limiting Based on Error Type

# In your application
module CustomBacktraceLimiting
  def self.limit_for(exception)
    case exception
    when SecurityError, NoMemoryError
      200  # Keep full context for critical errors
    when ActionController::RoutingError
      10   # Minimal context for routing errors
    else
      50   # Default
    end
  end
end

# Wrap the logging call
original_limit = RailsErrorDashboard.configuration.max_backtrace_lines
RailsErrorDashboard.configure do |c|
  c.max_backtrace_lines = CustomBacktraceLimiting.limit_for(exception)
end

RailsErrorDashboard::Commands::LogError.call(exception, context)

RailsErrorDashboard.configure do |c|
  c.max_backtrace_lines = original_limit
end

Compression (Advanced)

For extreme storage savings, compress backtraces:

# Custom migration to add compression
class CompressBacktraces < ActiveRecord::Migration[7.0]
  def up
    add_column :rails_error_dashboard_error_logs, :backtrace_compressed, :binary

    # Migrate existing backtraces
    RailsErrorDashboard::ErrorLog.find_each do |log|
      next if log.backtrace.nil?
      compressed = Zlib::Deflate.deflate(log.backtrace)
      log.update_column(:backtrace_compressed, compressed)
    end
  end
end

# Custom accessor in model
class ErrorLog < ApplicationRecord
  def backtrace
    return super if backtrace_compressed.nil?
    Zlib::Inflate.inflate(backtrace_compressed)
  end
end

Compression ratio: Typically 5-10x reduction (50KB → 5-10KB)

FAQ

Q: Will I lose important debugging information? A: No. The first 30-50 lines contain your application code and the immediate cause. Framework internals (lines 50+) are rarely needed.

Q: Can I view the full backtrace somewhere? A: Once truncated, the full backtrace is not stored. Consider using external error tracking (Sentry, Honeybadger) for full backtraces alongside this dashboard.

Q: Does this affect performance? A: Yes, positively! Smaller backtraces mean faster database operations and less storage cost.

Q: What if an error has a naturally short backtrace? A: No problem. Only backtraces LONGER than the limit are truncated. Short backtraces are stored as-is.

Q: Can I disable truncation? A: Yes, set config.max_backtrace_lines = 10000 (or any very high number).

Additional Resources

• Database Optimization Guide
• Performance Tuning Guide
• Configuration Options