Skip to content

sb-dev/openquant / .agents/skills/otel-instrumentation

otel-instrumentation

Verified

Configures trace spans, defines custom metrics, sets up log exporters, and optimizes sampling strategies for OpenTelemetry instrumentation. Use when instrumenting applications with traces, metrics, or logs. Triggers on requests for observability, telemetry, tracing, metrics collection, logging integration, or OTel setup.

FileKindSize
README.mdmarkdown4.8 KB
SKILL.mdskill md6.0 KB
rules/capture-database-query-parameters.mdmarkdown4.9 KB
rules/logs.mdmarkdown10.1 KB
rules/metrics.mdmarkdown22.7 KB
rules/platforms/k8s.mdmarkdown7.7 KB
rules/resolve-values.mdmarkdown11.3 KB
rules/resources.mdmarkdown10.9 KB
rules/sdks/browser.mdmarkdown5.2 KB
rules/sdks/dotnet.mdmarkdown20.6 KB
rules/sdks/go.mdmarkdown29.9 KB
rules/sdks/java.mdmarkdown17.2 KB
rules/sdks/nextjs.mdmarkdown16.2 KB
rules/sdks/nodejs.mdmarkdown24.7 KB
rules/sdks/php.mdmarkdown16.2 KB
rules/sdks/python.mdmarkdown18.4 KB
rules/sdks/ruby.mdmarkdown15.1 KB
rules/sdks/scala.mdmarkdown23.6 KB
rules/sensitive-data.mdmarkdown16.2 KB
rules/spans.mdmarkdown32.0 KB
rules/telemetry.mdmarkdown2.1 KB
rules/validation.mdmarkdown8.9 KB
rules/verify-dependencies.mdmarkdown6.6 KB

rules/sdks/ruby.md

View on GitHub
---
title: "Ruby Instrumentation"
impact: HIGH
tags:
  - ruby
  - backend
  - server
---

# Ruby Instrumentation

Instrument Ruby applications to generate traces, logs, and metrics for deep insights into behavior and performance.

## Use cases

- **HTTP Request Monitoring**: Understand outgoing and incoming HTTP requests through traces and metrics, with drill-downs to database level
- **Database Performance**: Observe which database statements execute and measure their duration for optimization
- **Error Detection**: Reveal uncaught errors and the context in which they happened

## Installation

```bash
bundle add opentelemetry-sdk opentelemetry-instrumentation-all opentelemetry-exporter-otlp
```

**Note**: Installing the gems alone is insufficient—you must initialize the SDK AND enable exporters.

### Verifying dependencies

Never write a gem name or version into the `Gemfile` from memory; verify it against RubyGems first, per [verify-dependencies](../verify-dependencies.md):

```bash
gem info --remote opentelemetry-instrumentation-rails
```

Prefer `bundle add <gem>` (no version) over hand-editing the `Gemfile`, so Bundler resolves the real latest version.

`Gemfile.lock` must move with the `Gemfile`: frozen installs (`BUNDLE_FROZEN=true`, `bundle config set frozen true`, deployment mode) refuse to install after a `Gemfile` change, listing the drift (`You have added to the Gemfile:`) and telling you to run `bundle install` elsewhere and commit the updated `Gemfile.lock`.
Regenerate the lockfile where Bundler runs — `bundle lock` resolves and writes `Gemfile.lock` without installing — per [verify-dependencies](../verify-dependencies.md#keeping-the-lockfile-in-step).
Only as a fallback, when no environment with Bundler exists outside the image build, have the builder stage reconcile with a non-frozen `bundle install` — frozen installs are what keep image builds reproducible, so prefer regenerating the lockfile and keeping them frozen.

## Environment variables

All environment variables that control the SDK behavior:

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `OTEL_SERVICE_NAME` | Yes | `unknown_service` | Identifies your service in telemetry data |
| `OTEL_TRACES_EXPORTER` | Yes | `none` | **Must set to `otlp`** to export traces |
| `OTEL_METRICS_EXPORTER` | No | `none` | Set to `otlp` to export metrics |
| `OTEL_LOGS_EXPORTER` | No | `none` | Set to `otlp` to export logs |
| `OTEL_TRACES_SAMPLER` | Yes | `parentbased_always_on` | **Set to `always_on`** — the default lets callers switch off tracing, see [sampling](../spans.md#sampling) |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Yes | `http://localhost:4318` | OTLP collector endpoint |
| `OTEL_EXPORTER_OTLP_HEADERS` | No | - | Headers for authentication (e.g., `Authorization=Bearer TOKEN`) |
| `OTEL_EXPORTER_OTLP_PROTOCOL` | No | `http/protobuf` | Protocol: `grpc`, `http/protobuf`, or `http/json` |
| `OTEL_RESOURCE_ATTRIBUTES` | No | - | Additional resource attributes (e.g., `deployment.environment.name=production`) |

**Critical**: Without `OTEL_TRACES_EXPORTER=otlp`, the SDK defaults to `none` and no telemetry is exported.

### Where to get configuration values

1. **OTLP Endpoint**: Your observability platform's OTLP endpoint
   - In Dash0: [Settings → Organization → Endpoints](https://app.dash0.com/settings/endpoints?s=eJwtyzEOgCAQRNG7TG1Cb29h5REMcVclIUDYsSLcXUxsZ95vcJgbxNObEjNET_9Eok9wY2FIlzlNUnJItM_GYAM2WK7cqmgdlbcDE0yjHlRZfr7KuDJj2W-yoPf-AmNVJ2I%3D)
   - Format: `https://<region>.your-platform.com`
2. **Auth Token**: API token for telemetry ingestion
   - In Dash0: [Settings → Auth Tokens → Create Token](https://app.dash0.com/settings/auth-tokens)
3. **Service Name**: Choose a descriptive name (e.g., `order-api`, `checkout-service`)

## Configuration

### 1. Initialize the SDK

The SDK must be initialized in code on startup, before any application or framework code runs.

**Rails projects** — create `config/initializers/opentelemetry.rb`:

```ruby
require 'opentelemetry/sdk'
require 'opentelemetry/instrumentation/all'
OpenTelemetry::SDK.configure do |c|
  c.use_all()
end
```

**Non-Rails projects** — add to your application entry point before any other requires:

```ruby
require 'opentelemetry/sdk'
require 'opentelemetry/instrumentation/all'
OpenTelemetry::SDK.configure do |c|
  c.use_all()
end
```

### 2. Set service name

```bash
export OTEL_SERVICE_NAME="my-service"
```

### 3. Enable exporters

**This step is required** - without it, no telemetry is sent:

```bash
# Required for traces
export OTEL_TRACES_EXPORTER="otlp"

# Optional: also export metrics and logs
export OTEL_METRICS_EXPORTER="otlp"
export OTEL_LOGS_EXPORTER="otlp"
```

### 4. Configure endpoint

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="https://<OTLP_ENDPOINT>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_AUTH_TOKEN"
```

### 5. Optional: target specific dataset

```bash
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_AUTH_TOKEN,Dash0-Dataset=my-dataset"
```

## Complete setup

### Using environment variables

```bash
# Service identification
export OTEL_SERVICE_NAME="my-service"

# Enable exporters (required!)
export OTEL_TRACES_EXPORTER="otlp"
export OTEL_TRACES_SAMPLER="always_on"
export OTEL_METRICS_EXPORTER="otlp"
export OTEL_LOGS_EXPORTER="otlp"

# Configure endpoint
export OTEL_EXPORTER_OTLP_ENDPOINT="https://<OTLP_ENDPOINT>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_AUTH_TOKEN"

bundle exec rails server
```

### Using .env file with dotenv

Add the `dotenv` gem and create a `.env` file:

**.env:**
```bash
OTEL_SERVICE_NAME=my-service
OTEL_TRACES_EXPORTER=otlp
OTEL_TRACES_SAMPLER=always_on
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=https://<OTLP_ENDPOINT>
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer YOUR_AUTH_TOKEN
```

**Run with:**
```bash
bundle exec rails server
```

## Local development

### Console exporter

For development without a collector, use the console exporter to see telemetry in your terminal:

```bash
export OTEL_SERVICE_NAME="my-service"
export OTEL_TRACES_EXPORTER="console"
export OTEL_METRICS_EXPORTER="console"
export OTEL_LOGS_EXPORTER="console"

bundle exec rails server
```

This prints spans, metrics, and logs directly to stdout—useful for verifying instrumentation works before configuring a remote backend.

### Without a collector

If you set `OTEL_TRACES_EXPORTER=otlp` but have no collector running, you will see connection errors.
This is expected behavior.

**Options:**
1. Use `console` exporter during development (recommended for quick testing)
2. Run a local OpenTelemetry Collector
3. Point directly to your observability backend

## Resource configuration

Set `service.name`, `service.version`, and `deployment.environment.name` for every deployment.
See [resource attributes](../resources.md) for the full list of required and recommended attributes.

## Kubernetes setup

See [Kubernetes deployment](../platforms/k8s.md) for pod metadata injection, resource attributes, and Dash0 Kubernetes Operator guidance.

## Supported libraries

The auto-instrumentation package automatically instruments:

| Category | Libraries |
|----------|-----------|
| HTTP | Rack, Rails, Sinatra, Faraday, Net::HTTP |
| Database | PG, MySQL2, ActiveRecord |
| Cache | Redis, Dalli |
| Messaging | Sidekiq, Resque, Bunny |
| External | RestClient, Ethon, HTTP.rb |
| GraphQL | GraphQL |
| Logging | Logger |

Refer to the [OpenTelemetry Ruby Contrib repository](https://github.com/open-telemetry/opentelemetry-ruby-contrib/tree/main/instrumentation) for the complete list.

## Database query parameters

The OpenTelemetry Ruby instrumentation packages (`opentelemetry-instrumentation-pg`, `opentelemetry-instrumentation-mysql2`, `opentelemetry-instrumentation-active_record`) do **not** capture prepared-statement parameter values, and there is no env var or instrumentation option to enable capture.
Read [capturing database query parameters](../capture-database-query-parameters.md) first.

The `:db_statement` option on `OpenTelemetry::Instrumentation::PG::Instrumentation` controls sanitization of `db.statement` only (`:include`, `:omit`, `:obfuscate`); it does not emit `db.query.parameter.<key>`.

```ruby
# Available knobs — none capture parameter values
OpenTelemetry::SDK.configure do |c|
  c.use 'OpenTelemetry::Instrumentation::PG', { db_statement: :obfuscate }
end
```

The auto-instrumentation creates and ends its span inside a `Module#prepend` wrapper around the driver method, so attaching attributes from application code after the call returns is a no-op.

## Custom spans

Add business context to auto-instrumented traces:

```ruby
tracer = OpenTelemetry.tracer_provider.tracer('my-service')

def process_order(order)
  tracer = OpenTelemetry.tracer_provider.tracer('my-service')
  tracer.in_span('order.process') do |span|
    span.set_attribute('order.id', order.id)
    span.set_attribute('order.total', order.total)
    result = save_order(order)
    result
  rescue StandardError => e
    span.status = OpenTelemetry::Trace::Status.error(e.message)
    ctx = span.context
    logger.error('order.process.failed',
      trace_id: ctx.hex_trace_id,
      span_id: ctx.hex_span_id,
      'exception.type': e.class.name,
      'exception.message': e.message,
      'exception.stacktrace': e.backtrace&.join("\n"))
    raise
  end
end
```

### Retrieving the active span

Auto-instrumentation creates spans you do not control directly (e.g., the `SERVER` span for an HTTP request).
To enrich these spans with business context or set their status, retrieve the active span from the current context.
See [adding attributes to auto-instrumented spans](../spans.md#adding-attributes-to-auto-instrumented-spans) for when to use this pattern.

```ruby
span = OpenTelemetry::Trace.current_span
span.set_attribute('order.id', params[:order_id])
span.set_attribute('tenant.id', request.headers['X-Tenant-Id'])
```

`OpenTelemetry::Trace.current_span` returns a non-recording span if no span is active.
Calling `set_attribute` or `status=` on a non-recording span is a no-op, so no guard is needed.

### Span status rules

See [span status code](../spans.md#span-status-code) for the full rules.
This section shows how to apply them in Ruby.

#### Always include a status message with `ERROR`

The argument to `Status.error` is the status message.
It must contain the error class and a short explanation — enough to understand the failure without opening the full trace.

```ruby
# BAD: no status message
span.status = OpenTelemetry::Trace::Status.error

# BAD: generic message with no diagnostic value
span.status = OpenTelemetry::Trace::Status.error('something went wrong')

# GOOD: specific message with error class and context
span.status = OpenTelemetry::Trace::Status.error("#{e.class}: #{e.message}")
```

Do not include backtraces in the status message.
Record those in a log record with `exception.stacktrace` instead.

```ruby
# BAD: backtrace in the status message
span.status = OpenTelemetry::Trace::Status.error(e.full_message)

# GOOD: short message only
span.status = OpenTelemetry::Trace::Status.error(e.message)
```

#### Use `OK` only for confirmed success

Set status to `OK` when application logic has explicitly verified the operation succeeded.
Leave status `UNSET` if the code simply did not encounter an error.

```ruby
# GOOD: explicit confirmation from downstream
response = Net::HTTP.get_response(uri)
if response.is_a?(Net::HTTPSuccess)
  span.status = OpenTelemetry::Trace::Status.ok
end

# BAD: setting OK speculatively
span.status = OpenTelemetry::Trace::Status.ok
some_method # might still fail after this point
```

## Structured logging

Configure your logging framework to serialize exceptions into a single structured field so that stack traces do not break the one-line-per-record contract.
See [logs](../logs.md) for general guidance on structured logging and exception stack traces.

### Semantic Logger

[semantic_logger](https://github.com/reidmorrison/semantic_logger) produces single-line JSON with exceptions serialized into structured fields.

```ruby
require 'semantic_logger'

SemanticLogger.add_appender(io: $stdout, formatter: :json)
logger = SemanticLogger['OrderService']

begin
  process_order(order)
rescue StandardError => e
  logger.error('order.failed', exception: e, order_id: order.id)
end
```

The JSON formatter serializes the exception class, message, and backtrace into structured fields, keeping each log record on a single line.

### Lograge (Rails)

For Rails applications, [lograge](https://github.com/roidrage/lograge) replaces the default multi-line request log with a single-line JSON entry.

```ruby
# config/environments/production.rb
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Json.new
```

Lograge does not handle exception backtraces directly.
Pair it with semantic_logger or a JSON formatter that serializes exceptions as single-line fields.

## Graceful shutdown

The Ruby SDK does not register shutdown hooks automatically.
Register an `at_exit` hook to flush and shut down providers before the process terminates, so buffered spans, metrics, and log records are not lost.

```ruby
at_exit do
  OpenTelemetry.tracer_provider.shutdown if OpenTelemetry.respond_to?(:tracer_provider)
  OpenTelemetry.meter_provider.shutdown if OpenTelemetry.respond_to?(:meter_provider)
  OpenTelemetry.logger_provider.shutdown if OpenTelemetry.respond_to?(:logger_provider)
end
```

Place the `at_exit` block immediately after `OpenTelemetry::SDK.configure` in your initializer.
`shutdown` flushes pending batches and releases resources.
The call blocks until export completes or the timeout expires (default 30 seconds).

## Troubleshooting

### No telemetry appearing

**Check exporters are enabled:**
```bash
echo $OTEL_TRACES_EXPORTER  # Should be "otlp" or "console", not empty
```

The SDK defaults `OTEL_TRACES_EXPORTER` to `none`, which silently discards all telemetry.

**Verify SDK is initialized:**
Ensure the `OpenTelemetry::SDK.configure` block runs before your application code.
In Rails, this means placing it in `config/initializers/opentelemetry.rb`.

### Connection errors

This means the SDK is working but cannot reach the collector:
- **No collector running**: Start a local collector or use `OTEL_TRACES_EXPORTER=console`
- **Wrong endpoint**: Check `OTEL_EXPORTER_OTLP_ENDPOINT` is correct
- **Port mismatch**: gRPC uses 4317, HTTP uses 4318

### Instrumentation not picking up libraries

Ensure `c.use_all()` is called inside the `OpenTelemetry::SDK.configure` block.
Verify that the `opentelemetry-instrumentation-all` gem is installed.
Some libraries require their instrumentation gem to be explicitly added to the Gemfile.

### "Exporter is empty" or similar warnings

Usually means `OTEL_TRACES_EXPORTER` (or metrics/logs) is not set.
Set it explicitly:
```bash
export OTEL_TRACES_EXPORTER="otlp"
```

## Resources

- [OpenTelemetry Ruby Documentation](https://opentelemetry.io/docs/languages/ruby/)
- [OpenTelemetry Ruby Contrib](https://github.com/open-telemetry/opentelemetry-ruby-contrib)
- [Environment Variable Specification](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/)
- [Dash0 Kubernetes Operator](https://github.com/dash0hq/dash0-operator)
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall