Metrics & Monitoring

Headroom provides comprehensive metrics for monitoring compression performance, cost savings, and system health through both the proxy server and the SDK.

Proxy Endpoints

Stats Endpoint

curl http://localhost:8787/stats

{
  "persistent_savings": {
    "lifetime": {
      "tokens_saved": 12500,
      "compression_savings_usd": 0.04
    }
  },
  "requests": {
    "total": 42,
    "cached": 5,
    "rate_limited": 0,
    "failed": 0
  },
  "tokens": {
    "input": 50000,
    "output": 8000,
    "saved": 12500,
    "savings_percent": 25.0
  },
  "cost": {
    "total_cost_usd": 0.15,
    "total_savings_usd": 0.04
  },
  "cache": {
    "entries": 10,
    "total_hits": 5
  }
}

Persistent savings are stored at ~/.headroom/proxy_savings.json and survive proxy restarts. Override the path with HEADROOM_SAVINGS_PATH.

Historical Savings

curl http://localhost:8787/stats-history

Returns durable compression history with hourly, daily, weekly, and monthly rollups. Supports CSV export:

curl "http://localhost:8787/stats-history?format=csv&series=daily"
curl "http://localhost:8787/stats-history?format=csv&series=monthly"

Prometheus Metrics

curl http://localhost:8787/metrics

# HELP headroom_requests_total Total requests processed
headroom_requests_total 1234

# HELP headroom_tokens_saved_total Total tokens saved
headroom_tokens_saved_total 5678900

# HELP headroom_persistent_savings_tokens_saved_total Durable lifetime input tokens saved by proxy compression
headroom_persistent_savings_tokens_saved_total 5678900

# HELP headroom_latency_ms_sum Sum of request latency in milliseconds
headroom_latency_ms_sum 152300
# HELP headroom_latency_ms_count Count of latency samples
headroom_latency_ms_count 1234

# HELP headroom_provider_cache_hit_requests_total Requests that read from the provider's prompt cache
headroom_provider_cache_hit_requests_total{provider="anthropic"} 456

OpenTelemetry (OTLP) Export

The proxy can also push its counters to any OTLP/HTTP endpoint. Install the extra and set four variables:

pip install "headroom-ai[proxy,otel]"

HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT=http://127.0.0.1:4318/v1/metrics
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy
HEADROOM_OTEL_RESOURCE_ATTRIBUTES=deployment.environment=prod

Exported counters include headroom.proxy.requests, headroom.proxy.tokens.input, and headroom.proxy.tokens.output. headroom.proxy.tokens.saved is the all-layer total: message/compression savings plus tool-schema deferral savings. The component counter headroom.proxy.tokens.tool_schema_saved exposes the deferral portion separately; headroom.compression.tokens.saved remains the compression-pipeline component.

Confirm the exporter is live with curl -s http://localhost:8787/stats | jq .otel.

If your application already configures a global OTEL meter provider, leave HEADROOM_OTEL_* unset — Headroom records into the ambient provider automatically.

Dynatrace

Point the exporter at your environment's OTLP API and add the API token as a header. The token needs the metrics.ingest scope.

HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT="https://<env-id>.live.dynatrace.com/api/v2/otlp/v1/metrics"
HEADROOM_OTEL_METRICS_HEADERS="Authorization=Api-Token dt0c01.XXXX"
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy

Restart the proxy, then search the Dynatrace metric explorer for headroom.proxy.tokens.saved — data appears within ~30s.

For an ActiveGate deployment, swap the base URL for https://<activegate>:9999/e/<env-id>/api/v2/otlp/v1/metrics. If you already run an OpenTelemetry Collector, send Headroom to it instead and add the cumulativetodelta processor — then the temporality variable is unnecessary and the collector holds the token.

Trace export is separate: Headroom's self-configured tracing targets Langfuse only. To land its spans in Dynatrace, leave HEADROOM_LANGFUSE_* unset and run the proxy under opentelemetry-instrument with the standard OTEL_EXPORTER_OTLP_ENDPOINT / OTEL_EXPORTER_OTLP_HEADERS / OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf variables; Headroom records into the ambient tracer provider.

Health Check

curl http://localhost:8787/health

{
  "status": "healthy",
  "version": "0.37.0",
  "uptime_seconds": 3600
}

SDK Metrics

import { HeadroomClient } from 'headroom-ai';

const client = new HeadroomClient();

// Get proxy stats
const stats = await client.proxyStats();
console.log(`Tokens saved: ${stats.tokens.saved}`);
console.log(`Savings: ${stats.tokens.savingsPercent}%`);

import { compress } from 'headroom-ai';

const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Tokens: ${result.tokensBefore} -> ${result.tokensAfter}`);
console.log(`Saved: ${result.tokensSaved} (${(result.compressionRatio * 100).toFixed(1)}%)`);
console.log(`Transforms: ${result.transformsApplied.join(', ')}`);

stats = client.get_stats()
print(f"Mode: {stats['config']['mode']}")
print(f"Tokens saved: {stats['session']['tokens_saved_total']}")
print(f"Requests optimized: {stats['session']['requests_optimized']}")

{
    "session": {
        "requests_total": 10,
        "requests_optimized": 8,
        "requests_audit": 2,
        "tokens_saved_total": 15000,
        "cache_hits": 3,
    },
    "config": {
        "mode": "optimize",
        "provider": "openai",
        "cache_optimizer": "openai-prefix-stabilizer",
        "semantic_cache": False,
    },
    "transforms": {
        "smart_crusher_enabled": True,
        "cache_aligner_enabled": True,
    },
}

from datetime import datetime, timedelta

metrics = client.get_metrics(
    start_time=datetime.utcnow() - timedelta(hours=1),
    limit=100,
)

for m in metrics:
    print(f"{m.timestamp}: {m.tokens_input_before} -> {m.tokens_input_after}")

summary = client.get_summary()
print(f"Total requests: {summary['total_requests']}")
print(f"Total tokens saved: {summary['total_tokens_saved']}")
print(f"Avg tokens saved per request: {summary['avg_tokens_saved']:.0f}")

Logging

import logging

# INFO level shows compression summaries
logging.basicConfig(level=logging.INFO)

# DEBUG level shows detailed transform decisions
logging.basicConfig(level=logging.DEBUG)

INFO:headroom.transforms.pipeline:Pipeline complete: 45000 -> 4500 tokens (saved 40500, 90.0% reduction)
INFO:headroom.transforms.smart_crusher:SmartCrusher applied top_n strategy: kept 15 of 1000 items
DEBUG:headroom.transforms.smart_crusher:Kept items: [0,1,2,42,77,97,98,99] (errors at 42, warnings at 77)

# Log to file
headroom proxy --log-file headroom.jsonl

# Increase verbosity (no --log-level flag; use the env var)
HEADROOM_LOG_LEVEL=debug headroom proxy

Cost Tracking

Budget Alerts

Set a budget limit in the proxy:

headroom proxy --budget 10.00

When the budget is exceeded, requests return a budget exceeded error, the /stats endpoint shows budget status, and logs indicate the budget state.

Measured vs Estimated Spend

Every cost record carries a basis — where its input-token count came from. When a provider response includes a usage breakdown, the basis is measured. When it doesn't, Headroom substitutes its own tokens_sent count so input cost isn't dropped from the budget, and the record's basis is estimated. Headroom logs one warning per model the first time this happens.

/stats keeps the two separable under cost.budget_basis:

{
  "cost": {
    "budget_limit_usd": 10.0,
    "budget_period": "daily",
    "budget_estimated_basis": "count",
    "budget_basis": {
      "total_usd": 3.1400,
      "measured_usd": 2.9000,
      "estimated_usd": 0.2400,
      "estimated_pct": 7.6,
      "records": 412,
      "estimated_records": 31
    }
  }
}

An estimate can drift in either direction, so you choose what it does to the hard limit:

headroom proxy --budget 10.00 --budget-estimated-basis count   # default

Env: HEADROOM_BUDGET_ESTIMATED_BASIS. headroom doctor reports the estimated share alongside the budget check.

Key Metrics to Monitor

Grafana Dashboard

A dashboard template ships in examples/grafana/headroom-dashboard.json. Its metric names match /metrics, but every panel query filters on a pool label (e.g. headroom_tokens_saved_total{pool=~"..."}) that no metric in headroom/proxy/prometheus_metrics.py actually emits, so the panel dropdowns come up empty on import. Edit the queries to drop the pool=~"..." filter, or use the ad-hoc queries below instead.

Example ad-hoc queries against the same metric family: