Repeated agent wakes usually rebuild the same expensive prompt shape. The useful split is simple: keep the byte-stable prefix first, keep the volatile wake-specific digest later, and keep run-specific context at the end so the repeated prefix stays cacheable.
## Repeated wake anatomy [#repeated-wake-anatomy]
Each wake tends to carry four different layers:
* stable instructions, project rules, and tool contracts
* the current task and other instruction-bearing fields
* the volatile per-wake memory digest
* recent loop state, tool output, and child-agent context
The stable layers should stay byte-identical across wakes. The volatile layers should move to the live zone, where they can change without invalidating the cacheable prefix.
## CacheAligner is detector-only [#cachealigner-is-detector-only]
CacheAligner does not rewrite messages. It inspects the prefix, emits warnings for volatile content, and records observability data so callers can fix their own assembly logic.
The detector reports:
| Field | What it tells you |
| ---------------------------------------- | -------------------------------------------------------------------- |
| `warnings` | Which parts of the prefix look unstable |
| `cache_metrics.stable_prefix_bytes` | Size of the stable prefix in bytes |
| `cache_metrics.stable_prefix_tokens_est` | Estimated token size of the stable prefix |
| `cache_metrics.stable_prefix_hash` | Hash of the stable prefix for repeated-wake comparison |
| `cache_metrics.prefix_changed` | Whether the stable prefix drifted since the last wake |
| `cache_metrics.previous_hash` | Hash from the previous wake, when available |
| `markers` | The emitted `stable_prefix_hash` marker for downstream observability |
If CacheAligner warns about drift, keep the prefix stable in the caller. The transform is a detector, not a repair pass.
## Stable prefix layout [#stable-prefix-layout]
Put the byte-stable prefix first, then the live wake digest, then any run-specific context.
That layout keeps the provider cache path predictable:
1. Stable instructions stay identical.
2. The wake digest changes without disturbing the cacheable prefix.
3. Recent tool output and child-agent context stay outside the repeatable prefix.
## Real wake measurements [#real-wake-measurements]
Use the issue comment guidance when measuring real wakes:
| Field | Record |
| --------------------- | ----------------------------------------------------------- |
| Prompt section id | Name the section that changed or repeated |
| Byte/token estimate | Capture the section size before and after curation |
| Digest version | Track which digest schema was used |
| Cache-hit expectation | Note whether the section should stay cacheable |
| Compressed size | Record the compacted payload size |
| Section type | Mark the section as instruction-bearing or safe to compress |
That checklist helps separate the repeated prefix from the volatile digest before you decide where CCR belongs.
## CCR digest curation [#ccr-digest-curation]
CCR keeps compression reversible. The compressed digest can stay compact while the original backing detail remains recoverable from the local store.
The pieces that matter here are:
* `headroom_retrieve` for on-demand recovery of stored originals
* `HEADROOM_CCR_TTL_SECONDS` for sizing the local store lifetime
* `compression_strategy` as the authoritative discriminator on stored CCR entries
For routing decisions, the same rule in plain terms is: headroom\_retrieve recovers originals, HEADROOM\_CCR\_TTL\_SECONDS sizes the local lifetime, compression\_strategy identifies the producing path, and shape inference is not the routing authority.
When a stored original expires, regenerate the digest or re-read the source content. Do not infer routing from payload shape. Use the stored `compression_strategy` metadata to understand how the original was produced.
## Digest routing [#digest-routing]
Route fields by how much exact wording they need at wake time.
| Field | Suggested handling | Why |
| ------------------------ | ------------------------------------------------------------------------- | --------------------------------------------------------------- |
| Current task | Verbatim | It is instruction-bearing and changes the next action |
| Hard constraints | Verbatim | These are safety and acceptance boundaries |
| Definitions of done | Verbatim | The wording needs to survive every wake intact |
| Irreversible decisions | Verbatim summary plus retrievable backing detail | The summary stays short, the backing detail stays exact |
| Open threads | Compact summary plus CCR-backed backing detail | The live thread can shrink while the source stays recoverable |
| Learnings | Compact summary | They inform the next wake without needing exact prose |
| File/search/tool outputs | CCR-backed compression | These are large backing details that should stay retrievable |
| Prose notes | Compact summary, CCR-backed when bulky | Keep the digest readable without losing source detail |
| Bulk JSON-ish arrays | SmartCrusher or ContentRouter, with CCR-backed backing detail when needed | Structured blobs are usually the first thing to explode in size |
The important boundary is simple: instruction-bearing fields stay verbatim, or they get a verbatim compact summary plus retrievable backing detail. CCR-backed content is the backing detail, not the instruction itself.
Hard constraints stay verbatim; current task text stays verbatim; file/search/tool outputs use CCR-backed backing detail when they are large enough to compress.
## Integration modes [#integration-modes]
Choose the integration mode by where the orchestrator controls message assembly.
| Mode | Use when | Notes |
| -------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| Proxy | You want spawned agents and normal client traffic to pass through a local provider endpoint | Good for `headroom proxy --mode cache` and provider base URL routing |
| Library | The orchestrator owns message assembly and wants to shape the digest before launch | Best when the caller can decide what becomes stable prefix versus live digest |
| MCP | Agents need on-demand compression and retrieval tools | Best when `headroom_retrieve` should be available as a tool |
| Proxy plus MCP | You need traffic shaping and tool-level retrieval together | Useful when both the provider edge and the agent toolset matter |
## Local-first deployment [#local-first-deployment]
Keep the deployment local to the user or session:
* run one local proxy or MCP process per user session
* do not assume a central proxy
* treat the local CCR store as process-local unless the deployment explicitly shares it
* size the TTL for the longest realistic autonomous run
* assume a store can expire before the run finishes, then regenerate or re-read the source content
That model keeps the data boundary obvious. The orchestrator can still recover backing detail, but the cacheable prefix stays small and stable.
## Source-backed caveats [#source-backed-caveats]
* Prompt-cache hits require a byte-identical stable prefix.
* CacheAligner identifies drift, it does not repair prompt assembly.
* CCR retrieval depends on store lifetime and stored hashes.
* Provider-neutral guidance still applies, so keep the guide away from Orcha-specific runtime claims.
* Use `compression_strategy` to read stored CCR intent, not payload shape.
Headroom integrates with [Agno](https://github.com/agno-agi/agno) (formerly Phidata) to compress context for AI agents. Wrap any Agno model for automatic optimization, and use hooks for observability.
## Installation [#installation]
```bash
pip install "headroom-ai[agno]" agno
```
## Quick start [#quick-start]
```python
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from headroom.integrations.agno import HeadroomAgnoModel
model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o"))
agent = Agent(model=model)
response = agent.run("What's the capital of France?")
print(f"Tokens saved: {model.total_tokens_saved}")
print(model.get_savings_summary())
# {'total_requests': 1, 'total_tokens_saved': 245, 'average_savings_percent': 12.3,
# 'total_tokens_before': 1990, 'total_tokens_after': 1745}
```
Works with any Agno provider:
```python
from agno.models.anthropic import Claude
from agno.models.google import Gemini
claude_model = HeadroomAgnoModel(wrapped_model=Claude(id="claude-sonnet-4-20250514"))
gemini_model = HeadroomAgnoModel(wrapped_model=Gemini(id="gemini-2.0-flash"))
```
## Observability hooks [#observability-hooks]
Use hooks for detailed tracking without modifying your model:
```python
from headroom.integrations.agno import (
HeadroomAgnoModel,
HeadroomPreHook,
HeadroomPostHook,
)
model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o"))
pre_hook = HeadroomPreHook()
post_hook = HeadroomPostHook(token_alert_threshold=10000)
agent = Agent(
model=model,
pre_hooks=[pre_hook],
post_hooks=[post_hook],
)
response = agent.run("Analyze this large dataset...")
# Check for alerts
if post_hook.alerts:
print(f"{len(post_hook.alerts)} requests exceeded threshold")
```
Or use the convenience factory:
```python
from headroom.integrations.agno import create_headroom_hooks
pre_hook, post_hook = create_headroom_hooks(
token_alert_threshold=5000,
log_level="DEBUG",
)
```
## Tool-heavy agents [#tool-heavy-agents]
Tool outputs (JSON, logs, search results) see the biggest compression gains at 70-90% reduction:
```python
from agno.tools.duckduckgo import DuckDuckGoTools
model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o"))
agent = Agent(
model=model,
tools=[DuckDuckGoTools()],
show_tool_calls=True,
)
response = agent.run("Research the latest AI developments")
print(f"Tokens saved: {model.total_tokens_saved}")
```
## Async support [#async-support]
```python
import asyncio
async def process():
model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o"))
response = await model.aresponse(messages)
async for chunk in model.aresponse_stream(messages):
print(chunk, end="", flush=True)
asyncio.run(process())
```
## Standalone message optimization [#standalone-message-optimization]
Optimize messages without wrapping a model:
```python
from headroom.integrations.agno import optimize_messages
optimized, metrics = optimize_messages(messages, model="gpt-4o")
print(f"Tokens saved: {metrics['tokens_saved']}")
```
## Session management [#session-management]
Reset metrics between sessions:
```python
model = HeadroomAgnoModel(wrapped_model=OpenAIChat(id="gpt-4o"))
# Session 1
agent.run("First conversation...")
print(model.get_savings_summary())
# Reset for new session
model.reset()
# Session 2 starts fresh
agent.run("Second conversation...")
```
## Supported providers [#supported-providers]
| Provider | Agno Model | Auto-Detected |
| --------- | -------------------------- | ------------- |
| OpenAI | `OpenAIChat`, `OpenAILike` | Yes |
| Anthropic | `Claude`, `AwsBedrock` | Yes |
| Google | `Gemini`, `VertexAI` | Yes |
| Groq | `Groq` | Yes |
| Mistral | `Mistral` | Yes |
| Ollama | `Ollama` | Yes |
Headroom wraps the Anthropic TypeScript SDK to automatically compress messages before every `messages.create()` call. All other methods pass through unchanged.
## Installation [#installation]
```bash
npm install headroom-ai @anthropic-ai/sdk
```
The TypeScript SDK sends messages to a local Headroom proxy for compression. Start the proxy before using the SDK:
```bash
pip install "headroom-ai[proxy]"
headroom proxy
```
## Quick start [#quick-start]
```ts twoslash
import { withHeadroom } from 'headroom-ai/anthropic';
import Anthropic from '@anthropic-ai/sdk';
const client = withHeadroom(new Anthropic());
const response = await client.messages.create({
model: 'claude-sonnet-4-5-20250929',
messages: longConversation,
max_tokens: 1024,
});
```
Every call to `client.messages.create()` compresses messages first. The response format is identical to the unwrapped client.
## How it works [#how-it-works]
`withHeadroom()` returns a proxy around your Anthropic client that intercepts `messages.create()`:
1. Converts Anthropic-format messages to OpenAI format
2. Sends them to the Headroom proxy's `/v1/compress` endpoint
3. Converts the compressed messages back to Anthropic format
4. Forwards the request to Anthropic as normal
### Message format conversion [#message-format-conversion]
The adapter handles the full Anthropic message format including content blocks:
| Anthropic format | OpenAI format |
| ----------------------------------------------- | --------------------------------------------------------- |
| `{ type: "text", text: "..." }` | `{ role: "user", content: "..." }` |
| `{ type: "tool_use", id, name, input }` | `{ tool_calls: [{ id, function: { name, arguments } }] }` |
| `{ type: "tool_result", tool_use_id, content }` | `{ role: "tool", tool_call_id, content }` |
This conversion is lossless. Your request and response behave identically to an unwrapped client.
`POST /v1/compress` does no format conversion and compresses Anthropic content blocks natively — see [Message format](/docs/proxy#message-format). If you are calling the endpoint directly (from a gateway, or LiteLLM's `headroom` guardrail), send Anthropic-shaped messages as-is; you get the same shape back. Only this TypeScript adapter converts, because it normalises on OpenAI types internally.
## Options [#options]
Pass compression options as the second argument:
```ts twoslash
import { withHeadroom } from 'headroom-ai/anthropic';
import Anthropic from '@anthropic-ai/sdk';
const client = withHeadroom(new Anthropic(), {
model: 'claude-sonnet-4-5-20250929',
baseUrl: 'http://localhost:8787',
});
```
## Streaming [#streaming]
Streaming works normally. Compression happens before the request:
```ts twoslash
import { withHeadroom } from 'headroom-ai/anthropic';
import Anthropic from '@anthropic-ai/sdk';
const client = withHeadroom(new Anthropic());
const stream = await client.messages.create({
model: 'claude-sonnet-4-5-20250929',
messages: longConversation,
max_tokens: 1024,
stream: true,
});
```
## Tool use [#tool-use]
Tool results are where compression has the biggest impact. Large JSON payloads from tool calls are compressed automatically:
```ts twoslash
import { withHeadroom } from 'headroom-ai/anthropic';
import Anthropic from '@anthropic-ai/sdk';
const client = withHeadroom(new Anthropic());
const response = await client.messages.create({
model: 'claude-sonnet-4-5-20250929',
max_tokens: 1024,
messages: [
{ role: 'user', content: 'What went wrong?' },
{
role: 'assistant',
content: [
{ type: 'tool_use', id: 'toolu_1', name: 'get_logs', input: { service: 'api' } },
],
},
{
role: 'user',
content: [
{
type: 'tool_result',
tool_use_id: 'toolu_1',
content: hugeLogOutput, // Compressed automatically
},
],
},
],
tools: [{ name: 'get_logs', description: 'Get logs', input_schema: { type: 'object', properties: {} } }],
});
```
Complete API reference for the Headroom Python and TypeScript SDKs.
## Core [#core]
### HeadroomClient [#headroomclient]
The main entry point for the Headroom SDK.
```ts twoslash
import { HeadroomClient } from 'headroom-ai';
const client = new HeadroomClient({
baseUrl: 'http://localhost:8787',
apiKey: 'your-api-key',
timeout: 30_000,
fallback: true,
retries: 2,
});
```
**Constructor Parameters**
```python
from headroom import HeadroomClient, OpenAIProvider, HeadroomConfig, SmartCrusherConfig, CacheAlignerConfig
from openai import OpenAI
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
default_mode="optimize",
config=HeadroomConfig(
smart_crusher=SmartCrusherConfig(min_tokens_to_crush=100),
cache_aligner=CacheAlignerConfig(enabled=True),
),
)
```
### chat.completions.create() [#chatcompletionscreate]
Create a chat completion with optional optimization.
The TypeScript SDK uses `compress()` to optimize messages before sending them to your LLM client:
```ts twoslash
import { compress } from 'headroom-ai';
const result = await compress(messages, {
model: 'gpt-4o',
tokenBudget: 100_000,
});
// Then pass result.messages to your LLM client
```
Accepts all standard OpenAI/Anthropic parameters plus Headroom-specific overrides:
```python
response = client.chat.completions.create(
model="gpt-4o",
messages=[...],
headroom_mode="optimize",
headroom_keep_turns=5,
)
```
`headroom_tool_profiles` is accepted but not currently wired through
`TransformPipeline.apply()`, so passing it changes nothing. To keep
specific tool outputs from being compressed as aggressively, use
`config.smart_crusher.max_items_after_crush` or
`config.smart_crusher.enabled = False` instead — see
[Compression Too Aggressive](/docs/troubleshooting#compression-too-aggressive).
### chat.completions.simulate() [#chatcompletionssimulate]
Preview optimization without making an API call.
```python
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=[...],
)
print(f"Tokens: {plan.tokens_before} -> {plan.tokens_after}")
print(f"Savings: {plan.tokens_saved/plan.tokens_before*100:.1f}%")
print(f"Transforms: {plan.transforms}")
```
**Returns:** `SimulationResult`
### compress() (TypeScript) [#compress-typescript]
Top-level function to compress messages via the Headroom proxy.
```ts twoslash
import { compress } from 'headroom-ai';
const result = await compress(messages, {
model: 'gpt-4o',
baseUrl: 'http://localhost:8787',
timeout: 15_000,
fallback: true,
retries: 2,
tokenBudget: 100_000,
});
```
### get\_stats() [#get_stats]
Quick stats for the current session (no database query).
```python
stats = client.get_stats()
# Returns dict with "session", "config", and "transforms" keys
```
### get\_metrics() [#get_metrics]
Query stored metrics from the database.
```python
from datetime import datetime, timedelta
metrics = client.get_metrics(
start_time=datetime.utcnow() - timedelta(hours=1),
limit=100,
)
```
### get\_summary() [#get_summary]
Aggregate statistics across all stored metrics.
```python
summary = client.get_summary()
# Returns dict with total_requests, total_tokens_before, total_tokens_after,
# total_tokens_saved, avg_tokens_saved, avg_cache_alignment,
# audit_count, optimize_count
```
### validate\_setup() [#validate_setup]
Validate that the client is configured correctly.
```python
result = client.validate_setup()
# Returns {"valid": bool, "provider": {...}, "storage": {...},
# "config": {...}, "cache_optimizer": {...}}, each with "ok"/"error"
if not result["valid"]:
for key in ("provider", "storage", "config", "cache_optimizer"):
if not result[key]["ok"]:
print(f" - {key}: {result[key]['error']}")
```
***
## Configuration [#configuration]
### SmartCrusherConfig [#smartcrusherconfig]
```python
from headroom import SmartCrusherConfig
config = SmartCrusherConfig(
min_tokens_to_crush=200,
max_items_after_crush=15,
variance_threshold=2.0,
preserve_change_points=True,
)
```
### CacheAlignerConfig [#cachealignerconfig]
```python
from headroom import CacheAlignerConfig
config = CacheAlignerConfig(
enabled=True,
use_dynamic_detector=True,
normalize_whitespace=True,
entropy_threshold=0.7,
)
```
### Context management [#context-management]
Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom never drops messages from the conversation history; it compresses only the newest content blocks (latest user message, latest tool result) and keeps the cache hot zone — system prompt, tools, and older turns — untouched. Use the `headroom_keep_turns` / `headroom_output_buffer_tokens` per-request overrides to tune behavior. The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes have been retired from the Python package (`from headroom import RollingWindowConfig` fails) and from the compression pipeline itself; the TypeScript SDK's `HeadroomConfig` type still exports the equivalent field names as unused legacy type surface, wired to nothing.
### HeadroomConfig [#headroomconfig]
The top-level config object that contains all sub-configurations:
```python
from headroom import HeadroomConfig
config = HeadroomConfig()
config.smart_crusher.min_tokens_to_crush = 100
config.cache_aligner.enabled = True
# Note: rolling_window has been removed — use headroom_keep_turns per-request instead
```
### RelevanceScorerConfig [#relevancescorerconfig]
***
## Results [#results]
### CompressResult (TypeScript) [#compressresult-typescript]
### SimulationResult (Python) [#simulationresult-python]
### WasteSignals (Python) [#wastesignals-python]
`plan.waste_signals` is a plain `dict[str, int]`, not a class instance:
### RequestMetrics (Python) [#requestmetrics-python]
Cost estimates are computed on demand inside `HeadroomClient` (`estimate_cost()` on the provider) rather than stored on `RequestMetrics`.
***
## Providers [#providers]
### OpenAIProvider [#openaiprovider]
```python
from headroom import OpenAIProvider
provider = OpenAIProvider()
counter = provider.get_token_counter("gpt-4o")
tokens = counter.count_text("Hello, world!")
limit = provider.get_context_limit("gpt-4o") # 128000
cost = provider.estimate_cost(input_tokens=1000, output_tokens=500, model="gpt-4o")
```
### AnthropicProvider [#anthropicprovider]
```python
from headroom import AnthropicProvider
from anthropic import Anthropic
provider = AnthropicProvider(
client=Anthropic(),
)
counter = provider.get_token_counter("claude-3-5-sonnet-latest")
tokens = counter.count_messages(messages) # Accurate count via API
```
### GoogleProvider [#googleprovider]
```python
from headroom.providers import GoogleProvider
provider = GoogleProvider()
```
***
## Relevance Scoring [#relevance-scoring]
### create\_scorer() [#create_scorer]
Factory function to create scorers:
```python
from headroom import create_scorer
# Auto-select best available scorer
scorer = create_scorer()
# Explicitly choose type
scorer = create_scorer(tier="hybrid", alpha=0.7)
```
### BM25Scorer [#bm25scorer]
Fast keyword-based scoring (zero dependencies):
```python
from headroom import BM25Scorer
scorer = BM25Scorer()
scores = scorer.score_batch(["item 1", "item 2"], "search query")
```
### EmbeddingScorer [#embeddingscorer]
Semantic similarity scoring (requires `headroom-ai[relevance]`):
```python
from headroom import EmbeddingScorer, embedding_available
if embedding_available():
scorer = EmbeddingScorer(model_name="BAAI/bge-small-en-v1.5")
scores = scorer.score_batch(items, query)
```
### HybridScorer [#hybridscorer]
Combines BM25 and embeddings:
```python
from headroom import HybridScorer
scorer = HybridScorer(alpha=0.5) # 50% BM25, 50% embedding
scores = scorer.score_batch(items, query)
```
***
## Transforms (Direct Use) [#transforms-direct-use]
### SmartCrusher [#smartcrusher]
```python
from headroom import SmartCrusher
import json
crusher = SmartCrusher()
result = crusher.crush(content=json.dumps({"results": [...]}), query="user query")
```
### CacheAligner [#cachealigner]
```python
from headroom import CacheAligner, Tokenizer, OpenAIProvider
provider = OpenAIProvider()
tokenizer = Tokenizer(provider.get_token_counter("gpt-4o"), "gpt-4o")
aligner = CacheAligner()
result = aligner.apply(messages, tokenizer)
```
### TransformPipeline [#transformpipeline]
```python
from headroom import TransformPipeline, SmartCrusher, CacheAligner
pipeline = TransformPipeline(transforms=[
SmartCrusher(),
CacheAligner(),
])
# model_limit is required for direct pipeline use — HeadroomClient
# supplies it automatically from the provider's context limits
result = pipeline.apply(messages, "gpt-4o", model_limit=128_000)
```
***
## Errors [#errors]
| Exception | Meaning |
| ------------------------- | ------------------------------------------------------- |
| `HeadroomError` | Base class for all errors |
| `HeadroomConnectionError` | Cannot reach proxy |
| `HeadroomAuthError` | 401 from proxy |
| `HeadroomCompressError` | Compression failed (includes `statusCode`, `errorType`) |
| `ConfigurationError` | Invalid configuration |
| `ProviderError` | Provider issues |
| `StorageError` | Storage failures |
| `TokenizationError` | Token counting failed |
| `CacheError` | Cache operations failed |
| `ValidationError` | Validation failures |
| `TransformError` | Transform execution failed |
Use `mapProxyError(status, type, message)` to convert proxy error responses to the correct class.
| Exception | Meaning |
| -------------------- | ------------------------------------ |
| `HeadroomError` | Base class for all Headroom errors |
| `ConfigurationError` | Invalid config values |
| `ProviderError` | Provider issue (unknown model, etc.) |
| `StorageError` | Database issue |
| `CompressionError` | Compression failed (rare) |
| `ValidationError` | Setup validation failed |
All exceptions include a `details` dict with additional context.
***
## Utilities [#utilities]
### Tokenizer [#tokenizer]
```python
from headroom import Tokenizer, count_tokens_text, count_tokens_messages, OpenAIProvider
provider = OpenAIProvider()
token_counter = provider.get_token_counter("gpt-4o")
# Quick counting
tokens = count_tokens_text("Hello, world!", token_counter)
# With tokenizer instance
tokenizer = Tokenizer(token_counter, "gpt-4o")
tokens = tokenizer.count_text("Hello")
tokens = tokenizer.count_messages(messages)
```
### generate\_report() [#generate_report]
Generate HTML/Markdown reports from stored metrics:
```python
from headroom import generate_report
report = generate_report(
store_url="sqlite:///headroom.db",
format="html",
period="day",
)
```
***
## TypeScript Message Types [#typescript-message-types]
The TypeScript SDK uses the standard OpenAI message format with `SystemMessage`, `UserMessage`, `AssistantMessage`, and `ToolMessage` variants.
Headroom sits between your application and the LLM provider. It intercepts the request, compresses the parts that carry the most redundant tokens — tool outputs, file reads, logs, search results — and forwards the optimized request upstream. The provider's response is returned unchanged.
## High-level flow [#high-level-flow]
```
+---------------------------------------------------------------+
| YOUR APPLICATION |
+---------------------------------------------------------------+
|
v
+---------------------------------------------------------------+
| HEADROOM |
| Proxy (FastAPI) · Python compress() · TS compress() |
| | |
| v |
| Transform pipeline ──▶ ContentRouter |
| (detect content type, route to one compressor) |
| | |
| v |
| Backend (direct · LiteLLM · any-llm) |
+---------------------------------------------------------------+
|
v
+---------------------------------------------------------------+
| OPENAI · ANTHROPIC · GOOGLE · BEDROCK · 100+ |
+---------------------------------------------------------------+
```
## Entry points [#entry-points]
Headroom can be used three ways, all feeding the same compression pipeline:
| Entry point | How it works | Code changes |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------ |
| **Proxy mode** | Run `headroom proxy` and point your client's base URL at it | Zero — just change the base URL |
| **SDK mode** | Call `compress()` (Python or TypeScript) on your messages before you send them, or wrap an existing client with `HeadroomClient` for automatic per-call optimization and stats | Minimal — one function call, or a client wrapper |
| **Integrations** | LangChain, Vercel AI SDK, Agno, Strands, LiteLLM, MCP adapters | Framework-specific setup |
In proxy mode the server is a FastAPI app with per-provider handlers (Anthropic, OpenAI, Gemini, Bedrock) that each run the same compression pipeline before forwarding through the selected [backend](/docs/proxy#cloud-providers).
## The compression pipeline [#the-compression-pipeline]
The proxy assembles a small, ordered pipeline. Every transform is independent, safe to skip, and **fails open** — on any error it returns the content unchanged and the request still goes through.
1. **Tool-result interceptor** *(canary opt-in)* — light structural interceptors such as ast-grep Read outlining. Requires `HEADROOM_ROLLOUT_CHANNEL=canary` plus `--intercept-tool-results`.
2. **CacheAligner** *(off by default)* — a detector that reports dynamic-prefix drift (dates, UUIDs, session tokens). It **never mutates, moves, or rewrites** content. It is disabled by default and hard-disabled inside the proxy; it exists to surface prefix-stability metrics, not to change your messages.
3. **ContentRouter** — the workhorse that does essentially all of the compression. See below.
The pipeline **never drops or reorders messages**. Earlier versions shipped a "context manager" stage (rolling-window / intelligent-context scoring) that deleted old turns to make the request fit the context window. That stage was **removed** — the pipeline no longer runs any dropping/scoring logic, and the `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are gone from the Python package (the TypeScript SDK's types still export the equivalent field names, but wired to nothing). Headroom now does *live-zone-only* compression: it compresses content in place and leaves the message list intact. See [Context Management](/docs/context-management).
### ContentRouter [#contentrouter]
ContentRouter detects the type of each content block and dispatches it to exactly one compressor:
| Detected content | Compressor | Typical savings |
| --------------------------------- | -------------------------------------------- | --------------- |
| JSON arrays (tool outputs) | SmartCrusher | 70–90% |
| Source code | CodeAwareCompressor (opt-in; off by default) | 40–70% |
| Search / grep results | SearchCompressor | 80–95% |
| Build / test logs | LogCompressor | 85–95% |
| Diffs | DiffCompressor | 40–80% |
| HTML | HTMLExtractor (trafilatura) | 70–90% |
| Tabular (CSV/TSV/markdown tables) | TabularCompressor | 60–90% |
| Structured config (YAML/TOML/INI) | ConfigCompressor | 40–70% |
| Plain text | TextCrusher | 30–60% |
| Anything else | Kompress (ML fallback) | varies |
To avoid recompressing the same content, ContentRouter keeps a **two-tier, TTL-bounded cache**: a *skip set* of content already known not to compress, and a *result cache* of previously compressed output. Default TTL is 30 minutes.
### Rust core [#rust-core]
The heaviest compressors run in a native Rust extension — `headroom._core`, built with PyO3 — that the proxy loads at startup. SmartCrusher and the search/log/diff compressors and content detection are Rust-backed; the Python classes you import are thin, API-compatible shims over them. The ML fallback (**Kompress**, a ModernBERT token compressor) runs separately through ONNX Runtime, either locally or offloaded to a remote endpoint (see [Text & Logs](/docs/text-and-logs)).
## Compression modes and provider caches [#compression-modes-and-provider-caches]
Headroom runs in one of two modes (`--mode`, default `cache`):
* **`cache` mode (default)** — compresses only the newest delta in each turn and forwards prior turns byte-faithfully, so the provider's prefix cache is never invalidated mid-conversation. Best for coding agents and any long, multi-turn session.
* **`token` mode** — prioritizes raw token removal and may recompress earlier turns, trading some cache stability for maximum savings.
Because prefix caching is where most of the cost savings live on multi-turn workloads, keeping the stable prefix intact matters. What each provider's cache buys you:
| Provider | Mechanism | Savings on cached tokens |
| --------- | ------------------------------------ | ------------------------ |
| Anthropic | `cache_control` on the stable prefix | up to \~90% |
| OpenAI | automatic prefix caching | up to \~50% |
| Google | `CachedContent` API | up to \~75% |
See [Cache Optimization](/docs/cache-optimization) and [Savings Profiles](/docs/proxy#savings-profiles) for how the two modes interact with the built-in profiles.
## CCR: Compress-Cache-Retrieve [#ccr-compress-cache-retrieve]
Compression is **reversible**. When ContentRouter compresses a tool output, the original is stored in a local Compress-Cache-Retrieve (CCR) store. If the model needs the full data, it calls a `headroom_retrieve` tool and gets the original back.
```
Compress: 1000 items -> 15 items (original stored in CCR)
Cache: hash-indexed local SQLite store
Retrieve: model calls headroom_retrieve("abc123") -> original 1000 items
```
CCR is **on by default**. Disable the markers and the injected retrieval tool with `--no-ccr`, or run a marker-free, format-native lossless mode with `--lossless`. See [Reversible Compression (CCR)](/docs/ccr).
## TOIN: Tool Output Intelligence Network [#toin-tool-output-intelligence-network]
TOIN learns which fields matter for a given tool over repeated calls — which items get retrieved, which fields carry signal — and feeds that back into SmartCrusher's importance scoring so compression gets sharper for the tools you actually use.
TOIN is **local and observation-only**: it aggregates statistics on your own machine (or your own proxy instance). Nothing about your tools or traffic is shared across users or sent off the box. For a brand-new tool type it falls back to statistical heuristics and improves as it observes more calls.
## What Headroom does not rewrite [#what-headroom-does-not-rewrite]
* **Your prompt text** — the natural-language instructions you write are preserved. Compression targets bulk content blocks (tool outputs, file reads, logs), not your intent.
* **System prompts** — preserved by default so the hottest part of the prefix cache stays stable. A savings profile can opt into compacting them.
* **Code** — passes through unchanged unless AST-based code compression is explicitly enabled (off by default).
* **Model responses** — returned from the provider unchanged.
* **Short content** — blocks below the minimum-token threshold pass through (overhead would exceed savings).
Headroom integrates with [AutoGen](https://github.com/microsoft/autogen) (`autogen-agentchat` >=0.7) to compress tool outputs before they enter the agent's model context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see the largest token reductions.
## Installation [#installation]
```bash
pip install headroom-ai autogen-agentchat
```
## Quick start [#quick-start]
Wrap tools in one line:
```python
from autogen_agentchat.agents import AssistantAgent
from autogen_core.tools import FunctionTool
from headroom.integrations.autogen import wrap_tools_with_headroom
def search_database(query: str) -> str:
"""Search the database and return results."""
return json.dumps({"results": [...], "total": 1000})
tool = FunctionTool(search_database, description="Search the database")
wrapped = wrap_tools_with_headroom([tool])
agent = AssistantAgent(
name="researcher",
model_client=model_client,
tools=wrapped,
)
```
## Per-tool metrics [#per-tool-metrics]
Track compression stats across all tool invocations:
```python
from headroom.integrations.autogen import get_tool_metrics
metrics = get_tool_metrics()
print(metrics.get_summary())
# {
# 'total_invocations': 25,
# 'total_compressions': 18,
# 'total_chars_saved': 450000,
# 'average_compression_ratio': 0.35,
# 'by_tool': {
# 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000},
# }
# }
```
Reset between sessions:
```python
from headroom.integrations.autogen import reset_tool_metrics
reset_tool_metrics()
```
## Custom configuration [#custom-configuration]
Control the compression threshold:
```python
wrapped = wrap_tools_with_headroom(
[search_tool, log_tool],
min_chars_to_compress=500, # Default: 1000
)
```
Use a dedicated metrics collector:
```python
from headroom.integrations.autogen import ToolMetricsCollector, wrap_tools_with_headroom
collector = ToolMetricsCollector()
wrapped = wrap_tools_with_headroom(
[search_tool],
metrics_collector=collector,
)
print(collector.get_summary())
```
## Wrapping individual tools [#wrapping-individual-tools]
For finer control, wrap tools individually:
```python
from headroom.integrations.autogen import HeadroomToolWrapper
wrapper = HeadroomToolWrapper(
search_tool,
min_chars_to_compress=500,
)
# Get the wrapped FunctionTool
compressed_tool = wrapper.as_function_tool()
agent = AssistantAgent(
name="researcher",
model_client=model_client,
tools=[compressed_tool],
)
```
## Async support [#async-support]
AutoGen tools are natively async. The wrapper handles both sync and async
tool functions transparently:
```python
async def async_search(query: str) -> str:
"""Async database search."""
results = await db.search(query)
return json.dumps(results)
tool = FunctionTool(async_search, description="Async search")
wrapped = wrap_tools_with_headroom([tool])
# Compression works identically for async tools
```
## How it works [#how-it-works]
AutoGen routes tool execution through `FunctionTool`, which wraps a plain
Python function. The function's return value is stringified and becomes
`FunctionExecutionResult.content` — what the LLM reads on its next turn.
`HeadroomToolWrapper` creates a new `FunctionTool` with a wrapper function that:
1. Calls the original function
2. Checks if the stringified output exceeds `min_chars_to_compress`
3. If so, compresses via Headroom's `compress_tool_result()`
4. Records metrics and returns the compressed string
The wrapper preserves the original tool's name, description, and parameter
schema, so it works as a drop-in replacement.
### Why not `tool_call_summary_formatter`? [#why-not-tool_call_summary_formatter]
AutoGen's `AssistantAgent` accepts a `tool_call_summary_formatter` parameter,
which looks like a natural hook. However, it only controls the **final summary
message** emitted after the tool loop exits — it does not touch the raw
`FunctionExecutionResult` that gets added to `model_context` (what the LLM
actually reads). Wrapping the function is the only clean interception point.
Headroom's core promise: compress context without losing accuracy. This page covers compression benchmarks, accuracy evaluations, and latency overhead. Every number below is measured locally and reproducible (see [Reproducing Results](#reproducing-results)).
For local inference, the main benefit is often faster prompt processing rather than lower API spend. See [Local LLM prefill benchmarking](/docs/local-llm-prefill) for a reproducible passthrough-vs-optimized proxy workflow.
## Compression Performance [#compression-performance]
Measured on Apple M-series (CPU), Headroom 0.37.0, via `python benchmarks/bench_latency.py` (10 warm iterations per scenario; latency is p50 compression overhead, not an LLM call).
| Content Type | Original | Compressed | Saved | Ratio | Latency (p50) |
| --------------------------------------- | ----------- | ---------- | ---------- | ------- | ------------- |
| JSON array — search results (100 items) | 10,200 | 5,300 | 4,900 | **48%** | 0.20ms |
| JSON array — search results (500 items) | 50,200 | 25,800 | 24,400 | **49%** | 0.77ms |
| Structured logs (100 entries) | 7,600 | 3,600 | 4,000 | **53%** | 0.17ms |
| Structured logs (500 entries) | 35,700 | 16,200 | 19,400 | **54%** | 0.51ms |
| Documentation text (20K tokens) | 20,100 | 1,600 | 18,400 | **92%** | 1.4ms |
| Python source (\~200 lines) | 2,600 | 2,600 | 0 | 0.0% | 1.3ms |
| **Total** | **126,400** | **55,100** | **71,300** | **56%** | **4.4ms** |
Python source shows 0% compression: SmartCrusher only compresses JSON arrays, and code passes through to preserve correctness (verified: `headroom/transforms/smart_crusher.py:242`).
## Accuracy Benchmarks [#accuracy-benchmarks]
### HTML Extraction [#html-extraction]
**Dataset**: Scrapinghub Article Extraction Benchmark (181 HTML pages with ground truth), via `allenai/scrapinghub-article-extraction-benchmark` on Hugging Face.
| Metric | Value |
| --------------- | ----- |
| **F1 Score** | 0.919 |
| **Precision** | 0.875 |
| **Recall** | 0.985 |
| **Compression** | 94.8% |
For LLM applications, recall is critical -- 98.5% means nearly all article content is preserved. The slight precision drop (some extra content) does not hurt LLM accuracy.
### JSON Compression (SmartCrusher) [#json-compression-smartcrusher]
**Test**: 100-entry JSON log array with one anomalous error entry (error code, resolution, and affected count) injected at position 67, compressed via the public `compress()` API with an OpenAI-format tool-result message.
| Metric | Value |
| ------------------------------- | --------------------------------- |
| Input tokens (before) | 4,937 |
| Input tokens (after) | 3,053 |
| Compression | **38.2%** |
| Error entry preserved in output | Yes (verified by substring check) |
SmartCrusher preserves first N items (schema), last N items (recency), all anomalies (errors, warnings), and statistical distribution. This test confirms the anomalous entry survives compression; it does not include an LLM-graded "correct answers" comparison, which would require a paid API call to reproduce.
### QA Accuracy Preservation [#qa-accuracy-preservation]
This comparison (`tests/test_evals/test_html_oss_benchmarks.py::TestQAAccuracyPreservation`) runs SQuAD questions against both the original HTML and the extracted text using an LLM judge, and only executes when `OPENAI_API_KEY` is set. No committed result artifact for it exists in this repo, so no number is published here. Set `OPENAI_API_KEY` and run `pytest tests/test_evals/test_html_oss_benchmarks.py -k qa_accuracy -v -s` to reproduce it yourself.
## Latency Overhead [#latency-overhead]
### SDK Compression Latency [#sdk-compression-latency]
Measured per-scenario on Apple M-series (CPU) via `python benchmarks/bench_latency.py --scenario json --iterations 10`. This is the local compression pipeline's own overhead — no LLM call is involved.
| Scenario | Tokens In | Tokens Out | Saved | p50 (ms) | p95 (ms) |
| -------------------------------- | --------- | ---------- | ----- | -------- | -------- |
| JSON: Search Results (100 items) | 10.2K | 5.3K | 4.9K | 0.20 | 0.21 |
| JSON: Search Results (500 items) | 50.2K | 25.8K | 24.4K | 0.77 | 1.1 |
| JSON: Search Results (1K items) | 100.5K | 51.7K | 48.8K | 1.4 | 1.6 |
| JSON: API Responses (500 items) | 38.9K | 21.8K | 17.0K | 0.77 | 0.80 |
| JSON: Database Rows (1K rows) | 43.7K | 16.2K | 27.5K | 0.78 | 0.81 |
| JSON: String Array (100 strings) | 1.1K | 226 | 825 | 0.15 | 0.16 |
| JSON: String Array (500 strings) | 4.9K | 228 | 4.6K | 0.11 | 0.12 |
| JSON: Number Array (200 numbers) | 1.2K | 146 | 1.1K | 0.17 | 0.18 |
| JSON: Mixed Array (250 items) | 2.3K | 1.1K | 1.2K | 0.24 | 0.25 |
### Cost-Benefit Analysis [#cost-benefit-analysis]
Net latency benefit = LLM time saved from fewer tokens minus compression overhead (at Claude Sonnet 4.5's 0.03ms/token prefill rate, $3.0/MTok input pricing):
| Scenario | Compress (ms) | LLM Saved (ms) | Net Benefit | Savings per 1K Requests |
| -------------------------------- | ------------- | -------------- | -------------- | ----------------------- |
| JSON: Search Results (100 items) | 0.20 | 146 | **+145.8ms** | $14.60 |
| JSON: Search Results (500 items) | 0.77 | 731 | **+730.2ms** | $73.09 |
| JSON: Search Results (1K items) | 1.4 | 1,464 | **+1,462.6ms** | $146.40 |
| JSON: API Responses (500 items) | 0.77 | 511 | **+510.2ms** | $51.10 |
| JSON: Database Rows (1K rows) | 0.78 | 824 | **+822.9ms** | $82.36 |
Compression paid for itself in latency for all 25 compressing scenarios measured (JSON, structured logs, agentic multi-turn, and long-document text) against Claude Sonnet 4.5. Slower and more expensive models (Opus) benefit even more, since the same compression overhead offsets a larger per-token prefill cost.
### Pipeline Step Timing [#pipeline-step-timing]
Measured via the same `bench_latency.py` run's per-transform breakdown (content detection + routing is the dominant step in every JSON scenario):
| Scenario | `content_router` p50 | % of total pipeline time |
| -------------------------------- | -------------------- | ------------------------ |
| JSON: String Array (500 strings) | 0.08ms | 68% |
| JSON: Search Results (100 items) | 0.16ms | 82% |
| JSON: Mixed Array (250 items) | 0.21ms | 85% |
| JSON: Database Rows (1K rows) | 0.72ms | 95% |
| JSON: Search Results (5K items) | 6.6ms | 99% |
ContentRouter accounted for 68--99% of pipeline cost across the 14 JSON scenarios measured (mean \~87%), the rest going to tokenization and the compression transform itself.
## Reproducing Results [#reproducing-results]
```bash
git clone https://github.com/headroomlabs-ai/headroom.git
cd headroom
pip install -e ".[evals,html]"
pytest tests/test_evals/ -v -s
# Latency and compression-ratio tables above
python benchmarks/bench_latency.py
```
LLM providers cache prompt prefixes to avoid reprocessing identical input on repeated calls. Headroom's **CacheAligner** is detector-only, so it surfaces prefix drift, reports observability data, and leaves message assembly to the caller.
CacheAligner is **disabled by default** and hard-disabled inside the proxy — it never runs on your traffic unless you explicitly enable it, and even then it only *reports* metrics; it never repairs a prefix. In proxy mode, prefix-cache stability comes from **cache mode** (`--mode cache`, the default), which compresses only the newest delta and forwards prior turns byte-faithfully. See [Savings Profiles](/docs/proxy#savings-profiles).
## What CacheAligner reports [#what-cachealigner-reports]
System prompts often contain dynamic content, such as dates, session IDs, and timestamps, that changes between requests. Even a single character difference at the start of a prompt invalidates the entire provider cache.
CacheAligner does not extract, move, normalize, reorder, strip, compress, or rewrite content. It detects volatile content and reports the stable prefix hash plus cache metrics so you can fix the prefix at the source:
| Signal | Meaning |
| ---------------------------------------- | --------------------------------------------------------- |
| `warnings` | The prefix contains unstable content |
| `cache_metrics.stable_prefix_bytes` | Stable prefix size in bytes |
| `cache_metrics.stable_prefix_tokens_est` | Stable prefix size in estimated tokens |
| `cache_metrics.stable_prefix_hash` | Stable prefix hash for repeated-wake comparison |
| `cache_metrics.prefix_changed` | The prefix drifted since the previous wake |
| `markers` | The emitted `stable_prefix_hash` marker for observability |
The prefix must stay byte-identical across requests for provider KV caches to reuse previously computed attention states.
## Provider-specific strategies [#provider-specific-strategies]
Each LLM provider implements caching differently. Headroom applies the optimal strategy for each.
### Anthropic [#anthropic]
Anthropic supports explicit `cache_control` blocks that mark content as cacheable. Cached input tokens cost **90% less** than regular input tokens.
Keep the stable prefix byte-identical, then place provider cache markers where your client or orchestrator already assembles the request. Headroom's job is to surface prefix instability, not repair it.
| Metric | Value |
| ------------------- | --------------------------- |
| Cache read discount | 90% off input price |
| Cache write cost | 25% premium on first write |
| Cache TTL | 5 minutes (extended on hit) |
### OpenAI [#openai]
OpenAI uses automatic **prefix caching**. If consecutive requests share the same message prefix, the provider reuses cached KV states. No explicit API markers are needed, but the prefix must be byte-identical.
CacheAligner tells you when the prefix changed, which is the only signal you need to keep OpenAI prefix caching effective.
| Metric | Value |
| ------------------- | ------------------------ |
| Cache read discount | 50% off input price |
| Activation | Automatic (prefix match) |
| Min prefix length | 1024 tokens |
### Google [#google]
Google provides the **CachedContent API**, which lets you explicitly cache large context (system instructions, documents, tools) and reference it across requests. Cached tokens cost **75% less**.
Keep the prefix stable in your integration layer; Headroom reports when the cacheable zone drifts. The CachedContent lifecycle itself also stays in your integration layer.
| Metric | Value |
| ------------------- | ---------------------------------- |
| Cache read discount | 75% off input price |
| Mechanism | Explicit CachedContent API objects |
| Min cache size | 32,768 tokens |
## What this means in practice [#what-this-means-in-practice]
Keep the stable prefix first, keep volatile content out of it, and treat CacheAligner warnings as a signal that the caller needs to move assembly logic.
CacheAligner surfaces prefix instability, provider caches reward byte-identical prefixes, and the caller owns the actual message layout.
## Cold-prefix recompaction (when the cache lapses) [#cold-prefix-recompaction-when-the-cache-lapses]
Byte-identical forwarding only pays off while the prompt cache is warm. When a
session goes idle past the provider's cache TTL, that cache is dead — so
re-sending the prefix verbatim buys nothing, and it's the one safe moment to
rewrite it. The **cold-prefix hook** detects a lapsed cache and recompacts the
whole prefix (cross-turn dedupe + superseded-read drop + lossless folds) instead,
then re-caches the smaller result. It only fires on a *confirmed*-cold turn — a
wrong call would bust a warm cache — so it reads the real TTL (exact for Claude
Code via its cache-control env vars; learned over time for other providers).
For models that re-send **reasoning** as plain text every turn (Kimi/GLM/DeepSeek),
a companion hook Kompresses that reasoning on warm turns and drops it on cold ones.
Both are off by default. See
[Cold-prefix hook & reasoning compaction](/docs/configuration#cold-prefix-hook--reasoning-compaction)
for the exact flags and when each is safe to enable.
Headroom's CCR (Compress-Cache-Retrieve) architecture makes compression **reversible**. When content is compressed, the original data is cached locally. If the LLM needs the full data, it retrieves it instantly.
Unlike traditional lossy compression, CCR guarantees that every piece of original data remains accessible in the compression store, even when the compressed content the LLM sees is much smaller. Actual token savings depend heavily on how redundant the source data is -- see [How Compression Works](/docs/how-compression-works#content-type-detection) for measured ranges per content type.
## The problem with traditional compression [#the-problem-with-traditional-compression]
Traditional compression forces a difficult tradeoff:
* **Aggressive compression** risks losing data the LLM needs
* **Conservative compression** misses out on token savings
CCR eliminates this tradeoff entirely. Compress aggressively, retrieve on demand.
## Architecture [#architecture]
CCR flows through four phases:
```
TOOL OUTPUT (1000 items)
-> SmartCrusher compresses to 20 items
-> Original cached with hash=abc123
-> Retrieval tool injected into context
LLM PROCESSING
Option A: LLM solves task with 20 items -> Done (90% savings)
Option B: LLM calls headroom_retrieve(hash=abc123)
-> Response Handler returns full data automatically
```
## Phase 1: Compression Store [#phase-1-compression-store]
When SmartCrusher compresses tool output:
1. The original content is stored, keyed by hash, with LRU-style eviction once the store hits capacity. In the proxy, this store defaults to a SQLite file (`ccr_store.db` in the workspace directory) rather than a pure in-process dict -- so it survives a proxy restart and is shared across worker processes. Set `HEADROOM_CCR_BACKEND=memory` to opt back into an in-process-only store. Other backend names are resolved via the `headroom.ccr_backend` setuptools entry-point group (e.g. a `redis` backend, if one is installed and registered) -- none besides `sqlite`/`memory` ship in this repo. See `headroom/cache/compression_store.py:1000-1017`.
2. A hash key is generated for retrieval
3. A marker is added to the compressed output:
```
[1000 items compressed to 20. Retrieve more: hash=abc123]
```
## Phase 2: Tool Injection [#phase-2-tool-injection]
Headroom injects a `headroom_retrieve` tool into the LLM's available tools:
```json
{
"name": "headroom_retrieve",
"description": "Retrieve original uncompressed data from Headroom cache",
"parameters": {
"hash": "The hash key from the compression marker"
}
}
```
The LLM sees this tool alongside your application's tools and can call it whenever the compressed data is insufficient.
## Phase 3: Response Handler [#phase-3-response-handler]
When the LLM calls `headroom_retrieve`:
1. The Response Handler intercepts the tool call
2. Data is retrieved from the local cache (around 1ms)
3. The result is added to the conversation
4. The API call continues automatically
The client never sees CCR tool calls on the Anthropic and OpenAI proxy paths; Headroom resolves them transparently there.
Buffered native Gemini requests resolve `headroom_retrieve` server-side and
return the model's final response. Streaming native Gemini requests keep the
existing forwarding behavior. When a response contains `headroom_retrieve`
alongside a client-owned function call, Headroom preserves both calls for the
client instead of resolving the mixed response. Google's OpenAI-compatible Gemini endpoint can
also return `finish_reason=MALFORMED_FUNCTION_CALL` on large function-response
continuations after CCR retrieval; that separate limitation remains tracked in
[issue #2041](https://github.com/headroomlabs-ai/headroom/issues/2041).
## Phase 4: Context Tracker [#phase-4-context-tracker]
Across multiple turns, the Context Tracker maintains awareness of all compressed content:
1. Remembers what was compressed in earlier turns
2. Analyzes new queries for relevance to compressed content
3. Proactively expands relevant data before the LLM asks
```
Turn 1: User searches for files
-> 500 files compressed to 15, cached (hash=abc123)
-> LLM answers with 15 files
Turn 5: User asks "What about the auth middleware?"
-> Context Tracker detects "auth" may match cached content
-> Proactively expands compressed data
-> LLM finds auth_middleware.py in the full list
```
## Retrieving originals [#retrieving-originals]
CCR works automatically on the Anthropic and OpenAI proxy paths, but you can also retrieve cached data programmatically:
```ts twoslash
import { compress } from "headroom-ai";
import type { CCRConfig } from "headroom-ai";
// CCR is enabled by default when compressing through the proxy.
const result = await compress(messages, {
model: "gpt-4o",
});
// Access compressed messages — CCR markers are embedded automatically
console.log(result.messages);
// CCR configuration options
const ccrConfig: CCRConfig = {
enabled: true,
injectTool: true, // Inject headroom_retrieve tool
injectRetrievalMarker: true, // Add retrieval markers to compressed output
feedbackEnabled: true, // Learn from retrieval patterns
storeMaxEntries: 1000, // Max cached items
storeTtlSeconds: 3600, // Cache TTL
};
```
The full CCR loop -- markers injected, `headroom_retrieve` tool added to the
request, and the model's retrieval call resolved automatically -- only runs
through `headroom proxy` (`headroom/proxy/handlers/*.py` wire tool injection
and the response handler; the direct SDK client does not). Point an OpenAI
client at the proxy to get it:
```python
from openai import OpenAI
# Start `headroom proxy` first (defaults to port 8787); use --no-optimize
# for a passthrough baseline or --no-ccr to disable CCR specifically.
client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key="local")
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
)
```
`HeadroomClient` (the direct SDK integration, no proxy) still routes JSON
tool outputs through SmartCrusher and embeds `<>` markers when a
compression ratio warrants one, and the original is stored and retrievable
via `headroom.cache.compression_store`. It does **not**, however, inject the
`headroom_retrieve` tool or intercept the model's retrieval call for you --
that wiring lives only in the proxy's request handlers. Resolve markers
yourself with `headroom.ccr.marker_resolution.resolve_markers_in_text` if
you're integrating CCR outside the proxy.
## Retention [#retention]
Proxy CCR originals are kept for 1800 seconds (30 minutes) by default. For longer autonomous
agent runs, set `HEADROOM_CCR_TTL_SECONDS` before starting the proxy:
```bash
HEADROOM_CCR_TTL_SECONDS=7200 headroom proxy
```
Check the effective setting at `/v1/retrieve/stats` under
`store.default_ttl_seconds`.
## Message-level CCR [#message-level-ccr]
> **Retired:** The "Message-level CCR via IntelligentContext" feature (where `IntelligentContext` would store dropped messages in CCR with a retrieval marker) was part of the `IntelligentContextConfig` API that has since been removed. Context management is now handled automatically by the pipeline without a separate configurable IntelligentContext stage. Tool-output CCR via SmartCrusher and ContentRouter remains fully supported.
## CCR-enabled components [#ccr-enabled-components]
| Component | What it compresses | CCR integration |
| ----------------- | -------------------------------- | ------------------------------------------- |
| **SmartCrusher** | JSON arrays (tool outputs) | Stores original array, marker includes hash |
| **ContentRouter** | Code, logs, search results, text | Stores original content by strategy |
## Why CCR matters [#why-ccr-matters]
| Approach | Risk | Savings |
| ----------------------- | ----------------- | ---------------------------------------------------------------------------- |
| No compression | None | 0% |
| Traditional compression | Data loss | Depends on content and target ratio |
| CCR compression | None (reversible) | Same range as traditional compression -- CCR changes the risk, not the ratio |
CCR gives you the savings of aggressive compression with zero risk: SmartCrusher, LogCompressor, SearchCompressor, and Kompress can all compress as aggressively as their configuration allows, because the original is always one retrieval away. See [How Compression Works](/docs/how-compression-works#content-type-detection) for measured savings ranges per compressor.
## Purpose [#purpose]
This page is the quick visual map for Headroom automation. Use it when opening a PR, reviewing a PR, cutting a release, or deciding which workflow owns a failure.
The short version:
* Pull requests are gated by PR governance, path-filtered CI, targeted e2e workflows, and human review.
* Release publishing is not triggered by every merge to `main`. `release-please` maintains a release PR; merging that release PR creates the tag and GitHub Release that trigger publishing.
* Docker images are built as multi-architecture digests first, then merged into tagged manifests.
* Docs are deployed by Vercel's own Git integration, not by a GitHub Actions job -- `.github/workflows/docs.yml` ("Validate Docs") only builds the Next.js/Fumadocs app on pull requests touching `docs/**` to catch a broken build before merge; it deploys nothing.
## PR Flow [#pr-flow]
```mermaid
flowchart TD
A[Open, edit, synchronize, or mark PR ready] --> B[PR Governance]
B --> B1{Template complete and ready?}
B1 -- no --> B2[Add needs author action label and governance comment]
B1 -- yes --> B3[Add ready for review label when no blocking status exists]
A --> C[Path filters decide workflow surface]
C --> D{Code paths changed?}
D -- yes --> E[CI changes job]
E --> F[lint: ruff, format, mypy]
E --> G[build-wheel: Rust extension wheel]
E --> H[prefetch-model: Hugging Face cache]
G --> I[test shards 1 to 4]
H --> I
G --> J[test-extras, test-agno, dashboard UI]
E --> K[build: release-profile wheel and sdist smoke]
C --> L{E2E paths changed?}
L -- yes --> M[Docker native e2e and platform wrapper checks]
C --> N{Init or wrap paths changed?}
N -- yes --> O[Init E2E, Wrap E2E, native init or wrap smoke tests]
C --> P{Rust paths changed?}
P -- yes --> Q[Rust fmt, clippy, tests, wheel build, audit]
C --> R{Devcontainer paths changed?}
R -- yes --> S[Devcontainer validation and linked worktree smoke]
C --> T{Release-critical paths changed?}
T -- yes --> U[Release dry-run: wheel matrix and smoke-import gates]
C --> V{Workflow files changed?}
V -- yes --> W[Workflow validation with actionlint and act dry-run]
F --> X{All required checks green?}
I --> X
J --> X
K --> X
M --> X
O --> X
Q --> X
S --> X
U --> X
W --> X
B3 --> X
X -- no --> Y[Fix, rebase, or request changes]
X -- yes --> Z[Human review and merge when approved]
```
## PR Decision Tree [#pr-decision-tree]
```mermaid
flowchart TD
A[Review an open PR] --> B{Draft?}
B -- yes --> C[Do not approve. Comment with remaining readiness steps.]
B -- no --> D{Governance label says needs author action?}
D -- yes --> E[Fix or ask author to complete template and real behavior proof]
D -- no --> F{Merge state dirty or behind?}
F -- dirty --> G[Resolve conflicts before reviewing final code]
F -- behind --> H[Rebase or update branch, then rerun checks]
F -- clean or unknown --> I{Any failing check?}
I -- yes --> J[Read failing logs, classify as stale-main, infra, or code bug]
J --> K{Can maintainer safely fix without changing author intent?}
K -- yes --> L[Patch, test locally, push with lease]
K -- no --> M[Request changes with exact file and line references]
I -- no --> N{Code review complete?}
N -- no --> O[Review diff, tests, docs, and behavior proof]
N -- yes --> P[Approve]
```
## Fork Workflow Approval [#fork-workflow-approval]
GitHub may leave product workflows in `action_required` for first-time or fork contributors. Approve only after the diff is safe enough to execute in CI.
```mermaid
flowchart TD
A[Fork PR has action_required workflows] --> B{Diff is understandable and not suspicious?}
B -- no --> C[Do not approve. Ask for changes or close if unsafe.]
B -- yes --> D{Workflow runs use pull_request with read-scoped token?}
D -- no --> E[Inspect workflow permissions before approving]
D -- yes --> F[Approve queued CI runs]
F --> G[Wait for fresh check results on latest head SHA]
```
## Release Flow [#release-flow]
```mermaid
flowchart TD
A[Merge ordinary PR to main] --> B[Release Please on push to main]
B --> C{Commit is releasable?}
C -- docs, ci, chore only --> D[No release PR change]
C -- fix, feat, breaking change --> E[Create or update Release PR]
E --> F[Release PR contains version bump and changelog]
F --> G{Ready to ship?}
G -- no --> H[Keep merging regular PRs; bot updates Release PR]
G -- yes --> I[Merge Release PR]
I --> J[release-please tags vX.Y.Z and publishes GitHub Release]
J --> K[release.yml starts on release: published]
K --> L[detect-version]
L --> M[build: sync versions, verify versions, changelog, npm packs]
M --> N[build-wheels matrix]
N --> O[collect-dist]
N --> P[smoke-import wheels]
O --> Q{Smoke import green?}
P --> Q
Q -- no --> R[Stop before publishing broken wheels]
Q -- yes --> S[publish PyPI]
M --> T[publish npm packages]
M --> U[publish GitHub Package Registry packages]
P --> V[publish Docker images through docker.yml]
S --> W{PyPI published or PYPI_SKIP=true?}
W -- no --> X[Do not update release assets]
W -- yes --> Y[Create or update GitHub Release assets and notes]
T --> Y
U --> Y
V --> Y
```
## Release Decision Tree [#release-decision-tree]
```mermaid
flowchart TD
A[Need a release?] --> B{Release PR exists?}
B -- no --> C[Merge at least one releasable conventional commit to main]
B -- yes --> D{Release PR checks green and changelog correct?}
D -- no --> E[Fix source PRs or release config, then let release-please update]
D -- yes --> F{Registry skip variables needed?}
F -- yes --> G[Set PYPI_SKIP, NPM_SKIP, or GH_PACKAGES_SKIP deliberately]
F -- no --> H[Merge Release PR]
G --> H
H --> I[Watch release.yml]
I --> J{Failure before publish?}
J -- yes --> K[Fix and rerun before any package is public]
J -- no --> L{Failure after partial publish?}
L -- yes --> M[Use rerun or skip variables to reach consistent GitHub Release state]
L -- no --> N[Release complete]
```
## Docker Publish Flow [#docker-publish-flow]
`docker.yml` can run directly on `push`, `workflow_dispatch`, or `release: published`, and it is also called by `release.yml`.
```mermaid
flowchart TD
A[Docker workflow starts] --> B[Matrix: variant x architecture]
B --> C[Build each platform image by digest only]
C --> D[Smoke-test image imports pydantic_core and headroom._core]
D --> E{Smoke test green?}
E -- no --> F[Stop before tag manifest]
E -- yes --> G[Upload digest marker]
G --> H[Per-variant manifest merge]
H --> I[Apply tags to multi-arch manifest]
I --> J{Release event?}
J -- yes --> K[Version tags and latest retag rules]
J -- no --> L[Branch, PR, dev, or manual tags as configured]
```
## Docs Validate + Deploy Flow [#docs-validate--deploy-flow]
`.github/workflows/docs.yml` ("Validate Docs") only validates -- it has no
deploy step. Deployment of the Next.js/Fumadocs site at
`docs.headroomlabs.ai` is owned entirely by Vercel's own Git integration,
outside GitHub Actions (`.github/workflows/docs.yml:1-9`).
```mermaid
flowchart TD
A[Docs change in PR] --> B{docs/** or docs.yml changed?}
B -- no --> C[Validate Docs workflow does not run]
B -- yes --> D[Validate Docs workflow: npm ci and npm run build]
D --> E{Build green?}
E -- no --> F[Fix before merge]
E -- yes --> G[PR review and normal checks]
G --> H[Merge to main]
H --> I[Vercel Git integration builds and deploys, independent of GitHub Actions]
```
## Manual Validation Flow [#manual-validation-flow]
Use this when editing workflows or release automation.
```mermaid
flowchart TD
A[Edit workflow or release scripts] --> B[Run local workflow validation]
B --> C[actionlint]
B --> D[act dry-run fixtures]
C --> E{Local validation green?}
D --> E
E -- no --> F[Fix before opening or updating PR]
E -- yes --> G[Push PR]
G --> H[workflow-validation job reruns same validation in CI]
```
Recommended local command:
```bash
bash scripts/validate-workflows.sh
```
For release dry-runs:
```bash
act workflow_dispatch -W .github/workflows/release.yml -e .github/act/dry-run.json
```
## Gate Summary [#gate-summary]
| Flow | Trigger | Main gates | Success condition |
| --------------- | ------------------------------------------ | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| PR governance | `pull_request_target`, schedule, manual | Template, readiness labels, merge state, check labels | PR has no governance blockers |
| CI | PR, push to `main`, manual | Path filter, lint, mypy, wheel build, model prefetch, test shards, package smoke | Required jobs green or path-skipped |
| Rust | Rust paths, schedule | fmt, clippy, cargo test, wheel build, audit | Rust checks green; nightly parity is allowed to fail in Phase 0 |
| E2E | CLI, install, wrap, Docker, package paths | Docker init/wrap, native init/wrap/install, platform smoke | Relevant lifecycle checks green |
| Release dry-run | PRs touching release-critical paths | Version detection, wheel matrix, smoke imports | Publish path can build before merge |
| Release publish | GitHub Release published by release-please | Version sync, changelog, wheels, smoke import, PyPI gate, npm, GPR, Docker | Public packages and GitHub Release assets are consistent |
| Docs validate | PR touching `docs/**`, manual | Next.js/Fumadocs build | Build succeeds; deploy itself is done by Vercel's Git integration, not this workflow |
## Workflow Ownership [#workflow-ownership]
| Workflow | Owns |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| `.github/workflows/pr-health.yml` | PR body governance, readiness labels, rebase/conflict/failing-check labels |
| `.github/workflows/ci.yml` | Python lint, type checks, wheel build, test shards, package smoke, workflow validation |
| `.github/workflows/rust.yml` | Rust workspace quality gates and native wheel smoke artifacts |
| `.github/workflows/init-e2e.yml` | Dockerized `headroom init` behavior |
| `.github/workflows/wrap-e2e.yml` | Dockerized `headroom wrap` behavior |
| `.github/workflows/init-native-e2e.yml` | Host-specific `headroom init -g` smoke tests |
| `.github/workflows/install-native-e2e.yml` | Host-specific install CLI smoke tests |
| `.github/workflows/wrap-native-e2e.yml` | Host-specific wrap prepare-only smoke tests |
| `.github/workflows/devcontainers.yml` | Devcontainer startup and linked worktree compatibility |
| `.github/workflows/release-please.yml` | Release PR aggregation from conventional commits |
| `.github/workflows/release.yml` | Release build, wheel smoke-import gates, registry publishing, GitHub Release assets |
| `.github/workflows/docker.yml` | GHCR multi-architecture image builds and manifests |
| `.github/workflows/docs.yml` | Validates the Next.js/Fumadocs build on docs PRs; deploys nothing -- Vercel's own Git integration owns the actual deploy |
If your Claude models live on **Azure AI Foundry**, you can still get Headroom's
prompt compression. Headroom sits between Claude Code and Azure: it shrinks the
big stuff in each request (file reads, logs, tool output) and forwards the rest to
Azure using **your own Azure credentials**. You keep your Azure setup; Headroom
just makes each call cheaper.
## What you get [#what-you-get]
* **Fewer input tokens** on every Claude Code request to Azure AI Foundry —
large file reads, logs, and tool output are compressed before forwarding —
so you pay for less.
* **Same answers** — compression is reversible and content-aware.
* **No new secrets** — Headroom never holds your Azure credentials. Claude Code
keeps authenticating to Azure AI Foundry with its own `api-key` or Entra Bearer
token; Headroom passes it through.
## Before you start [#before-you-start]
You should already have Claude Code working against Azure AI Foundry **without**
Headroom. That means these are set in your shell (or your `~/.claude/settings.json`
`env` block):
```bash
export CLAUDE_CODE_USE_FOUNDRY=1
export ANTHROPIC_FOUNDRY_RESOURCE=
# e.g. ANTHROPIC_FOUNDRY_RESOURCE=my-org-claude
```
No `ANTHROPIC_API_KEY` is needed — Foundry mode uses your Azure credentials.
## Run it (one command) [#run-it-one-command]
```bash
pip install "headroom-ai[proxy]"
headroom wrap claude
```
That's it. Because `CLAUDE_CODE_USE_FOUNDRY=1` is set, `headroom wrap claude`
automatically:
1. derives your Azure AI Foundry endpoint from `ANTHROPIC_FOUNDRY_RESOURCE`,
2. starts the Headroom proxy with that endpoint as the upstream,
3. points Claude Code's Foundry endpoint at the proxy (`ANTHROPIC_FOUNDRY_BASE_URL`),
4. leaves your Azure resource, model, and credentials untouched.
You'll see a line like:
```
Foundry mode: ANTHROPIC_FOUNDRY_BASE_URL=http://127.0.0.1:8787/anthropic
→ upstream https://my-org-claude.services.ai.azure.com/anthropic
```
Use Claude Code exactly as you normally would.
## How it works [#how-it-works]
```
Claude Code ──(Foundry request)──▶ Headroom ──(compressed)──▶ Azure AI Foundry (Claude)
in Foundry mode compresses your resource
(your api-key / Entra) ──────── passed through ───────────▶ authenticates you
```
Claude Code sends its Foundry request (Anthropic API format) to Headroom. Headroom
compresses the messages, then forwards to your Azure AI Foundry resource endpoint —
`https://{ANTHROPIC_FOUNDRY_RESOURCE}.services.ai.azure.com/anthropic` — with your
auth headers passed through unchanged.
## If you have ANTHROPIC\_FOUNDRY\_BASE\_URL set explicitly [#if-you-have-anthropic_foundry_base_url-set-explicitly]
If your environment already has `ANTHROPIC_FOUNDRY_BASE_URL` set to the full Azure
endpoint URL, Headroom uses it directly and `ANTHROPIC_FOUNDRY_RESOURCE` is not
needed. Either configuration works.
## Check that compression is working [#check-that-compression-is-working]
1. Open the dashboard: [http://localhost:8787/dashboard](http://localhost:8787/dashboard).
"Tokens saved" should climb as you use Claude Code.
2. Or check response headers: `x-headroom-tokens-before`, `x-headroom-tokens-after`,
`x-headroom-tokens-saved`.
## Troubleshooting [#troubleshooting]
**Headroom says "ANTHROPIC\_BASE\_URL" instead of "Foundry mode"**
`CLAUDE_CODE_USE_FOUNDRY` is not set in the shell where you ran `headroom wrap
claude`. Make sure to export it before running, or set it in your shell profile.
**Claude Code fails with a 401 / auth error**
Your Azure credentials are not being forwarded correctly. Verify that Claude Code
works against Azure AI Foundry directly (without Headroom) before wrapping. If it
works direct but not through Headroom, open an issue with the proxy log.
**"tokens saved" is always 0**
Check the [dashboard](http://localhost:8787/dashboard) — if requests are flowing
but savings are 0, content may be below the compression threshold or the Rust
extension may not be installed (`pip install "headroom-ai[proxy]"` includes it).
If your Claude models live on **Google Vertex AI**, you can still get Headroom's
prompt compression. Headroom sits between Claude Code and Vertex: it shrinks the
big stuff in each request (file reads, logs, tool output) and forwards the rest to
Vertex using **your own Google credentials**. You keep your GCP setup; Headroom
just makes each call cheaper.
## What you get [#what-you-get]
* **Fewer input tokens** on every Claude Code request to Vertex — large file
reads, logs, and tool output are compressed before forwarding — so you pay
Vertex for less.
* **Same answers** — compression is reversible and content-aware.
* **No new secrets** — Headroom never holds your Google credentials. Claude Code
keeps authenticating to Vertex with its own ADC token; Headroom passes it through.
## Before you start [#before-you-start]
You should already have Claude Code working against Vertex **without** Headroom.
That means these are set in your shell:
```bash
export CLAUDE_CODE_USE_VERTEX=1
export ANTHROPIC_VERTEX_PROJECT_ID=
export CLOUD_ML_REGION=us-east5 # your Vertex region (or "global")
gcloud auth application-default login # or set GOOGLE_APPLICATION_CREDENTIALS
```
No `ANTHROPIC_API_KEY` is needed — Vertex mode uses your Google login.
## Run it (one command) [#run-it-one-command]
```bash
pip install "headroom-ai[proxy]"
headroom wrap claude
```
That's it. Because `CLAUDE_CODE_USE_VERTEX=1` is set, `headroom wrap claude`
automatically:
1. starts the Headroom proxy,
2. points Claude Code's Vertex endpoint at it (`ANTHROPIC_VERTEX_BASE_URL`),
3. leaves your project, region, and Google login untouched.
You'll see a line like:
```
Vertex mode: ANTHROPIC_VERTEX_BASE_URL=http://127.0.0.1:8787
→ compress, then forward to Vertex with your GCP ADC token
```
Use Claude Code exactly as you normally would.
## How it works [#how-it-works]
```
Claude Code ──(Vertex request)──▶ Headroom ──(compressed)──▶ Vertex AI (Claude)
in Vertex mode compresses your project + region
(your ADC token) ───────────── passed through ───────────▶ authenticates you
```
Claude Code sends its normal Vertex `…:rawPredict` / `:streamRawPredict` request to
Headroom. Headroom compresses the messages (keeping the Vertex request shape
intact), then forwards to the correct regional Vertex host — derived from the
request itself, so multi-region and `global` both work — using the Google token
Claude Code already attached.
## Check that compression is working [#check-that-compression-is-working]
1. Open the dashboard: [http://localhost:8787/dashboard](http://localhost:8787/dashboard).
"Tokens saved" should climb as you use Claude Code.
2. Or look at the response headers on a request: `x-headroom-tokens-before`,
`x-headroom-tokens-after`, `x-headroom-tokens-saved`.
If "tokens saved" stays at 0 on large prompts, see Troubleshooting below.
## Troubleshooting [#troubleshooting]
* **It still goes straight to Google (no savings).** Make sure `CLAUDE_CODE_USE_VERTEX=1`
is exported *in the same shell* before `headroom wrap claude`. The wrapper only
switches to Vertex mode when it sees that variable.
* **Wrong region / 404 from Vertex.** Confirm `CLOUD_ML_REGION` matches a region
where your Claude model is enabled. `global` is supported and maps to the
non-regional host.
* **Auth errors.** Headroom forwards your token as-is — if `gcloud auth
application-default login` (or `GOOGLE_APPLICATION_CREDENTIALS`) works for Claude
Code without Headroom, it works with it.
## Alternative: let Headroom talk to Vertex for you [#alternative-let-headroom-talk-to-vertex-for-you]
If you'd rather **not** run Claude Code in Vertex mode, you can have Headroom be the
translator instead: Claude Code speaks plain Anthropic to Headroom, and Headroom
calls Vertex on your behalf.
```bash
export HEADROOM_BACKEND=litellm-vertex_ai # note the _ai suffix
export HEADROOM_REGION=us-east5
export VERTEXAI_PROJECT=
export GOOGLE_APPLICATION_CREDENTIALS=/path/sa.json # or gcloud ADC
export ANTHROPIC_API_KEY=placeholder # Claude Code needs *a* key to start
headroom wrap claude --backend litellm-vertex_ai --region us-east5
```
The native Vertex-mode flow above is recommended — it keeps your existing GCP auth
and has the smallest moving parts. Use this alternative only if you can't set
`CLAUDE_CODE_USE_VERTEX`.
## Notes [#notes]
* Pick a Claude model that is enabled in your Vertex project/region
(e.g. `claude-sonnet-4-6`, `claude-haiku-4-5`).
* Streaming, tool use, and prompt caching all work through Headroom.
* Want to point at a private Vertex gateway instead of Google's host? Start the
proxy with `--vertex-api-url https://your-gateway` and Headroom will forward there.
Headroom's CodeAwareCompressor uses tree-sitter to parse source code into an AST, then selectively compresses function bodies while preserving the structural elements that LLMs need -- imports, signatures, type annotations, and error handlers.
## Why AST-Aware Compression? [#why-ast-aware-compression]
Naive truncation breaks code. Cutting a function in half leaves invalid syntax that confuses the LLM. CodeAwareCompressor guarantees:
* **Syntax validity** -- output always parses correctly
* **Structural preservation** -- imports, signatures, types, decorators are kept intact
* **Lightweight** -- \~50MB of tree-sitter parsers, loaded lazily and cached
## Supported Languages [#supported-languages]
| Tier | Languages | Support Level |
| ------ | ------------------------------- | ------------------------- |
| Tier 1 | Python, JavaScript, TypeScript | Full AST analysis |
| Tier 2 | Go, Rust, Java, C, C++, C#, PHP | Function body compression |
## What Gets Preserved vs Compressed [#what-gets-preserved-vs-compressed]
**Always preserved:**
* Import statements
* Function and method signatures
* Class definitions
* Type annotations
* Decorators
**Compressed:**
* Function bodies (implementations)
* Comments (unless configured to preserve)
* Verbose docstrings (configurable: full, first line, or removed)
## Example [#example]
```python
from headroom.transforms import CodeAwareCompressor, CodeCompressorConfig
# min_tokens_for_compression defaults to 100; a snippet this small needs an
# explicit override or it passes through unmodified (see "Skip small content" below).
compressor = CodeAwareCompressor(CodeCompressorConfig(min_tokens_for_compression=10))
code = '''
import os
from typing import List
def process_items(items: List[str]) -> List[str]:
"""Process a list of items."""
results = []
for item in items:
if not item:
continue
processed = item.strip().lower()
results.append(processed)
return results
'''
result = compressor.compress(code, language="python")
print(result.compressed)
# import os
# from typing import List
#
# def process_items(items: List[str]) -> List[str]:
# """Process a list of items."""
# results = []
# # [6 lines omitted]
# pass
# # [32 tokens compressed. Retrieve more: hash=. Expires in 5m.]
print(f"Compression: {result.compression_ratio:.0%}") # ~57%
print(f"Syntax valid: {result.syntax_valid}") # True
```
## Configuration [#configuration]
```python
from headroom.transforms import CodeAwareCompressor, CodeCompressorConfig, DocstringMode
config = CodeCompressorConfig(
preserve_imports=True, # Always keep imports
preserve_signatures=True, # Always keep function signatures
preserve_type_annotations=True, # Keep type hints
preserve_decorators=True, # Keep decorators
docstring_mode=DocstringMode.FIRST_LINE, # FULL, FIRST_LINE, REMOVE
target_compression_rate=0.2, # Keep 20% of tokens
max_body_lines=5, # Lines to keep per function body
min_tokens_for_compression=100, # Skip small content
language_hint=None, # Auto-detect if None
fallback_to_kompress=True, # Use Kompress for unknown langs
)
compressor = CodeAwareCompressor(config)
result = compressor.compress(code)
```
### Configuration Options [#configuration-options]
| Option | Default | Description |
| ---------------------------- | ------------ | -------------------------------------------------------- |
| `preserve_imports` | `True` | Keep all import statements |
| `preserve_signatures` | `True` | Keep function/method signatures |
| `preserve_type_annotations` | `True` | Keep type hints |
| `preserve_decorators` | `True` | Keep decorators |
| `docstring_mode` | `FIRST_LINE` | How to handle docstrings: `FULL`, `FIRST_LINE`, `REMOVE` |
| `target_compression_rate` | `0.2` | Fraction of tokens to keep (0.2 = keep 20%) |
| `max_body_lines` | `5` | Max lines to keep per function body |
| `min_tokens_for_compression` | `100` | Skip files smaller than this |
| `language_hint` | `None` | Override language detection |
| `fallback_to_kompress` | `True` | Use Kompress for unsupported languages |
## Before and After [#before-and-after]
```python
# Before (full source file)
def process_data(items: List[str]) -> Dict[str, int]:
"""Process items and count occurrences."""
result = {}
for item in items:
item = item.strip().lower()
if item in result:
result[item] += 1
else:
result[item] = 1
return result
# After (signature preserved, body compressed)
def process_data(items: List[str]) -> Dict[str, int]:
"""Process items and count occurrences."""
result = {}
for item in items:
# ... (5 lines compressed)
pass
```
The LLM sees the function's purpose, its input/output types, and the general approach -- enough to reason about the code without needing every implementation line.
## Installation [#installation]
```bash
# Install tree-sitter language pack
pip install "headroom-ai[code]"
```
## Memory Management [#memory-management]
Tree-sitter parsers are lazy-loaded and cached. You can free memory when done:
```python
from headroom.transforms import is_tree_sitter_available
from headroom.transforms.code_compressor import unload_tree_sitter
# Check if tree-sitter is installed
print(is_tree_sitter_available()) # True
# Free memory when done
unload_tree_sitter()
```
## Performance [#performance]
| Metric | Value |
| --------------- | ---------------------------- |
| Compression | 40-70% token reduction |
| Speed | \~10-50ms per file |
| Memory | \~50MB (tree-sitter parsers) |
| Syntax validity | Guaranteed |
ContentRouter always *detects* source code automatically, but whether it routes that code to CodeAwareCompressor depends on the entry point. `headroom proxy` enables code-aware routing by default (`HEADROOM_CODE_AWARE_ENABLED` defaults to on; disable with `--no-code-aware` or `HEADROOM_CODE_AWARE_ENABLED=0`). The one-function `compress()` API and `HeadroomClient`, however, build `ContentRouterConfig` with its own default, `enable_code_aware=False` -- there, detected code falls back to Kompress unless you construct your own pipeline with `ContentRouterConfig(enable_code_aware=True)`. Direct usage of `CodeAwareCompressor`, as shown on this page, is unaffected either way. See [How Compression Works](/docs/how-compression-works#content-type-detection).
Older versions of `headroom wrap codex` could run Codex with a temporary `CODEX_HOME` named `headroom-codex-home-*`. Chats and configuration created during that wrapped session stayed in the temporary home, so they disappeared from the normal Codex history after the wrapper exited. This recovery migrates the retained state back into the durable Codex home without deleting either source.
This procedure can recover temporary homes that still exist and pinned sources retained by an interrupted recovery. A temporary home that was already deleted cannot be reconstructed unless one of those retained copies exists. Headroom reports deleted temporary homes still referenced by the Codex thread database. It does not treat paths pasted into prompts or error messages as recovery sources. See [issue #2159](https://github.com/headroomlabs-ai/headroom/issues/2159) for the regression details.
## Before you recover [#before-you-recover]
Close Codex and any process using the temporary or durable Codex homes. Recovery detects changes made while creating each backup and aborts, but quiet homes are required for a consistent migration.
The target defaults to `$CODEX_HOME` when it is set, otherwise `~/.codex`. Check that this is the durable home you normally use before confirming the migration.
## Automatic recovery [#automatic-recovery]
The first interactive `headroom wrap codex` with the fixed version searches Python's temporary directory, `$TMPDIR`, `/tmp`, `/private/tmp`, and the macOS `/private/var/folders/*/*/T` roots for non-empty `headroom-codex-home-*` directories. If it finds any, it lists them and offers to back up and recover them before Codex starts.
Declining the prompt changes nothing. You can run the manual command later.
## Manual recovery [#manual-recovery]
Let Headroom discover retained temporary homes and `source-pinned/` copies left by interrupted or failed recovery attempts, then preview the migration:
```bash
headroom recover codex
```
If no recoverable copy remains, the command audits the durable Codex thread database, rollout files, and `history.jsonl`. It reports indexed active and archived chats, surviving rollouts missing from the thread index, and history-only records whose rollouts no longer exist. History-only prompt text cannot reconstruct a full chat transcript.
Codex filters the default resume picker by the current working directory. Run `codex resume --all` yourself to display indexed chats from every working directory. Headroom does not launch Codex during recovery.
To select a known temporary home and an explicit durable target:
```bash
headroom recover codex \
--source /path/to/headroom-codex-home-12345 \
--target "${CODEX_HOME:-$HOME/.codex}"
```
Repeat `--source` to merge multiple homes. Headroom shows the target and every source before asking for confirmation. Use `--yes` only in automation where those paths have already been reviewed.
## What gets migrated [#what-gets-migrated]
When a source used the localhost `headroom` model provider injected by the old wrapper, recovery rewrites that provider in both SQLite thread rows and rollout `session_meta` records to the active target provider. This also repairs newer target records left by an earlier broken recovery. A user-defined remote provider named `headroom` is preserved.
* `history.jsonl` and other JSONL indexes are combined without duplicate records. Malformed input is excluded from the result and copied to the backup quarantine.
* Session rollouts under `sessions/` and `archived_sessions/` keep the newer file when the same path exists in both homes.
* SQLite databases are merged only when their tables, indexes, triggers, views, and migration checksums are compatible. Primary-key conflicts keep rows from the newer database. Recovered thread rows are rewritten to the durable rollout path, including rows restored from a pinned source after the original temporary home was deleted.
* `config.toml` tables are merged recursively. Values from the newer config win, and localhost Headroom routing injected by the old wrapper is removed. A user-defined remote provider is preserved.
* Other files, including credentials and user settings, keep the newer copy. The durable target wins equal modification-time ties, and missing source files never delete target files.
Sockets, lock files, SQLite journals, FIFOs, and other runtime-only artifacts are recorded but not copied.
## Backups and rollback [#backups-and-rollback]
Every source is pinned before migration. The existing durable home, including its current Codex configuration, is backed up before any merge begins. Backups are owner-only and retained next to the target:
```text
/.headroom-codex-recovery//
├── source-pinned/
├── target-before/
├── target-failed/ # only when a merge is rolled back
├── manifest.json
└── quarantine/ # only when malformed input is found
```
`target-before/` is absent when the target did not exist before recovery. `manifest.json` records copied, merged, quarantined, and skipped paths.
If configuration parsing, SQLite schema validation, integrity checks, foreign-key checks, or filesystem writes fail, Headroom atomically renames the failed target to `target-failed/` before restoring `target-before/`. This avoids recursive-deletion races with SQLite runtime files and preserves the failed merge for inspection. When the target did not exist before recovery, the partial target is retained only as `target-failed/`. The pinned source and recovery backup remain available for inspection.
Recovery also refuses overlapping source and target paths, target symlink traversal, and a source that changes while it is being pinned.
## After recovery [#after-recovery]
The command prints the retained backup path after each successful source merge. Review its `manifest.json`, then start Codex normally and confirm that the recovered chats and settings appear. Keep the backup until you have verified the durable history.
Fixed versions of `headroom wrap codex` run Codex against the durable home and apply proxy routing only to the launched process, so new wrapped sessions remain visible after Headroom exits.
Headroom can be configured via the SDK constructor, proxy command line, environment variables, or per-request overrides.
## Runtime Rollout Channels [#runtime-rollout-channels]
Headroom uses rollout channels to control which behaviors an already-installed
artifact may expose. They do not select a package or released version.
| Variable | Default | Purpose |
| ----------------------------------------- | -------- | -------------------------------------------------------------------------------- |
| `HEADROOM_ROLLOUT_CHANNEL` | `stable` | Selects `stable`, `beta`, `canary`, or `dev`. |
| `HEADROOM_FEATURES` | unset | Comma-separated feature names to request explicitly. |
| `HEADROOM_DISABLE_FEATURES` | unset | Comma-separated feature names to force off. Disable wins over every enable path. |
| `HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES` | unset | Break-glass override for emergency mitigation only. |
See [Runtime Rollouts](/docs/runtime-rollouts) for policy, provenance, and
contributor rules.
If Codex history disappeared after using an older wrapper, see [Recover Codex State](/docs/codex-recovery) before wrapping Codex again.
## SDK Modes (`default_mode` / `headroom_mode`) [#sdk-modes-default_mode--headroom_mode]
These modes apply to SDK usage via `HeadroomClient(default_mode=...)` or per-request `headroom_mode=...`. They are **not** the same as the proxy `--mode` flag.
| Mode | Behavior | Use Case |
| ---------- | -------------------------------------- | ------------------------------------------- |
| `audit` | Observes and logs, no modifications | Production monitoring, baseline measurement |
| `optimize` | Applies safe, deterministic transforms | Production optimization |
| `simulate` | Returns plan without API call | Testing, cost estimation |
> **Proxy `--mode` is a separate axis**: `--mode cache` (the default) freezes prior turns so the provider's prefix cache is never busted, which is where the savings come from on a real agent workload. `--mode token` maximizes visible per-request compression at the cost of cache stability; prefer `cache` unless you are specifically diagnosing compression numbers. The proxy does not accept `audit`, `optimize`, or `simulate`.
## SDK Configuration [#sdk-configuration]
```ts twoslash
import { HeadroomClient } from 'headroom-ai';
// Reads from HEADROOM_BASE_URL and HEADROOM_API_KEY automatically
const client = new HeadroomClient();
// Or configure explicitly
const explicit = new HeadroomClient({
baseUrl: 'http://localhost:8787',
apiKey: 'your-api-key',
timeout: 30_000,
fallback: true,
retries: 2,
});
```
```python
from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
# Mode: "audit" (observe only) or "optimize" (apply transforms)
default_mode="optimize",
# Enable provider-specific cache optimization
enable_cache_optimizer=True,
# Enable query-level semantic caching
enable_semantic_cache=False,
# Override default context limits per model
model_context_limits={
"gpt-4o": 128000,
"gpt-4o-mini": 128000,
},
# Database location (defaults to temp directory)
# store_url="sqlite:////absolute/path/to/headroom.db",
)
```
## Per-Request Overrides [#per-request-overrides]
Override configuration for individual requests:
```ts twoslash
import { compress } from 'headroom-ai';
const result = await compress(messages, {
model: 'gpt-4o',
tokenBudget: 100_000,
timeout: 15_000,
});
```
```python
response = client.chat.completions.create(
model="gpt-4o",
messages=[...],
# Override mode for this request
headroom_mode="audit",
# Reserve more tokens for output
headroom_output_buffer_tokens=8000,
# Keep last N turns (don't compress)
headroom_keep_turns=5,
# Skip compression for specific tools
headroom_tool_profiles={
"important_tool": {"skip_compression": True}
},
)
```
### Proxy upstream override (`x-headroom-base-url`) [#proxy-upstream-override-x-headroom-base-url]
When using the proxy, send the `x-headroom-base-url` request header to route a
single request to a different upstream instead of the configured provider URL.
This lets a client that speaks a provider's wire format authenticate against a
compatible gateway (for example an OpenAI-compatible endpoint, or an
Anthropic-Messages gateway such as OpenCode Zen) without changing the proxy
configuration.
The header is honored by the OpenAI-compatible routes, the Anthropic Messages
route (`POST /v1/messages`), and the generic passthrough route. The proxy
forwards the request to `` + the original request path
(e.g. `/v1/messages`). An empty or whitespace-only value is ignored and the
configured upstream is used.
```bash
curl http://127.0.0.1:8787/v1/messages \
-H "content-type: application/json" \
-H "anthropic-version: 2023-06-01" \
-H "x-headroom-base-url: https://opencode.ai/zen/go" \
-H "x-api-key: " \
-d '{"model":"glm-5.2","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
```
When `HEADROOM_STRIP_INTERNAL_HEADERS` is `enabled` (the default), the proxy
reads this header for routing and then strips it before forwarding upstream.
#### Configured secret headers are not sent to arbitrary upstreams [#configured-secret-headers-are-not-sent-to-arbitrary-upstreams]
`ANTHROPIC_TARGET_API_HEADERS` / `OPENAI_TARGET_API_HEADERS` hold operator
secrets. Because `x-headroom-base-url` is chosen by the *client*, those headers
are only attached when the destination is one the operator designated:
* a host in the configured provider targets (`ANTHROPIC_TARGET_API_URL`,
`OPENAI_TARGET_API_URL`, and the Gemini/Vertex/Cloud Code equivalents), or
* a host listed in `HEADROOM_UPSTREAM_ALLOWED_HOSTS` (comma-separated).
A request to any other upstream is **still proxied** — it just does not carry
your configured headers, and the proxy logs
`upstream_extra_headers_withheld host=` once per host. If you route to a
gateway via this header and need your configured headers to reach it, add its
host to `HEADROOM_UPSTREAM_ALLOWED_HOSTS`:
```bash
export HEADROOM_UPSTREAM_ALLOWED_HOSTS="gateway.internal,api.example-gateway.ai"
```
Matching is on the parsed hostname and is exact — no wildcards — so
`api.anthropic.com.evil.example` and `https://api.anthropic.com@evil.example`
do not match `api.anthropic.com`.
## SmartCrusher Configuration [#smartcrusher-configuration]
Fine-tune JSON compression behavior:
```python
from headroom.transforms import SmartCrusherConfig
config = SmartCrusherConfig(
# Maximum items to keep after compression
max_items_after_crush=15,
# Minimum tokens before applying compression
min_tokens_to_crush=200,
# Fraction of items always kept from the start/end
first_fraction=0.3,
last_fraction=0.15,
# Variance threshold for statistical analysis
variance_threshold=2.0,
)
```
## CacheAligner Configuration [#cachealigner-configuration]
Control prefix stabilization for provider cache hit rates:
```python
from headroom import CacheAlignerConfig
config = CacheAlignerConfig(
# Enable/disable cache alignment
enabled=True,
# Use custom regex patterns instead of the built-in dynamic-content
# detector (use_dynamic_detector=False activates date_patterns)
use_dynamic_detector=False,
date_patterns=[
r"Today is \w+ \d+, \d{4}",
r"Current time: .*",
],
)
```
## Context Window Management [#context-window-management]
Context management is now automatic. Use per-request overrides to control behavior:
```python
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
# Reserve tokens for model output
headroom_output_buffer_tokens=4000,
# Keep last N turns uncompressed
headroom_keep_turns=3,
)
```
The `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` classes are no longer part of Headroom. Context management now happens automatically inside the pipeline (live-zone-only compression).
### Claude 1M context window (`headroom wrap claude --1m`) [#claude-1m-context-window-headroom-wrap-claude---1m]
`headroom wrap claude --1m` opts a Claude Code session into Anthropic's 1M-token context window by selecting a `[1m]`-suffixed model id, which makes Claude Code send the `context-1m` beta header. The model that `--1m` targets is resolved in this order:
1. an explicit `--model` / `ANTHROPIC_MODEL` value (used as-is, with a `[1m]` suffix appended when missing),
2. otherwise `HEADROOM_1M_MODEL`, when set,
3. otherwise the built-in default (currently `claude-opus-5`).
Set `HEADROOM_1M_MODEL` to point `--1m` at a specific model without pinning `ANTHROPIC_MODEL` globally, so the default can follow a new Opus generation without a code change:
```bash
# Route --1m at a specific model for this shell / session
export HEADROOM_1M_MODEL=claude-opus-5
headroom wrap claude --1m
```
`HEADROOM_1M_MODEL` is a fallback only: an explicit `--model` or `ANTHROPIC_MODEL` always wins. The value may be given with or without the `[1m]` suffix; both `claude-opus-5` and `claude-opus-5[1m]` are accepted, and the suffix is added when absent.
## Pipeline Extensions [#pipeline-extensions]
Use a `headroom.pipeline_extension` entry point when you need to normalize or annotate requests before they leave Headroom. The `PRE_SEND` stage is the right place for provider-specific request cleanup, such as turning `content: null` into `content: ""` for upstreams that reject OpenAI-spec tool-call messages.
```python
from headroom.pipeline import PipelineEvent, PipelineStage
class NormalizeNullContent:
def on_pipeline_event(self, event: PipelineEvent) -> PipelineEvent:
if event.stage is not PipelineStage.PRE_SEND or not event.messages:
return event
for message in event.messages:
if (
message.get("role") == "assistant"
and message.get("content") is None
and message.get("tool_calls")
):
message["content"] = ""
return event
```
Register it in `pyproject.toml`:
```toml
[project.entry-points."headroom.pipeline_extension"]
normalize_null_content = "my_pkg.normalize:NormalizeNullContent"
```
If the upstream base URL itself must vary per request, use the `x-headroom-base-url` override header in addition to the normalization hook.
## Proxy Configuration [#proxy-configuration]
### Command Line Options [#command-line-options]
```bash
headroom proxy \
--port 8787 \ # Port to listen on
--host 0.0.0.0 \ # Host to bind to
--mode cache \ # default; freezes prior turns for prefix-cache stability
--budget 10.00 \ # Daily budget limit in USD
--log-file headroom.jsonl # Log file path
```
### Feature Flags [#feature-flags]
```bash
# Disable optimization (passthrough mode)
headroom proxy --no-optimize
# Disable semantic caching
headroom proxy --no-cache
# Preserve provider prefix-cache stability instead of maximizing token removal
headroom proxy --mode cache
# Enable memory and live learning
headroom proxy --memory
headroom proxy --learn --min-evidence 3
```
## Environment Variables [#environment-variables]
| Variable | Description | Default |
| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| `HEADROOM_HOST` | Proxy bind host | `127.0.0.1` |
| `HEADROOM_PORT` | Proxy bind port | `8787` |
| `HEADROOM_MODE` | Proxy optimization mode: `token` or `cache` | `cache` |
| `HEADROOM_WORKERS` | Uvicorn worker count | `1` |
| `HEADROOM_LIMIT_CONCURRENCY` | Maximum concurrent connections before 503 | `1000` |
| `HEADROOM_MAX_CONNECTIONS` | Maximum upstream HTTP connections | `500` |
| `HEADROOM_MAX_KEEPALIVE` | Maximum upstream keep-alive connections | `100` |
| `HEADROOM_KEEPALIVE_EXPIRY` | Seconds an idle upstream keep-alive connection is kept open | `90` |
| `HEADROOM_HTTP_PROXY` | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT | -- |
| `HEADROOM_BUDGET` | Daily budget limit in USD | -- |
| `HEADROOM_TELEMETRY` | Set to `on` for **local-only** usage stats (powers your own `/stats` and dashboard; nothing is sent externally) | `off` |
| `HEADROOM_STATELESS` | Set to `true` to disable filesystem writes | `false` |
| `HEADROOM_MODEL_LIMITS` | Custom model config (JSON string or file path) | -- |
| `HEADROOM_BASE_URL` | Base URL of the Headroom proxy (TypeScript SDK) | `http://localhost:8787` |
| `HEADROOM_API_KEY` | Optional API key for authenticated Headroom endpoints (TypeScript SDK) | -- |
| `HEADROOM_CONFIG_DIR` | Canonical config (read-mostly) root. Derives `models.json` and per-plugin config paths when set. | `~/.headroom/config` |
| `HEADROOM_WORKSPACE_DIR` | Canonical workspace (read-write state) root. Derives savings, memory DB, logs, TOIN, subscription state, and more when set. | `~/.headroom` |
| `HEADROOM_SAVINGS_PATH` | Override persistent savings file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |
| `HEADROOM_TOIN_PATH` | Override TOIN telemetry file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |
| `HEADROOM_SUBSCRIPTION_STATE_PATH` | Override subscription tracker state file. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |
| `HEADROOM_PERIODIC_TOIN_STATS` | Controls periodic TOIN stats logging in long-lived proxy workers. Set to `0`, `false`, `off`, or `no` to disable the 5-minute stats loop without disabling TOIN learning or request-time feedback. | `true` |
| `HEADROOM_MEMORY_INJECTION_MODE` | Memory-context routing mode: `live_zone_tail` (default) or `disabled`. The legacy `system_prompt` mode was retired by PR-A2; supplying it raises. | `live_zone_tail` |
| `HEADROOM_PROXY_PYTHON_FORWARDER_MODE` | Python forwarder serialization mode. `byte_faithful` (default) forwards original request bytes verbatim when no transform mutated the body and re-serializes canonically only when needed — keeps Anthropic prompt-cache hit-rate intact. `legacy_json_kwarg` is an explicit operator opt-in for emergency rollback to the historical `httpx ... json=body` behavior. NOT a fallback — only flip on explicit operator decision. | `byte_faithful` |
| `HEADROOM_STRIP_INTERNAL_HEADERS` | Python proxy: whether to strip internal `x-headroom-*` request headers (e.g. `x-headroom-bypass`, `x-headroom-mode`, `x-headroom-user-id`, `x-headroom-stack`, `x-headroom-base-url`) before every upstream forwarder call (PR-A5, fixes P5-49). `enabled` (default) stops fingerprinting / leakage. `disabled` is an explicit operator opt-in for diagnostic shadow tracing — NOT a fallback. Inbound reads of these headers (bypass gating, memory user-id resolution) are unaffected because they read `request.headers` directly. | `enabled` |
| `HEADROOM_PROXY_STRIP_INTERNAL_HEADERS` | Rust proxy: same policy as `HEADROOM_STRIP_INTERNAL_HEADERS` but for the Rust transparent proxy. Stripping happens inside `build_forward_request_headers` so both HTTP and WebSocket upstream calls are gated by one flag. `enabled` default; `disabled` operator opt-in for diagnostic shadow tracing. Response-side `X-Headroom-*` injection (e.g. `x-headroom-tokens-saved`) is unrelated and stays. | `enabled` |
| `HEADROOM_EMBEDDER_RUNTIME` | Set to `pytorch_mps` to run the memory embedder via the torch sentence-transformers backend on the Apple GPU (MPS). Only engages when Apple MPS is actually available; otherwise it logs a warning and uses the existing default embedder selection path. `pytorch_mps` is the only accepted value. Requires the `[pytorch-mps]` extra. See [Memory](/docs/memory#embedding-runtime--gpu-offload-apple-silicon). | default embedder selection |
| `HEADROOM_KOMPRESS_BACKEND` | Selects the Kompress (ModernBERT) inference backend, separate from `HEADROOM_EMBEDDER_RUNTIME` above. `auto` tries ONNX CPU then falls back to PyTorch; `onnx`/`onnx_cpu` and `onnx_coreml` force ONNX Runtime (CPU or CoreML); `pytorch` and `pytorch_mps` force PyTorch (`pytorch_mps` targets Apple's GPU, with `mps`/`torch_mps` accepted as aliases). | `auto` |
| `ORT_DYLIB_PATH` | Path to the ONNX Runtime shared library loaded by the Rust core (magika detection, fastembed embeddings), which loads ORT dynamically on every platform. Auto-pinned at `import headroom` to the library inside the `onnxruntime` pip package (`onnxruntime.dll` / `libonnxruntime.so*` / `libonnxruntime*.dylib`); set it yourself to override. Without a pin, ML detection degrades to the non-ONNX tiers — and on Windows the bare DLL search can resolve to the Windows ML System32 build (1.17.x on Win11 24H2+), which deadlocks ONNX session init — see [Troubleshooting](/docs/troubleshooting#windows-ml-content-detection-hangs-or-silently-falls-back). | auto-pinned |
| `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS` | Upper bound (integer seconds, > 0) on magika's one-time ONNX session init in the Rust detection chain. On timeout the init error is cached and detection uses the non-ML fallback tiers for the rest of the process; a warning is logged. Safety net for environments where the dylib pin above does not apply. | `5` |
| `HEADROOM_REQUEST_TIMEOUT` | Request timeout in seconds | `300` |
| `HEADROOM_BETA_HEADER_STICKY` | Controls per-session `anthropic-beta` / `OpenAI-Beta` re-echo. `enabled` (default): the proxy unions beta tokens across turns within a session — if the client sends a token in turn N and omits it in turn N+1, the proxy re-injects it to preserve prefix-cache stability. `disabled`: the client's value is forwarded verbatim with no accumulation. Any other value raises at request time. See [Session Beta Header Tracking](/docs/configuration#session-beta-header-tracking). | `enabled` |
| `HEADROOM_BETA_TRACKER_MAX_SESSIONS` | LRU capacity of the in-memory session beta tracker. Once full, the oldest session entry is evicted. | `1000` |
| `HEADROOM_PROXY_BETA_HEADER_STICKY` | Rust proxy: same per-conversation beta-token union as `HEADROOM_BETA_HEADER_STICKY`, applied to `anthropic-beta` / `openai-beta` on the intercepted `/v1/messages`, `/v1/chat/completions`, and `/v1/responses` routes. Requires the compression interceptor (`HEADROOM_PROXY_COMPRESSION=1`) — with it off the Rust proxy is a strict byte-pipe and this flag has no effect (startup warns). Unlike the Python tracker (keyed on model + system prompt), sessions are keyed per conversation, shared with the cache-drift detector — parallel conversations never inherit each other's tokens. `enabled` default; `disabled` forwards the client value verbatim and keeps no state. Tracker capacity is fixed at 1000 sessions. | `enabled` |
| `HEADROOM_MODEL_ROUTER_ENABLED` | Enable cost-aware model routing. `1`/`true`/`yes`/`on`/`enabled` turns it on and requires `HEADROOM_MODEL_ROUTES`. See [Cost-aware model routing](/docs/configuration#cost-aware-model-routing). | `off` |
| `HEADROOM_MODEL_ROUTES` | JSON array of ordered routing rules for cost-aware model routing (schema below). | -- |
| `HEADROOM_THINKING_COMPACT` | Compact plain-text reasoning that models re-send every turn (Kimi/GLM/DeepSeek `reasoning_content` / inline ``): Kompress it on warm turns, drop it on cold turns. No-op for Claude/Codex/OpenAI (encrypted reasoning). See [Cold-prefix hook](#cold-prefix-hook--reasoning-compaction). | `off` |
| `HEADROOM_THINKING_COMPACT_KEEP_LAST` | Most-recent assistant turns whose reasoning is left intact (the active reasoning the model still uses). Only applies with `HEADROOM_THINKING_COMPACT`. | `1` |
| `HEADROOM_COLD_RECOMPACT` | On a confirmed-cold turn (idle past the real cache TTL), recompact the whole prefix — cross-turn dedupe + superseded-read drop + lossless folds — instead of forwarding a byte-identical prefix to a dead cache. Warm turns unchanged. Pair with `HEADROOM_DEDUPE`. See [Cold-prefix hook](#cold-prefix-hook--reasoning-compaction). | `off` |
| `HEADROOM_DEDUPE` | Whole-conversation verbatim cross-turn dedup in the router (cache-safe, information-preserving via retrieval markers). Superseded-read drop + lossless folds run without it; this adds verbatim dedup. | `off` |
| `HEADROOM_CACHE_TTL_LEARN` | Append per-turn cache-outcome observations (provider, model, idle, hit/miss) to `cache_ttl_observations.jsonl` for the offline `headroom-cache-ttl` learner. Observation-only (no request-behavior change); respects `HEADROOM_STATELESS`; the log is size-bounded. | `off` |
| `HEADROOM_KOMPRESS_ENDPOINT` / `HEADROOM_KOMPRESS_ENDPOINT_TOKEN` | Offload ML compression (Kompress) to a remote endpoint instead of the local ONNX model — used by reasoning compaction and the router when set. | -- |
| `HEADROOM_1M_MODEL` | Fallback model that `headroom wrap claude --1m` targets when neither `--model` nor `ANTHROPIC_MODEL` is set. Accepts the id with or without the `[1m]` suffix (added when absent); an explicit `--model` / `ANTHROPIC_MODEL` always wins. See [Claude 1M context window](#claude-1m-context-window-headroom-wrap-claude---1m). | `claude-opus-5` |
For provider-only proxying, prefer `HEADROOM_HTTP_PROXY` over process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY`. HTTPX reads those global variables, but Headroom also passes them through to tool executions.
### Cold-prefix hook & reasoning compaction [#cold-prefix-hook--reasoning-compaction]
Two related optimizations target tokens that agent harnesses re-send every turn:
model **reasoning** (re-billed as input on Kimi/GLM/DeepSeek and on Claude 4.6+),
and the **prefix** itself once its prompt cache has lapsed. All flags below are
**off by default** — the base proxy is unchanged until you opt in.
**What to set for what:**
| Goal | Set | Notes |
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Shrink Kimi/GLM/DeepSeek reasoning re-sent each turn | `HEADROOM_THINKING_COMPACT=1` (+ `HEADROOM_KOMPRESS_ENDPOINT` for the best ratio) | Warm turns Kompress the reasoning; cold turns drop it. No-op for Claude/Codex/OpenAI (their reasoning is an encrypted handle, already cheap on resend). |
| Recompact a dead-cache prefix instead of re-sending it whole | `HEADROOM_COLD_RECOMPACT=1` (+ `HEADROOM_DEDUPE=1`) | Fires only when the cache has genuinely lapsed. Works in both `cache` and `token` mode. |
| Learn each provider's real cache TTL | `HEADROOM_CACHE_TTL_LEARN=1`, then run the `headroom-cache-ttl` estimator periodically | Sharpens cold detection for providers that don't expose a TTL (Kimi/OpenAI). |
**How cold detection knows the TTL.** A wrong "cold" call would recompact a
*warm* prefix and bust the cache, so the TTL must be right:
* **Claude Code:** read exactly, from the request's `cache_control.ttl` plus CC's
own controls — `ENABLE_PROMPT_CACHING_1H` (1h), `FORCE_PROMPT_CACHING_5M` (5m),
and `DISABLE_PROMPT_CACHING` (+ per-model `DISABLE_PROMPT_CACHING_`),
which turns caching off and makes every turn a free recompaction candidate.
* **Kimi / OpenAI / Codex:** they don't expose a TTL, so detection uses a
conservative default until the learner (`HEADROOM_CACHE_TTL_LEARN` + the
`headroom-cache-ttl` plugin) fills in the real value from observed hits/misses.
**Running the estimator.** With `HEADROOM_CACHE_TTL_LEARN=1` set on the proxy,
run the batch estimator periodically (cron / launchd — it is deliberately
out-of-process so it can never touch the request path):
```bash
headroom-cache-ttl # reads cache_ttl_observations.jsonl, writes cache_ttl_learned.json
headroom-cache-ttl --dry-run # print the estimate table without writing
```
It emits a TTL for a `provider/model` only when the observations bound it on
both sides — hits at some idle prove the cache was still alive (lower bound),
`ttl_expiry` misses at a larger idle prove it was dead (upper bound) — and it
writes the upper end of that interval: overestimating a TTL only skips a
recompaction, while underestimating one would bust a warm cache. Estimates are
never pooled across models: one model's expiry says nothing about another's
TTL, so a cross-model aggregate could close one model's interval with another's
death evidence. Keys without death-evidence beyond their last observed hit are
skipped, as is any model with no estimate of its own; both keep the
conservative static default. The proxy picks up a rewritten table by mtime; no
restart needed.
**Can these be on by default?**
* `HEADROOM_THINKING_COMPACT` — **stays opt-in.** It rewrites model inputs
(reasoning the model actively uses) and depends on Kompress, so its quality /
latency trade should be a deliberate choice.
* `HEADROOM_COLD_RECOMPACT` — **opt-in today; a candidate to default for Claude
Code** once TTL detection is field-validated. It only fires on confirmed-cold
turns and recompacts losslessly, but a mis-read TTL would bust a warm cache
(expensive), so it waits for confidence. For Kimi/OpenAI it should stay opt-in
until the learner has data.
* `HEADROOM_CACHE_TTL_LEARN` — **the safest to default on:** observation-only, a
size-bounded local log, and it respects `HEADROOM_STATELESS`. Kept opt-in for
now so nothing is written to disk unasked.
### Cost-aware model routing [#cost-aware-model-routing]
Complementary to content compression, Headroom can rewrite the upstream model per request to stretch quota and control spend, for example by sending small, tool-free requests to a cheaper model. Routing is opt-in and disabled by default, so behavior is unchanged unless you configure it.
Enable it with `HEADROOM_MODEL_ROUTER_ENABLED=1` and declare ordered rules in `HEADROOM_MODEL_ROUTES` (a JSON array). The router evaluates rules top to bottom and the first rule whose conditions all match wins; every decision is logged with a reason so routing stays observable. Each rule object supports:
| Field | Type | Meaning |
| ------------------ | ----------------- | -------------------------------------------------------------------------- |
| `to_model` | string (required) | Model to route to when the rule matches. |
| `max_input_tokens` | integer | Match only when the estimated input size is at or below this. |
| `min_input_tokens` | integer | Match only when the estimated input size is at or above this. |
| `require_no_tools` | boolean | Match only when the request declares no tools (a proxy for low-risk work). |
| `from_models` | list of strings | Restrict the rule to these source models. Omit for any source model. |
| `name` | string | Label surfaced in the decision log. |
```bash
export HEADROOM_MODEL_ROUTER_ENABLED=1
export HEADROOM_MODEL_ROUTES='[
{"name": "small-no-tools", "max_input_tokens": 4000, "require_no_tools": true,
"from_models": ["claude-sonnet-4-6"], "to_model": "claude-haiku-4-5"}
]'
```
Notes:
* Input size is a fast, tokenizer-free estimate over the messages, tools, and top-level `system` prompt, meant for tier selection rather than exact accounting.
* A malformed rule fails open: it is skipped (never silently widened), and the rest of the rules still apply.
* Routing is skipped for byte-faithful passthrough requests (`x-headroom-bypass: true` or `x-headroom-mode: passthrough`), so those are never model-rewritten.
* Routing currently applies on the Anthropic `/v1/messages` path.
### Session Beta Header Tracking [#session-beta-header-tracking]
When running as a proxy, Headroom maintains a per-session union of `anthropic-beta` (and `OpenAI-Beta`) tokens via `SessionBetaTracker`. The session key is derived from the `x-headroom-session-id` header if present, otherwise from `md5(model + system_prompt[:500])[:16]` — stable across turns of the same conversation.
**Why:** clients such as Claude Code and Codex CLI may drop a beta token between consecutive turns. Because `anthropic-beta` is part of the request bytes that determine the upstream prefix-cache key, a dropped token would bust the cache mid-conversation. The tracker re-injects any token seen earlier in the session so the cache key stays stable.
**Trade-off:** once the proxy has seen a beta token in a session it will continue re-sending it for the rest of that session, even if the client stops including it. Stopping the token on the client side alone is not sufficient — the proxy re-injects it. Set `HEADROOM_BETA_HEADER_STICKY=disabled` to pass the client's `anthropic-beta` value verbatim and bypass this accumulation.
```bash
# Disable sticky beta re-echo
export HEADROOM_BETA_HEADER_STICKY=disabled
headroom proxy ...
```
Note: disabling sticky mode may reduce prefix-cache hit rates for clients that legitimately drop-and-re-add beta tokens across turns.
### Filesystem Contract [#filesystem-contract]
Headroom resolves every on-disk resource through a two-root model
(`HEADROOM_CONFIG_DIR` + `HEADROOM_WORKSPACE_DIR`) with additive
precedence rules: explicit argument > per-resource env var > derived
from canonical root > default. Every legacy env var continues to work
unchanged.
See the **[Filesystem Contract](/docs/filesystem-contract)**
page for the full bucket table, plugin-author guidance, and the Docker
naming overlap note (`HEADROOM_WORKSPACE` is *not* the same as
`HEADROOM_WORKSPACE_DIR`).
## Custom Model Configuration [#custom-model-configuration]
Configure context limits and pricing for new or custom models:
```json
{
"anthropic": {
"context_limits": {
"claude-4-opus-20250301": 200000,
"claude-custom-finetune": 128000
},
"pricing": {
"claude-4-opus-20250301": {
"input": 15.00,
"output": 75.00,
"cached_input": 1.50
}
}
},
"openai": {
"context_limits": {
"gpt-5": 256000,
"ft:gpt-4o:my-org": 128000
}
}
}
```
Save as `${HEADROOM_CONFIG_DIR}/models.json` (defaults to
`~/.headroom/config/models.json`), or set `HEADROOM_MODEL_LIMITS` to a
JSON string or file path. Installs that still have
`~/.headroom/models.json` (the legacy location) continue to work.
Settings are resolved in this order (later overrides earlier):
1. Built-in defaults
2. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location); falls
back to `~/.headroom/models.json` (legacy) when the canonical file
is absent
3. `HEADROOM_MODEL_LIMITS` environment variable
4. SDK constructor arguments
### Pattern-Based Inference [#pattern-based-inference]
Unknown models are automatically inferred from naming patterns:
| Pattern | Inferred Settings |
| ------------ | ------------------------------------- |
| `*opus*` | 200K context, Opus-tier pricing |
| `*sonnet*` | 200K context, Sonnet-tier pricing |
| `*haiku*` | 200K context, Haiku-tier pricing |
| `gpt-4o*` | 128K context, GPT-4o pricing |
| `o1*`, `o3*` | 200K context, reasoning model pricing |
## Provider-Specific Settings [#provider-specific-settings]
Provider classes configure token counting and context/pricing overrides, not
cache toggles -- provider-specific cache optimization (Anthropic
`cache_control` breakpoints, OpenAI prefix stabilization, Google
`CachedContent`) is controlled by `enable_cache_optimizer` on `HeadroomClient`
(see [SDK Configuration](#sdk-configuration) above), which auto-detects the
right strategy from the provider you pass in.
```python
from headroom import OpenAIProvider
provider = OpenAIProvider(
context_limits={"gpt-4o": 128000},
)
```
```python
from headroom import AnthropicProvider
from anthropic import Anthropic
provider = AnthropicProvider(
# Optional: enables accurate token counting via Anthropic's Token Count API
client=Anthropic(),
)
```
```python
from headroom.providers import GoogleProvider
# Optional: pass a configured google.generativeai module for API-based
# token counting via countTokens; omit for estimation-based counting.
provider = GoogleProvider()
```
## Tool Profiles [#tool-profiles]
Skip or customize compression for specific tools:
```python
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
headroom_tool_profiles={
"important_tool": {"skip_compression": True},
"search_tool": {"max_items_after_crush": 25},
},
)
```
## Configuration Precedence [#configuration-precedence]
Settings are applied in this order (later overrides earlier):
1. Default values
2. Environment variables
3. SDK constructor arguments
4. Per-request overrides
## Validation [#validation]
Validate your configuration at startup:
```python
result = client.validate_setup()
if not result["valid"]:
print("Configuration issues:")
for check in ("provider", "storage", "config", "cache_optimizer"):
if result[check]["error"]:
print(f" - {check}: {result[check]['error']}")
```
Context management is now handled automatically inside the pipeline (live-zone-only compression). Headroom **never drops messages** from the conversation history and does not do position-based or score-based context management.
## How It Works [#how-it-works]
Headroom compresses only the **newest content blocks** — the latest user message and the latest tool result / tool output. Compression is type-aware and reversible via [CCR](/docs/ccr), so the LLM can retrieve the original content on demand.
The **cache hot zone** — the system prompt, tool definitions, and older turns — is never mutated. Leaving the prefix untouched preserves provider prompt caching, so cache hit rates stay stable across turns.
```
Conversation with a large latest tool result
-> Identify the live zone (newest user message + latest tool output)
-> Compress the live zone type-aware, cache original in CCR (hash=def456)
-> Insert marker: "compressed, retrieve: def456"
-> Older turns, tools, and system prompt are forwarded byte-for-byte
```
## Protection rules [#protection-rules]
Headroom enforces several protections to ensure model output quality:
### Output buffer reservation [#output-buffer-reservation]
A configurable number of tokens is reserved for the model's response. The context budget is calculated as:
```
context_budget = model_context_limit - output_buffer_tokens
```
This prevents the input from consuming the entire context window and leaving no room for the model to respond.
### System message protection [#system-message-protection]
System messages are never dropped. They contain critical instructions, persona definitions, and tool descriptions that the model needs throughout the conversation.
### Turn protection [#turn-protection]
The last N messages are always preserved, ensuring the model has immediate conversational context. The one-function `compress()` API exposes this as `CompressConfig.protect_recent`, which defaults to `4` messages (roughly the last 2 user/assistant turns) -- see `headroom/compress.py:110-112`.
## Configuration [#configuration]
Context management is now automatic. Use per-request overrides to control behavior:
```ts twoslash
import { compress } from "headroom-ai";
const result = await compress(messages, {
model: "gpt-4o",
tokenBudget: 32000,
});
console.log(`Compressed: ${result.tokensBefore} -> ${result.tokensAfter}`);
```
```python
from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
default_mode="optimize",
)
# Per-request overrides
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
headroom_output_buffer_tokens=8000, # More room for long responses
)
```
`HeadroomClient.chat.completions.create()` also accepts a `headroom_keep_turns`
parameter, but as of this package version it is accepted and threaded through
the wrapper methods without being forwarded into the transform pipeline (see
`headroom/client.py` -- `TransformPipeline.apply()`'s accepted kwargs, listed
in its docstring, don't include a turns/keep-turns option). Passing it does
not raise an error, but it currently has no effect; use `compress()`'s
`protect_recent` (see [Turn protection](#turn-protection) above) if you need
this control today.
> **Note:** The `IntelligentContextConfig`, `ScoringWeights`, and `RollingWindowConfig` classes are no longer part of Headroom. Context management is now handled automatically inside the pipeline (live-zone-only compression).
Headroom integrates with [CrewAI](https://github.com/crewAIInc/crewAI) to compress tool outputs before they enter the agent's LLM context. Tool-heavy agents that return large JSON arrays, database results, or verbose logs see the largest token reductions.
## Installation [#installation]
```bash
pip install headroom-ai crewai
```
## Quick start [#quick-start]
Wrap tools in one line:
```python
from crewai import Agent, Crew, Task
from crewai.tools.base_tool import tool
from headroom.integrations.crewai import wrap_tools_with_headroom
@tool
def search_database(query: str) -> str:
"""Search the database and return results."""
return json.dumps({"results": [...], "total": 1000})
wrapped = wrap_tools_with_headroom([search_database])
agent = Agent(
role="Researcher",
goal="Answer questions using data",
backstory="You research things.",
tools=wrapped,
)
task = Task(description="Find all active users", agent=agent, expected_output="Summary")
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff()
```
## Per-tool metrics [#per-tool-metrics]
Track compression stats across all tool invocations:
```python
from headroom.integrations.crewai import get_tool_metrics
metrics = get_tool_metrics()
print(metrics.get_summary())
# {
# 'total_invocations': 25,
# 'total_compressions': 18,
# 'total_chars_saved': 450000,
# 'average_compression_ratio': 0.35,
# 'by_tool': {
# 'search_database': {'invocations': 15, 'compressions': 12, 'chars_saved': 320000},
# 'fetch_logs': {'invocations': 10, 'compressions': 6, 'chars_saved': 130000},
# }
# }
```
Reset between sessions:
```python
from headroom.integrations.crewai import reset_tool_metrics
reset_tool_metrics()
```
## Custom configuration [#custom-configuration]
Control the compression threshold:
```python
wrapped = wrap_tools_with_headroom(
[search_database, fetch_logs],
min_chars_to_compress=500, # Default: 1000
)
```
Use a dedicated metrics collector instead of the global one:
```python
from headroom.integrations.crewai import ToolMetricsCollector, wrap_tools_with_headroom
collector = ToolMetricsCollector()
wrapped = wrap_tools_with_headroom(
[search_database],
metrics_collector=collector,
)
# After crew run
print(collector.get_summary())
```
## Wrapping individual tools [#wrapping-individual-tools]
For finer control, wrap tools individually:
```python
from headroom.integrations.crewai import HeadroomToolWrapper
wrapper = HeadroomToolWrapper(
search_database,
min_chars_to_compress=500,
)
# Use wrapper directly — it's a BaseTool
agent = Agent(role="Researcher", tools=[wrapper], ...)
```
## How it works [#how-it-works]
CrewAI tools extend `BaseTool` with a `run()` → `_run()` execution flow.
`HeadroomToolWrapper` subclasses `BaseTool` and overrides `_run()` to:
1. Call the original tool's `run()` method
2. Check if the output exceeds `min_chars_to_compress`
3. If so, compress via Headroom's `compress_tool_result()`
4. Record metrics and return the compressed output
The wrapper preserves the original tool's name, description, and argument
schema, so it works as a drop-in replacement anywhere CrewAI expects a tool.
Run Headroom without installing Python or Node.js on the host. The install scripts add a native `headroom` wrapper that keeps **Headroom itself** in Docker while orchestrating the rest of your workflow on the host OS.
## One-line install [#one-line-install]
### Linux [#linux]
```bash
curl -fsSL https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.sh | bash
```
### macOS (bash 4.3+) [#macos-bash-43]
```bash
curl -fsSL https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.sh | "$(brew --prefix bash)/bin/bash"
```
Stock `/bin/bash` on macOS is 3.2, so install a newer bash first (for example via Homebrew) and run the installer with that shell. The installed wrapper pins that same bash interpreter so later invocations stay on the supported runtime.
### Windows PowerShell [#windows-powershell]
```powershell
irm https://raw.githubusercontent.com/headroomlabs-ai/headroom/main/scripts/install.ps1 | iex
```
## What the installer does [#what-the-installer-does]
1. Verifies Docker is installed and available.
2. Pulls `ghcr.io/headroomlabs-ai/headroom:latest` by default, or reuses / pulls `HEADROOM_DOCKER_IMAGE` when you set a custom image override.
3. Installs a `headroom` wrapper into `~/.local/bin` or `~/bin`.
4. Updates shell startup files so the wrapper directory is on `PATH`.
The wrapper keeps Headroom inside Docker and mounts host state back into the container so native behavior stays consistent:
* project workspace → `/workspace`
* `~/.headroom`
* `~/.claude`
* `~/.codex`
* `~/.gemini`
Port `8787` stays the default, so `http://localhost:8787` works the same way as a native install.
Published releases also push versioned GHCR tags such as `ghcr.io/headroomlabs-ai/headroom:0.37.0`, and those images are built with the same synced package version used for the matching PyPI and npm release.
## How the wrapper behaves [#how-the-wrapper-behaves]
### Native Headroom commands [#native-headroom-commands]
These run directly inside the container:
```bash
headroom proxy
headroom learn
headroom mcp install
headroom memory list
```
For `proxy`, the wrapper publishes the selected port back to the host:
```bash
docker run --rm -it \
-p 8787:8787 \
-v "$PWD:/workspace" \
-w /workspace \
ghcr.io/headroomlabs-ai/headroom:latest \
headroom proxy --host 0.0.0.0 --port 8787
```
### `wrap` commands [#wrap-commands]
`wrap` is host-oriented in Docker-native mode:
* the wrapper starts the Headroom proxy in Docker
* container-side prep writes Headroom config and memory into mounted host files
* the target CLI itself is launched on the host by the wrapper
Supported host wrap flows:
* `headroom wrap claude`
* `headroom wrap codex`
* `headroom wrap aider`
* `headroom wrap cursor`
* `headroom wrap openclaw`
* `headroom unwrap openclaw`
OpenClaw remains host-native in Docker-native mode:
* the host must already have the `openclaw` CLI installed
* `headroom wrap openclaw` installs/configures the Headroom plugin through the host `openclaw` CLI
* plugin auto-start still launches the installed host `headroom` wrapper from `PATH`, which then runs Headroom in Docker
* local plugin source mode (`--plugin-path`) is also supported, but it may require host `npm` when build steps are needed
## Persistent Docker lifecycle from the native wrapper [#persistent-docker-lifecycle-from-the-native-wrapper]
The Docker-native `headroom` wrapper exposes the persistent Docker lifecycle directly:
```bash
headroom install apply --profile default --preset persistent-docker
headroom install status
headroom install restart
headroom install remove
```
In Docker-native mode this surface is intentionally scoped to **persistent-docker**:
* supported: `apply`, `status`, `start`, `stop`, `restart`, `remove`
* supported flags: `--profile`, `--port`, `--backend`, `--anyllm-provider`, `--region`, `--mode`, `--memory`, `--no-telemetry`, `--image`
* not supported: `persistent-service`, `persistent-task`, or provider/user/system mutation flags such as `--scope`, `--providers`, and `--target`
Those broader lifecycle and config-mutation flows still belong to the Python-native `headroom install ...` command.
Persistent Docker deployments launched by the wrapper also tag the proxy process with deployment metadata, so `/health` reports the active `profile`, `preset`, `runtime`, `supervisor`, and `scope` the same way the Python install subsystem does.
## Docker Compose support [#docker-compose-support]
Use `docker/docker-compose.native.yml` when you want an explicit compose-managed proxy or CLI shell, or when you prefer compose over the native wrapper's `headroom install ...` surface.
### Persistent Docker runtime [#persistent-docker-runtime]
The `proxy` service uses `restart: unless-stopped`, so compose can act as the always-on Docker runtime for Headroom:
```bash
export HEADROOM_HOST_HOME="$HOME"
export HEADROOM_WORKSPACE="$PWD"
docker compose -f docker/docker-compose.native.yml up -d proxy
```
```powershell
$env:HEADROOM_HOST_HOME = $HOME
$env:HEADROOM_WORKSPACE = (Get-Location).Path
docker compose -f docker/docker-compose.native.yml up -d proxy
```
This is a supported persistent-Docker path when you want the proxy managed explicitly through Compose instead of the installed wrapper.
These are two different variables — both are set by the compose file, and both are retained for backward compatibility:
* **`HEADROOM_WORKSPACE`** (host-side) is the directory the compose file bind-mounts into the container as `/workspace`. It behaves like CWD in a native (non-Docker) run.
* **`HEADROOM_WORKSPACE_DIR`** (inside the container) is the canonical Headroom state root from the [filesystem contract](/docs/filesystem-contract). The compose file sets it to `/tmp/headroom-home/.headroom` so the proxy resolves savings, logs, TOIN, and memory under the bind-mounted `${HOME}/.headroom`.
You do not need to set `HEADROOM_WORKSPACE_DIR` manually when using the shipped compose file — it is already in the `environment:` block.
### macOS / Linux [#macos--linux]
```bash
export HEADROOM_HOST_HOME="$HOME"
export HEADROOM_WORKSPACE="$PWD"
docker compose -f docker/docker-compose.native.yml up proxy
```
### Windows PowerShell [#windows-powershell-1]
```powershell
$env:HEADROOM_HOST_HOME = $HOME
$env:HEADROOM_WORKSPACE = (Get-Location).Path
docker compose -f docker/docker-compose.native.yml up proxy
```
You can also run one-off CLI commands through compose:
```bash
docker compose -f docker/docker-compose.native.yml run --rm cli learn
docker compose -f docker/docker-compose.native.yml run --rm cli mcp install
```
## Environment passthrough [#environment-passthrough]
The wrapper forwards Headroom and provider environment variables into the container, including common prefixes such as:
* `HEADROOM_`
* `ANTHROPIC_`
* `OPENAI_`
* `GEMINI_`
* `AWS_`
* `GOOGLE_` / `GOOGLE_CLOUD_`
* `AZURE_`
* `OTEL_`
That keeps provider auth and runtime config working without maintaining a separate env file for the container.
## Notes [#notes]
* Docker is the only required Headroom runtime dependency on the host.
* Wrapped tools like Claude Code, Codex CLI, Aider, and Cursor still run on the host when you use `headroom wrap ...`.
* The install scripts are idempotent: rerunning them refreshes the wrapper and image without duplicating shell profile blocks.
* For persistent service and task installs, use the Python-native `headroom install ...` workflow — see [Persistent Installs](/docs/persistent-installs).
* For Docker-native `headroom install ...`, the wrapper persists its profile manifest under `~/.headroom/deploy//`.
## Next steps [#next-steps]
Headroom provides explicit exceptions for debugging, with a core safety guarantee: **compression failures never break your LLM calls**. If compression fails, the original content passes through unchanged.
## Error Hierarchy [#error-hierarchy]
```
HeadroomError (base class)
+-- HeadroomConnectionError # Cannot reach proxy
+-- HeadroomAuthError # 401 from proxy
+-- HeadroomCompressError # Compression failed (with statusCode)
+-- ConfigurationError # Invalid configuration
+-- ProviderError # Provider issues
+-- StorageError # Storage failures
+-- TokenizationError # Token counting failed
+-- CacheError # Cache operations failed
+-- ValidationError # Validation failures
+-- TransformError # Transform execution failed
```
```ts twoslash
import {
HeadroomError,
HeadroomConnectionError,
HeadroomAuthError,
HeadroomCompressError,
ConfigurationError,
ProviderError,
mapProxyError,
} from 'headroom-ai';
```
```
HeadroomError (base class)
+-- ConfigurationError # Invalid configuration
+-- ProviderError # Provider issues (unknown model, etc.)
+-- StorageError # Database/storage failures
+-- CompressionError # Compression failures (rare)
+-- TokenizationError # Token counting failed
+-- CacheError # Cache operations failed
+-- ValidationError # Setup validation failures
+-- TransformError # Transform execution failed
```
```python
from headroom import (
HeadroomError,
ConfigurationError,
ProviderError,
StorageError,
CompressionError,
TokenizationError,
CacheError,
ValidationError,
TransformError,
)
```
## Catching Errors [#catching-errors]
```ts twoslash
import { compress, HeadroomConnectionError, HeadroomAuthError, HeadroomCompressError, HeadroomError } from 'headroom-ai';
try {
const result = await compress(messages, { model: 'gpt-4o' });
} catch (e) {
if (e instanceof HeadroomConnectionError) {
console.error('Cannot reach proxy:', e.message);
} else if (e instanceof HeadroomAuthError) {
console.error('Auth failed:', e.message);
} else if (e instanceof HeadroomCompressError) {
console.error(`Compress failed (${e.statusCode}):`, e.message);
} else if (e instanceof HeadroomError) {
console.error('Headroom error:', e.message, e.details);
}
}
```
```python
from headroom import (
HeadroomClient,
HeadroomError,
ConfigurationError,
StorageError,
)
try:
client = HeadroomClient(...)
response = client.chat.completions.create(...)
except ConfigurationError as e:
print(f"Config issue: {e}")
print(f"Details: {e.details}")
except StorageError as e:
print(f"Storage issue: {e}")
# Headroom continues to work, just without metrics persistence
except HeadroomError as e:
print(f"Headroom error: {e}")
```
## Error Types in Detail [#error-types-in-detail]
### ConfigurationError [#configurationerror]
Raised when configuration is invalid. In the Python SDK (0.37.0),
`ConfigurationError` is importable and exported from `headroom`, but
`HeadroomClient` itself does not currently raise it: an invalid
`default_mode` string is validated by the `HeadroomMode` enum and raises a
plain **`ValueError`**, not `ConfigurationError`.
```ts twoslash
import { ConfigurationError } from 'headroom-ai';
// ConfigurationError is thrown when the proxy returns
// a configuration_error type in its error response
```
```python
try:
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
default_mode="invalid_mode",
)
except ValueError as e:
print(f"Invalid mode: {e}")
# 'invalid_mode' is not a valid HeadroomMode
```
### ProviderError [#providererror]
Reserved for provider-specific issues, but not currently raised anywhere in
the SDK. In particular, an unknown model name does **not** raise
`ProviderError`: `Provider.get_context_limit()` is documented to "never
raise an exception -- uses sensible defaults for unknown models," and only
logs a one-time `WARNING` naming the fallback it used.
```python
import logging
logging.basicConfig(level=logging.WARNING)
# Does NOT raise ProviderError -- falls back to a default context limit
# and logs: WARNING:headroom.providers.openai:Unknown OpenAI model
# 'unknown-model-xyz': using default limit (...). To configure explicitly,
# set HEADROOM_MODEL_LIMITS env var or add to ~/.headroom/models.json
response = client.chat.completions.create(
model="unknown-model-xyz",
messages=[...],
)
```
### StorageError [#storageerror]
Reserved for database/storage failures. `HeadroomClient.get_metrics()` and
`get_summary()` currently delegate straight to the storage backend
(`headroom/client.py`) without wrapping failures in `StorageError`, so a
broken database today surfaces as the underlying driver's own exception
(for example `sqlite3.OperationalError`) rather than `StorageError`. Catch
the base `HeadroomError` -- or `Exception` around storage calls specifically
\-- until that wrapping lands.
```python
try:
metrics = client.get_metrics()
except Exception as e:
metrics = [] # Continue without historical metrics
```
### CompressionError [#compressionerror]
Reserved for compression failures. As of 0.37.0 this class is exported but
not raised anywhere in the codebase -- there is no "strict mode" that
triggers it. Compression failures are instead caught internally by broad
`except Exception` handlers throughout `headroom/transforms/`, which fail
open and return the original, uncompressed content. This is what backs the
safety guarantee below.
### HeadroomConnectionError (TypeScript) [#headroomconnectionerror-typescript]
Raised when the TypeScript SDK cannot connect to the Headroom proxy.
```ts twoslash
import { compress, HeadroomConnectionError } from 'headroom-ai';
try {
await compress(messages, { model: 'gpt-4o' });
} catch (e) {
if (e instanceof HeadroomConnectionError) {
console.error('Is the proxy running? Start with: headroom proxy');
}
}
```
## Proxy Error Mapping [#proxy-error-mapping]
The TypeScript SDK automatically maps proxy error responses to the correct error class:
| HTTP Status | Proxy Error Type | TypeScript Class |
| ----------- | --------------------- | ----------------------- |
| 401 | -- | `HeadroomAuthError` |
| 4xx/5xx | `configuration_error` | `ConfigurationError` |
| 4xx/5xx | `provider_error` | `ProviderError` |
| 4xx/5xx | `storage_error` | `StorageError` |
| 4xx/5xx | `tokenization_error` | `TokenizationError` |
| 4xx/5xx | `cache_error` | `CacheError` |
| 4xx/5xx | `validation_error` | `ValidationError` |
| 4xx/5xx | `transform_error` | `TransformError` |
| 4xx/5xx | (other) | `HeadroomCompressError` |
The `mapProxyError()` function handles this mapping:
```ts twoslash
import { mapProxyError } from 'headroom-ai';
const error = mapProxyError(400, 'configuration_error', 'Invalid mode');
// Returns a ConfigurationError instance
```
## Error Details [#error-details]
All Headroom exceptions include a `details` dict/object with additional context:
```ts twoslash
import { HeadroomError } from 'headroom-ai';
// HeadroomError.details is Record | undefined
// HeadroomCompressError also has .statusCode and .errorType
```
```python
try:
client = HeadroomClient(...)
except HeadroomError as e:
print(f"Error: {e}")
print(f"Type: {type(e).__name__}")
print(f"Details: {e.details}")
# Details might include:
# - field: which config field caused the error
# - provider: which provider was involved
# - model: which model was requested
# - original_error: underlying exception
```
## Safety Guarantee [#safety-guarantee]
If compression fails, the original content passes through unchanged. Your LLM calls never fail due to Headroom:
```python
messages = [
{"role": "tool", "content": "malformed json {{{"}
]
# This will NOT raise an exception
# The malformed content passes through unchanged
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
)
```
## Best Practices [#best-practices]
1. **Catch specific exceptions** rather than broad `Exception` to avoid hiding real bugs
2. **Don't rely on storage failures raising** -- as of 0.37.0 they surface as the
underlying driver's own exception, not `StorageError` (see above)
3. **Validate on startup** with `client.validate_setup()` to catch configuration issues early
4. **Enable logging** at WARNING level to see when compression falls back safely
```python
import logging
logging.basicConfig(level=logging.WARNING)
# WARNING:headroom.transforms.smart_crusher:SmartCrusher audit_safe: 1 protected
# row(s) could not be preserved through compression (...). Failing closed:
# returning original uncompressed.
```
`headroom learn` analyzes past coding agent sessions, finds what went wrong, correlates each failure with what eventually worked, and writes specific project-level learnings that prevent the same mistakes next session.
## Quick Start [#quick-start]
```bash
# See recommendations for current project (dry-run, no changes)
headroom learn
# Write recommendations to CLAUDE.local.md and MEMORY.md
headroom learn --apply
# Analyze a specific project
headroom learn --project ~/my-project --apply
# Analyze all projects
headroom learn --all --apply
# Write to the team-shared CLAUDE.md instead of CLAUDE.local.md
headroom learn --apply --target CLAUDE.md
```
## Success Correlation [#success-correlation]
The core innovation. Instead of cataloging failures ("Read failed 5 times"), Headroom finds what the model did to **fix** each failure:
* **Failed**: `Read axion-formats/src/main/java/.../FirstClassEntity.java`
* **Then succeeded**: `Read axion-scala-common/src/main/scala/.../FirstClassEntity.scala`
* **Learning**: "`FirstClassEntity` is at `axion-scala-common/`, not `axion-formats/`"
This produces specific, actionable corrections -- not generic advice.
## What It Learns [#what-it-learns]
### Environment Facts [#environment-facts]
Which runtime commands work vs fail.
```markdown
### Environment
- **Python**: use `uv run python` (not `python3` -- modules not available outside venv)
```
### File Path Corrections [#file-path-corrections]
Wrong paths the model keeps guessing, with the correct locations.
```markdown
### File Path Corrections
- `axion-common/src/.../AxionSparkConstants.scala`
-> actually at `axion-spark-common/src/.../AxionSparkConstants.scala`
```
### Search Scope [#search-scope]
Which directories to search in (narrow paths fail, broader ones work).
```markdown
### Search Scope
- Don't search `axion-model/` -> use `axion/` (the repo root)
```
### Command Patterns [#command-patterns]
How commands should (and should not) be run.
```markdown
### Command Patterns
- **user_prefers_manual**: User rejected gradle 18 times -- show the command, don't execute
- **python_runtime**: Use `uv run python` not `python3` (ModuleNotFoundError)
```
### Known Large Files [#known-large-files]
Files that need `offset`/`limit` with Read.
```markdown
### Known Large Files
- `proxy/server.py` (~8000 lines) -- always use offset/limit
```
## Where Learnings Go [#where-learnings-go]
| Pattern | Destination | Why |
| ------------------------------------------------------- | ------------------- | -------------------------------------------------------------- |
| Environment, paths, search scope, commands, large files | **CLAUDE.local.md** | Personal facts (machine-specific paths), gitignored by default |
| Missing paths, retry patterns, permissions | **MEMORY.md** | May change, agent-specific |
`CLAUDE.local.md` lives in your project directory. It is the **personal** local
memory file in Claude Code's [memory convention](https://docs.claude.com/en/docs/claude-code/memory)
— meant to be gitignored — so machine-specific learnings (absolute paths,
tool-discovery byproducts) don't pollute the team-shared `CLAUDE.md`. Make sure
`CLAUDE.local.md` is listed in your `.gitignore`. Pass `--target CLAUDE.md` to
opt into the shared file instead, or `--target ` for any custom location.
MEMORY.md lives in `~/.claude/projects/*/memory/`.
If an older Headroom version already wrote a learned-patterns block into your
team-shared `CLAUDE.md`, the next `headroom learn --apply` moves it into
`CLAUDE.local.md` and prints a warning so you can review the diff before
committing. If `CLAUDE.md` contained nothing but the Headroom block, it is
removed entirely.
## Marker-Based Updates [#marker-based-updates]
Headroom manages a clearly-delimited section in each file:
```markdown
## Headroom Learned Patterns
*Auto-generated by `headroom learn` -- do not edit manually*
...
```
On re-run, only the content between markers is replaced. Your existing file content is preserved.
## Architecture [#architecture]
The system is built with an adapter pattern so it can support multiple agent systems:
* **Scanners** read tool-specific log formats (e.g., `~/.claude/projects/*.jsonl`) and produce normalized `ToolCall` sequences
* **Analyzers** work on `ToolCall` data -- same analysis logic for any agent system
* **Writers** output to tool-specific context injection mechanisms (e.g., CLAUDE.md)
To add support for a new agent (e.g., Cursor), you write a Scanner that reads its log format and a Writer that outputs to `.cursorrules`. The analyzers stay the same.
## CLI Reference [#cli-reference]
```bash
headroom learn [OPTIONS]
Options:
--project PATH Project directory to analyze (default: current directory)
--all Analyze all discovered projects
--apply Write recommendations (default: dry-run)
--target TEXT Context file to write to, Claude Code only (default:
CLAUDE.local.md). Relative to project root, or absolute.
--agent TEXT Agent to analyze (e.g. claude, codex, gemini, opencode)
--model TEXT LLM model to use for analysis
--workers INT Number of parallel workers
```
## Real-World Results [#real-world-results]
A prior version of this page cited specific counts (tool calls analyzed,
failure rate, corrections extracted) for a real run. We could not trace
those numbers to a committed benchmark, test, or dataset in this repo, and
they appear to have been copied from `wiki/learn.md` rather than measured
independently -- so we removed them rather than publish an unverifiable
figure. Run `headroom learn` (dry-run by default) on your own session
history to see real counts for your project.
Headroom writes configuration, runtime state, logs, and caches to a small set of well-known paths under the user's home directory. This page is the source of truth for where those paths live, how to override them, and how they behave inside Docker containers.
## Two-root model [#two-root-model]
| Variable | Default | Purpose | Typical access |
| ------------------------ | -------------------- | ---------------------------------------------------------------------------------------- | -------------- |
| `HEADROOM_CONFIG_DIR` | `~/.headroom/config` | User/admin-authored configuration (model catalogs, plugin settings, etc.) | Read-mostly |
| `HEADROOM_WORKSPACE_DIR` | `~/.headroom` | Runtime state written by the proxy and CLI (savings, logs, memory DB, telemetry, caches) | Read-write |
Both variables are recognized by the Python proxy / CLI and the npm SDK. They are **additive** — every pre-existing per-resource env var (`HEADROOM_SAVINGS_PATH`, `HEADROOM_TOIN_PATH`, `HEADROOM_SUBSCRIPTION_STATE_PATH`, `HEADROOM_MODEL_LIMITS`, ...) continues to work with identical semantics.
## Precedence [#precedence]
For every per-resource helper, resolution follows this order:
```
explicit argument
│ falls through when None/""
▼
per-resource env var (e.g. HEADROOM_SAVINGS_PATH)
│ falls through when unset/blank
▼
derived from canonical root
│ e.g. ${HEADROOM_WORKSPACE_DIR}/proxy_savings.json
▼
default (e.g. ~/.headroom/proxy_savings.json)
```
Examples:
* `HEADROOM_WORKSPACE_DIR=/mnt/state` → savings land at `/mnt/state/proxy_savings.json` unless `HEADROOM_SAVINGS_PATH` overrides.
* `HEADROOM_SAVINGS_PATH=/custom/savings.json` always wins, even when `HEADROOM_WORKSPACE_DIR` is set.
* Unset both and the default is `~/.headroom/proxy_savings.json`.
## Bucket assignments [#bucket-assignments]
### Workspace bucket (`HEADROOM_WORKSPACE_DIR`) [#workspace-bucket-headroom_workspace_dir]
| Resource | Default path | Legacy env var |
| -------------------------- | ------------------------------------------ | ---------------------------------- |
| Proxy savings ledger | `${WORKSPACE_DIR}/proxy_savings.json` | `HEADROOM_SAVINGS_PATH` |
| TOIN telemetry JSON | `${WORKSPACE_DIR}/toin.json` | `HEADROOM_TOIN_PATH` |
| Subscription tracker state | `${WORKSPACE_DIR}/subscription_state.json` | `HEADROOM_SUBSCRIPTION_STATE_PATH` |
| Memory SQLite | `${WORKSPACE_DIR}/memory.db` | CLI `--memory-db-path` |
| Native memory directory | `${WORKSPACE_DIR}/memories/` | `MemoryConfig.native_memory_dir` |
| License cache | `${WORKSPACE_DIR}/license_cache.json` | — |
| Session stats JSONL | `${WORKSPACE_DIR}/session_stats.jsonl` | — |
| Memory sync state | `${WORKSPACE_DIR}/sync_state.json` | — |
| Memory bridge state | `${WORKSPACE_DIR}/bridge_state.json` | — |
| Proxy log directory | `${WORKSPACE_DIR}/logs/` | — |
| HTTP 400 debug dumps | `${WORKSPACE_DIR}/logs/debug_400/` | — |
| Deployment profiles | `${WORKSPACE_DIR}/deploy/` | — |
| Beacon lock file | `${WORKSPACE_DIR}/.beacon_lock_` | — |
### Config bucket (`HEADROOM_CONFIG_DIR`) [#config-bucket-headroom_config_dir]
| Resource | Default path | Legacy env var |
| --------------- | ---------------------------------- | ------------------------------------------ |
| Models catalog | `${CONFIG_DIR}/models.json` | `HEADROOM_MODEL_LIMITS` (content override) |
| Plugin settings | `${CONFIG_DIR}/plugins//...` | — |
### Backward compatibility — models.json [#backward-compatibility--modelsjson]
`models.json` historically lived at `~/.headroom/models.json` (i.e. in the workspace root, not in `config/`). For a seamless migration the Python providers check **both** locations in this order:
1. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location)
2. `${HEADROOM_WORKSPACE_DIR}/models.json` (legacy fallback)
Existing installs continue to work unchanged. New installs are encouraged to put `models.json` in the config bucket.
## Plugin authors [#plugin-authors]
Two helpers give plugins isolated, per-plugin directories under both roots:
### Python [#python]
```python
from headroom import paths
cfg_dir = paths.plugin_config_dir("my-plugin")
# → ~/.headroom/config/plugins/my-plugin
state_dir = paths.plugin_workspace_dir("my-plugin")
# → ~/.headroom/plugins/my-plugin
cfg_dir.mkdir(parents=True, exist_ok=True)
(cfg_dir / "settings.json").write_text("{}")
```
### npm SDK [#npm-sdk]
```typescript
import { pluginConfigDir, pluginWorkspaceDir } from "@headroom/sdk";
const cfgDir = pluginConfigDir("my-plugin");
const stateDir = pluginWorkspaceDir("my-plugin");
```
Plugin-author helpers reject names containing `/` or `\` to keep the namespace flat.
## Docker naming overlap: `HEADROOM_WORKSPACE` vs `HEADROOM_WORKSPACE_DIR` [#docker-naming-overlap-headroom_workspace-vs-headroom_workspace_dir]
These are **two different variables** with different semantics, both retained for backward compatibility:
| Variable | Scope | Meaning |
| ------------------------ | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `HEADROOM_WORKSPACE` | Host-side (Docker) | Directory to bind-mount into the container as `/workspace` (equivalent to CWD in native runs). Used by `docker-compose.native.yml`. |
| `HEADROOM_WORKSPACE_DIR` | Inside the container | Canonical Headroom state root. Resolves to `/tmp/headroom-home/.headroom` inside the official container image, which in turn bind-mounts to `${HOME}/.headroom` on the host. |
The official Docker bootstrap (compose file, `scripts/install.sh`, and the Python `install` command) sets `HEADROOM_WORKSPACE_DIR` and `HEADROOM_CONFIG_DIR` inside the container so the proxy resolves state to the bind-mounted path without any user action.
## Project-scoped `.headroom/` directories [#project-scoped-headroom-directories]
A few code paths deliberately use **project-local** `.headroom/` paths resolved relative to the current working directory rather than the canonical workspace root:
* `headroom/proxy/server.py` — project-scoped memory DB default
* `headroom/memory/mcp_server.py` — project-scoped memory DB default
* `headroom/cli/wrap.py` — project-scoped memory and hook artifacts
These **do not obey** `HEADROOM_WORKSPACE_DIR`. This is intentional: it preserves the "project memory lives in the project directory" invariant. Users who want a single centrally located memory store can pass `--memory-db-path ` explicitly or set the path via the plugin API.
## Legacy per-resource env vars [#legacy-per-resource-env-vars]
Every legacy env var continues to work with its original semantics (raw string in, raw string out — no tilde expansion, no path-separator normalization), ensuring byte-for-byte backward compatibility.
Full list:
* `HEADROOM_SAVINGS_PATH`
* `HEADROOM_TOIN_PATH`
* `HEADROOM_SUBSCRIPTION_STATE_PATH`
* `HEADROOM_MODEL_LIMITS` (content override — JSON string or file path)
## See also [#see-also]
Use `headroom wrap grok-build` to route Grok Build LLM traffic through the local Headroom proxy. The wrapper starts or reuses the proxy, injects a reversible `[model.grok-build]` override into `~/.grok/config.toml` (or `$GROK_HOME/config.toml`), and prints next steps for launching `grok`.
## Quick Start [#quick-start]
```bash
headroom wrap grok-build
```
In another terminal, from the same project directory:
```bash
grok
```
When you are done:
```bash
headroom unwrap grok-build
```
## What `wrap grok-build` Does [#what-wrap-grok-build-does]
| Step | What happens |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Proxy | Starts the Headroom proxy unless `--no-proxy` is set |
| Model config | Writes or updates `[model.grok-build] base_url` in Grok's `config.toml`, pointing at `http://127.0.0.1:/v1` (with optional `/p/` prefix for savings attribution) |
| Existing config | If you already have a `[model.grok-build]` table, Headroom rewrites `base_url` in place instead of appending a duplicate table (invalid TOML) |
| MCP install | `headroom mcp install` can register Headroom MCP via `GrokRegistrar` |
| Backup | Snapshots `config.toml` to `config.toml.headroom-backup` before the first injection |
## Options [#options]
```bash
headroom wrap grok-build \
--port 8787 \ # Proxy port (default: 8787)
--no-proxy \ # Use an existing proxy instead of starting one
--learn \ # Enable live traffic learning
--memory # Enable persistent memory
```
## Environment Variables [#environment-variables]
| Variable | Description |
| ------------- | --------------------------------------------------- |
| `GROK_HOME` | Override Grok config directory (default: `~/.grok`) |
| `XAI_API_KEY` | Grok API key (also accepts `GROK_CODE_XAI_API_KEY`) |
## Persistent Install [#persistent-install]
`grok_build` is an install target for `headroom install apply --providers manual --target grok_build`. The install manifest records proxy env values for Grok Build alongside other wrapped agents.
## Unwrap [#unwrap]
`headroom unwrap grok-build` restores the pre-wrap `config.toml` from backup when available, or strips Headroom marker blocks and in-place `base_url` rewrites when no backup exists.
Headroom automatically detects what kind of content you're sending and routes it to the right compressor. You don't need to configure anything -- just call `compress()` and the pipeline handles the rest.
## The Pipeline [#the-pipeline]
Every request flows through a short pipeline:
```
┌──────────────┐ ┌────────────────┐
│ CacheAligner │────>│ ContentRouter │
│ (off by │ │ │
│ default) │ │ Detect type & │
│ report drift │ │ route to best │
│ for cache │ │ compressor │
└──────────────┘ └────────────────┘
```
1. **CacheAligner** *(detector-only, off by default)* reports dynamic-prefix drift (dates, session context) so callers can keep the static prefix cacheable. It never rewrites your messages.
2. **ContentRouter** inspects each content block and routes it to one compressor -- SmartCrusher for JSON arrays, CodeAwareCompressor for source code, LogCompressor for build output, and so on. This is where essentially all compression happens.
## Content Type Detection [#content-type-detection]
The router auto-detects content type by analyzing structure and patterns. No manual hints required.
| Content Type | Detection Signal | Compressor | Typical Savings |
| --------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| JSON arrays | Valid JSON with array elements | SmartCrusher | Varies with redundancy -- measured 26-54% on realistic (non-duplicate) tool-output arrays of 100-10,000 items; highly repetitive arrays compress far more. See `benchmarks/compression_benchmark.py`. |
| Source code | Syntax patterns, indentation, keywords | Kompress via the library `compress()`/`HeadroomClient` path (`enable_code_aware` defaults to `False` there); CodeAwareCompressor via `headroom proxy` (`HEADROOM_CODE_AWARE_ENABLED` defaults to on) | workload-dependent |
| Search results | `file:line:content` format | SearchCompressor | 80-95% (measured \~91% on a synthetic 40-file ripgrep-style corpus) |
| Build/test logs | Timestamps, log levels, pytest/npm markers | LogCompressor | 85-95% (measured \~93% on a synthetic 1,000-line log with clustered errors) |
| Diffs | Unified diff format | DiffCompressor | 60-80% (measured \~70% on a multi-file diff with default `max_context_lines`/`max_hunks_per_file` limits) |
| HTML | Tag structure | HTMLExtractor | 70-90% (module docstring; measured \~81% on a synthetic nav/ads/script-heavy page) |
| Tabular (CSV/TSV/markdown tables) | Delimited rows / table syntax | TabularCompressor | Varies with redundancy -- TabularCompressor bridges to SmartCrusher, so it inherits the same range (measured 26-28% on non-duplicate rows) |
| Structured config (YAML/TOML/INI) | Config syntax | ConfigCompressor | 40-70% (measured \~50% on a commented/blank-line-heavy YAML file) |
| Plain text | Text with no stronger signal | Kompress | workload-dependent |
| Anything else | Fallback chain, then passthrough if no safe saving is found | Kompress / passthrough | varies |
## Quick Start [#quick-start]
```ts twoslash
import { compress } from "headroom-ai";
const messages = [
{ role: "system" as const, content: "You are a helpful assistant." },
{ role: "user" as const, content: "Summarize this data" },
{ role: "tool" as const, content: '{"results": [...]}', tool_call_id: "call_1" },
];
const result = await compress(messages);
console.log(`Tokens saved: ${result.tokensSaved}`);
console.log(`Compression ratio: ${result.compressionRatio}`);
```
```python
from headroom import compress
messages = [
{"role": "user", "content": "Summarize the tool output."},
{"role": "tool", "tool_call_id": "call_1", "content": large_output},
]
result = compress(messages, model="gpt-4o")
print(result.messages)
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
```
## Configuring the Compressor [#configuring-the-compressor]
```ts twoslash
import { compress } from "headroom-ai";
const result = await compress(messages, {
model: "gpt-4o",
tokenBudget: 50000,
});
console.log(`Before: ${result.tokensBefore} tokens`);
console.log(`After: ${result.tokensAfter} tokens`);
console.log(`Transforms: ${result.transformsApplied.join(", ")}`);
```
```python
from headroom import CompressConfig, compress
config = CompressConfig(
compress_user_messages=True,
protect_recent=0,
target_ratio=0.5, # Kompress keep ratio; structural compressors use their own rules
min_tokens_to_compress=250,
)
result = compress(messages, model="gpt-4o", config=config)
print(f"Saved: {result.tokens_saved} tokens")
print(result.transforms_applied)
```
## Structure Preservation [#structure-preservation]
Headroom doesn't blindly truncate. It identifies what matters in each content type and preserves it:
| Content Type | What's Preserved | What's Compressed |
| ------------ | ------------------------------------------------------ | ---------------------------------- |
| **JSON** | Keys, brackets, booleans, nulls, short values, UUIDs | Long string values, whitespace |
| **Code** | Imports, function signatures, class definitions, types | Function bodies, comments |
| **Logs** | Timestamps, log levels, error messages, stack traces | Repeated patterns, verbose details |
| **Text** | High-entropy tokens (IDs, hashes), headers | Low-information content |
## Real Compression Ratios [#real-compression-ratios]
Compression ratio and latency both depend heavily on the shape of the input
(how redundant the data is, how large the payload is, and -- for Kompress --
what `target_ratio` is configured). There's no single number that holds
across workloads, so treat the ranges in the table above as directional and
reproduce them on your own traffic with the harnesses in
[`benchmarks/`](https://github.com/headroomlabs-ai/headroom/tree/main/benchmarks)
(for example `compression_benchmark.py`, `bench_transforms.py`,
`text_crusher_quality_eval.py`). What stays constant across content types is
*what's preserved*:
| Content Type | What's Preserved |
| -------------------- | -------------------- |
| JSON (large arrays) | All keys, structure |
| Source code (Python) | Signatures, imports |
| Search results | Relevant matches |
| Build logs | Errors, stack traces |
| Plain text | High-entropy tokens |
## Compressing multiple documents [#compressing-multiple-documents]
The public `compress()` API accepts an LLM message list. Put documents in
separate messages or call it once per independent request; this preserves the
same message shape you will send to the provider:
```python
from headroom import compress
messages = [
{"role": "user", "content": "Compare these documents."},
{"role": "tool", "tool_call_id": "doc_1", "content": first_document},
{"role": "tool", "tool_call_id": "doc_2", "content": second_document},
]
result = compress(messages, model="gpt-4o")
```
## What Happens Under the Hood [#what-happens-under-the-hood]
When you call `compress()`, here is the full sequence:
1. **Safety gates** -- protected tools, recent turns, frozen cache prefixes, and minimum-size rules decide what may change.
2. **Content detection** -- Magika when available plus deterministic pattern matching identifies the content type.
3. **Structural compression** -- format-specific handlers compact JSON, logs, search results, tables, configuration, HTML, and other recognized structures.
4. **Fallback compression** -- Kompress handles eligible text when its runtime is available; a no-saving or failed transform passes the original through.
5. **CCR storage in proxy flows** -- when CCR is enabled, originals for marker-bearing compression are retained for later retrieval. `--lossless` and `--no-ccr` intentionally use different contracts; see [Reversible Compression](/docs/ccr).
The pipeline works out of the box with no configuration. All detection, routing, and compression happens automatically. Configuration is available when you need fine-grained control.
Vision models charge by the token, and images are expensive. A single 1024x1024 image costs \~765 tokens on OpenAI. Headroom's image compression uses a trained ML router to analyze your query and automatically select the optimal compression technique, saving 40-90% of image tokens.
## How It Works [#how-it-works]
```
User uploads image + asks question
|
[Query Analysis]
TrainedRouter (MiniLM from HuggingFace)
Classifies: "What animal is this?" -> full_low
|
[Image Analysis]
SigLIP analyzes image properties
(has text? complex? fine details?)
|
[Apply Compression]
OpenAI: detail="low"
Anthropic: Resize to 512px
Google: Resize to 768px
|
Compressed request to LLM
```
The router is a fine-tuned MiniLM classifier (`chopratejas/technique-router` on HuggingFace) with 93.7% accuracy across 1,157 training examples.
## Compression Techniques [#compression-techniques]
| Technique | Savings | When Used | Example Query |
| ----------- | ------- | ----------------------- | -------------------------------------------------- |
| `full_low` | \~87% | General understanding | "What is this?", "Describe the scene" |
| `preserve` | 0% | Fine details needed | "Count the whiskers", "Read the serial number" |
| `crop` | 50-90% | Region-specific queries | "What's in the corner?", "Focus on the background" |
| `transcode` | \~99% | Text extraction | "Read the sign", "Transcribe the document" |
## Quick Start [#quick-start]
### With Headroom Proxy (Zero Code Changes) [#with-headroom-proxy-zero-code-changes]
```bash
# Start the proxy
headroom proxy --port 8787
# Connect your client -- images are compressed automatically
ANTHROPIC_BASE_URL=http://localhost:8787 claude
```
### With HeadroomClient [#with-headroomclient]
```python
from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
default_mode="optimize",
)
response = client.chat.completions.create(
model="gpt-4o",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "What animal is this?"},
{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
]
}]
)
# Image automatically compressed with detail="low" (87% savings)
```
### Direct API [#direct-api]
```python
from headroom.image import ImageCompressor
compressor = ImageCompressor()
# Compress images in messages
compressed_messages = compressor.compress(messages, provider="openai")
# Check savings
print(f"Saved {compressor.last_savings:.0f}% tokens")
print(f"Technique: {compressor.last_result.technique.value}")
```
## Provider Support [#provider-support]
The compressor adapts its strategy per provider:
| Provider | Compression Method | Details |
| ----------------- | ------------------- | ------------------------------------------ |
| **OpenAI** | Sets `detail="low"` | Native detail parameter |
| **Anthropic** | Resizes to 512px | PIL-based resize |
| **Google Gemini** | Resizes to 768px | Optimized for Gemini's 768x768 tile system |
### Token Savings by Provider [#token-savings-by-provider]
**OpenAI** (1024x1024 image):
| Technique | Before | After | Savings |
| ---------- | ---------- | ---------- | ------- |
| `full_low` | 765 tokens | 85 tokens | 89% |
| `preserve` | 765 tokens | 765 tokens | 0% |
**Anthropic** (1024x1024 image):
| Before | After | Savings |
| -------------- | ------------ | ------- |
| \~1,398 tokens | \~349 tokens | 75% |
**Google Gemini** (1536x1536 image):
| Before | After | Savings |
| ---------------------- | ------------------- | ------- |
| 1,032 tokens (4 tiles) | 258 tokens (1 tile) | 75% |
Unlike the OpenAI and Anthropic figures above (computed from the formulas in
`headroom/image/tile_optimizer.py` and verified to match this page's numbers
exactly), Headroom has no Gemini token-counting formula in the codebase --
these figures describe Google's publicly documented tile pricing, not
something this repo computes or tests.
## Configuration [#configuration]
```python
from headroom.image import ImageCompressor
compressor = ImageCompressor(
model_id="chopratejas/technique-router", # HuggingFace model
use_siglip=True, # Enable image analysis
device="cuda", # Use GPU if available (auto, cuda, cpu, mps)
)
```
### Proxy Configuration [#proxy-configuration]
```bash
# Image optimization is part of the ContentRouter path when the optional image
# dependencies are installed. There are no public proxy CLI image toggles in
# the current release.
headroom proxy
```
## Performance [#performance]
| Metric | Value |
| ------------------- | ------------------------------------------------------------------------------------------------------- |
| Image resize | \~5-20ms (measured \~9ms median for a 1024px -> 512px PIL/LANCZOS resize) |
| First request | Model download, cached after (duration depends on network) |
| Router accuracy | 93.7% (fine-tuned MiniLM classifier, 1,157 training examples -- see `headroom/image/trained_router.py`) |
| Model size | \~100MB (PyTorch MiniLM router) or \~32MB (ONNX INT8 router, preferred in production) |
| GPU memory (SigLIP) | \~400MB |
When using the Headroom proxy, image compression happens automatically on every request that contains images. No code changes needed.
Headroom compresses everything your AI agent reads -- tool outputs, database results, file reads, RAG retrievals, API responses -- before it reaches the LLM. The model sees less noise, responds faster, and costs less.
## Quick preview [#quick-preview]
```ts twoslash
import { compress } from 'headroom-ai';
const messages = [
{ role: 'user' as const, content: 'Analyze these results' },
];
const result = await compress(messages, { model: 'gpt-4o' });
// compressionRatio is tokensAfter / tokensBefore, so savings is 1 - ratio.
console.log(`Saved ${result.tokensSaved} tokens (${((1 - result.compressionRatio) * 100).toFixed(0)}%)`);
```
```python
from headroom import compress
from openai import OpenAI
messages = [{"role": "user", "content": "Analyze these results"}]
result = compress(messages, model="gpt-4o")
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=result.messages,
)
print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
```
## What gets compressed [#what-gets-compressed]
| Content type | What happens | Typical savings |
| -------------------------- | ------------------------------------------------------------ | ------------------------------------- |
| JSON arrays (tool outputs) | Statistical analysis keeps errors, anomalies, boundaries | varies with array size and redundancy |
| Source code | AST-aware compression preserves signatures, collapses bodies | opt-in; disabled by default |
| Build/test logs | Keeps failures and errors, drops passing noise | varies with log verbosity |
| Search results | Ranks by relevance, keeps top matches | varies with result set size |
| Plain text | ModernBERT token classification removes redundancy | varies with redundancy |
| Git diffs | Preserves change hunks, drops unchanged context | varies with diff size |
| Images | ML router selects optimal resize/quality tradeoff | varies with image content |
Savings depend heavily on how repetitive the content is -- see
[Benchmarks](/docs/benchmarks) and the reproducible scenario table below for
measured numbers on real workloads.
## Where Headroom fits [#where-headroom-fits]
```
Your Agent / App
|
| tool outputs, logs, DB reads, RAG results, file reads, API responses
v
Headroom <-- proxy, Python library, TS SDK, or framework integration
|
v
LLM Provider (OpenAI, Anthropic, Google, Bedrock, 100+ via LiteLLM)
```
Headroom works as a **transparent proxy** (zero code changes), a **Python function** (`compress()`), a **TypeScript function** (`compress()`), or a **framework integration** (LangChain, Agno, Strands, LiteLLM, Vercel AI SDK, MCP).
## Real-world results [#real-world-results]
Headroom is built for the scenario that matters most in agent workflows: a
single critical error or fact buried inside a large, mostly-routine tool
output. SmartCrusher preserves error items, anomalies (values outside normal
statistical range), and first/last boundaries by construction -- not by
keyword matching, but by statistical analysis of field variance -- so the
signal survives even when the surrounding noise is compressed away
aggressively.
Measured on local, seeded test data built from real MCP server output
formats. These are not production telemetry, and they are reproducible:
| Scenario | Before | After | Savings |
| ------------------------- | ------ | ------ | ------- |
| Code search (100 results) | 17,199 | 13,597 | **21%** |
| SRE incident debugging | 55,957 | 24,340 | **57%** |
| Codebase exploration | 58,801 | 33,895 | **42%** |
| GitHub issue triage | 46,067 | 32,429 | **30%** |
```bash
uv run python benchmarks/index_proof_table.py --seed 20260902
```
Token counts are from the `gpt-5.6` tokenizer. The corpus is generated, so the
exact before/after counts move slightly with the seed; the savings hold across
seeds (Code search 21%, SRE 56--57%, triage 29--30%, exploration 41--50% over
five seeds). Savings depend on how repetitive your tool output is, so treat
these as a shape, not a promise.
See [Benchmarks](/docs/benchmarks) for the wider suite.
## Key Features [#key-features]
## Framework Integrations [#framework-integrations]
## Nothing is lost [#nothing-is-lost]
Compressed content goes into the CCR store (Compress-Cache-Retrieve). The LLM gets a `headroom_retrieve` tool and can fetch full originals when it needs more detail. Compression is aggressive but reversible.
## Next steps [#next-steps]
***
## Install options [#install-options]
**uv tool** - you want the `headroom` CLI (`deploy`, `proxy`, `wrap`, `doctor`, `mcp`, `learn`, and the other CLI commands) installed once on your machine in an isolated app environment.
**pip** - you're writing Python, or you need the CLI, regardless of what language your app is in.
**npm** - you're writing TypeScript/Node and want inline `compress()`, SDK wrapping (`withHeadroom`), or Vercel AI SDK middleware.
## Python [#python]
Headroom requires **Python 3.10+** and is published as `headroom-ai` on PyPI.
Release wheels are built for CPython **3.10 through 3.13** on Linux
(manylinux\_2\_28 x86\_64 / aarch64), macOS (Apple Silicon and Intel), and
Windows x86\_64. The Rust extension uses the CPython stable ABI, so those wheels
are forward-compatible with newer supported CPython versions; optional
dependencies may impose their own Python/platform limits. Platforms outside
that matrix fall back to the source distribution and need a Rust/native
toolchain.
### CLI install with uv [#cli-install-with-uv]
For a host-level `headroom` command on macOS Apple Silicon or Linux, prefer
`uv tool install`. It keeps Headroom in a dedicated app environment instead of
tying it to the current project or shell Python.
```bash
uv tool install --python 3.13 "headroom-ai[all]"
headroom --version
```
On macOS with Homebrew, `python3` may point at a newer interpreter than the
current Headroom wheel set. Passing `--python 3.13` keeps installation on a
wheel-supported interpreter. If Python 3.13 is missing, install it with
Homebrew or let uv download a managed interpreter:
```bash
brew install python@3.13
uv tool install --python 3.13 "headroom-ai[all]"
```
If `headroom` is installed but your shell cannot find it, add uv's tool
directory to `PATH`:
```bash
uv tool update-shell
```
For MCP clients such as Codex that do not inherit your interactive shell
`PATH`, configure the absolute executable path returned by `command -v
headroom`:
```toml
[mcp_servers.headroom]
command = "/Users/you/.local/bin/headroom"
args = ["mcp", "serve"]
```
### Core package [#core-package]
```bash
pip install headroom-ai
```
The core package includes the `compress()` function, SmartCrusher, CacheAligner, and live-zone ContentRouter compression. No heavy dependencies.
> **Note:** IntelligentContext / RollingWindow (score-based history dropping) were retired in PR-B1. Headroom compresses fresh tool output and new turns only — it does not drop conversation history.
### Extras [#extras]
Install only what you need, or grab everything with `[all]`:
```bash
pip install "headroom-ai[all]"
```
| Extra | What it adds | Install command |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
| `proxy` | Proxy server, HTTP API, MCP runtime, local ONNX Kompress path | `pip install "headroom-ai[proxy]"` |
| `proxy-prod` | `proxy` plus gunicorn on Unix production hosts | `pip install "headroom-ai[proxy,proxy-prod]"` |
| `ml` | PyTorch/Hugging Face Kompress backend and model tooling | `pip install "headroom-ai[ml]"` |
| `code` | CodeCompressor (tree-sitter AST parsing) | `pip install "headroom-ai[code]"` |
| `memory` | Persistent memory (sqlite-vec, sentence-transformers) — pure-Python default backend, no compiler | `pip install "headroom-ai[memory]"` |
| `vector` | Optional HNSW vector backend (hnswlib) — needs a C++ toolchain; **not in `[all]`** | `pip install "headroom-ai[vector]"` |
| `memory-stack` | Optional Qdrant + Neo4j memory backend helpers; **not in `[all]`** | `pip install "headroom-ai[memory-stack]"` |
| `relevance` | fastembed-based relevance scoring (BAAI/bge-small-en-v1.5, ONNX) | `pip install "headroom-ai[relevance]"` |
| `image` | Image compression (Pillow, ONNX runtime, OCR) | `pip install "headroom-ai[image]"` |
| `reports` | HTML/Markdown report generation (Jinja2) | `pip install "headroom-ai[reports]"` |
| `otel` | OpenTelemetry exporter (OTLP) | `pip install "headroom-ai[otel]"` |
| `voice` | Voice/audio support | `pip install "headroom-ai[voice]"` |
| `voice-train` | Voice training dependencies; **not in `[all]`** | `pip install "headroom-ai[voice-train]"` |
| `mcp` | MCP server tools (`headroom_compress`, `headroom_retrieve`, `headroom_stats`) | `pip install "headroom-ai[mcp]"` |
| `langchain` | LangChain `HeadroomChatModel` wrapper; **not in `[all]`** | `pip install "headroom-ai[langchain]"` |
| `agno` | Agno `HeadroomAgnoModel` wrapper; **not in `[all]`** | `pip install "headroom-ai[agno]"` |
| `strands` | AWS Strands Agents integration; **not in `[all]`** | `pip install "headroom-ai[strands]"` |
| `crewai` | CrewAI integration; **not in `[all]`** | `pip install "headroom-ai[crewai]"` |
| `autogen` | AutoGen AgentChat integration; **not in `[all]`** | `pip install "headroom-ai[autogen]"` |
| `anyllm` | any-llm multi-provider backend (Python 3.11+); **not in `[all]`** | `pip install "headroom-ai[anyllm]"` |
| `bedrock` | Native AWS Bedrock credentials/backend support; **not in `[all]`** | `pip install "headroom-ai[bedrock]"` |
| `evals` | Evaluation framework (GSM8K, SQuAD, BFCL benchmarks) | `pip install "headroom-ai[evals]"` |
| `pytorch-mps` | Apple-GPU (MPS) memory-embedder offload — **macOS only**, not in `[all]` (torch + sentence-transformers); opt in with `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps` | `pip install "headroom-ai[pytorch-mps]"` |
| `html` | HTML main-content extraction with trafilatura | `pip install "headroom-ai[html]"` |
| `spreadsheet` | `.xlsx` / `.xls` ingestion with openpyxl and xlrd | `pip install "headroom-ai[spreadsheet]"` |
| `sandbox` | Torch-free proxy bundle for constrained environments | `pip install "headroom-ai[sandbox]"` |
| `all` | Runtime bundle: `proxy,code,ml,memory,relevance,image,reports,otel,evals,voice,html,mcp,spreadsheet` | `pip install "headroom-ai[all]"` |
`[all]` is the full built-in runtime bundle, not every integration or optional
backend. The rows marked “not in `[all]`” must be requested explicitly.
You can combine extras:
```bash
pip install "headroom-ai[proxy,langchain,ml]"
```
### Source builds and unsupported targets [#source-builds-and-unsupported-targets]
Windows x86\_64, Intel/Apple Silicon macOS, and Linux x86\_64/aarch64 have
release wheels. If pip selects the source distribution on another target (or
because a matching wheel is unavailable), the build needs Rust plus the
platform C/C++ toolchain. On Windows without MSVC on `PATH`, for example,
you'll see:
```text
error: linker `link.exe` not found
note: please ensure that Visual Studio 2017 or later, or Build Tools for
Visual Studio were installed with the Visual C++ option
```
To install the prerequisites:
1. **MSVC toolchain** — install **[Build Tools for Visual Studio](https://visualstudio.microsoft.com/visual-cpp-build-tools/)**
and select the *"Desktop development with C++"* workload (this gives
you `link.exe`). VS Code on its own is not enough.
2. **Rust** — install via [rustup](https://rustup.rs/). Choose the
`stable-x86_64-pc-windows-msvc` toolchain so Cargo uses the MSVC
linker you just installed.
3. **Open a fresh PowerShell** so the installer's PATH updates take
effect, then run the install:
```powershell
uv tool install --python 3.13 "headroom-ai[all]"
# or
pip install "headroom-ai[all]"
```
If you'd rather avoid a source build, use a supported interpreter/platform or
run Headroom through Docker — see the [Docker](#docker) section below.
### pipx [#pipx]
`pipx` creates one virtual environment per app. If that environment uses an
unsupported Python version, `pipx` may resolve an older compatible Headroom
release instead of the newest one.
Use Python 3.13 explicitly. If you already use uv, prefer the
[`uv tool`](#cli-install-with-uv) path above.
```bash
pipx install --python python3.13 "headroom-ai[all]"
```
For a pinned release, replace `` with the version you require:
```bash
pipx install --python python3.13 "headroom-ai[all]=="
```
Check which Python an existing `pipx` environment uses:
```bash
pipx list
```
### Verify the install [#verify-the-install]
```bash
python -c "import headroom; print(headroom.__version__)"
```
## TypeScript / Node.js [#typescript--nodejs]
The TypeScript SDK is published as `headroom-ai` on npm. It requires **Node.js 18+**. It is a library you import — it does **not** install the `headroom` CLI (`headroom wrap`, `headroom proxy`, etc.), which ships only with the Python package above.
```bash
npm install headroom-ai
```
Or with other package managers:
```bash
pnpm add headroom-ai
yarn add headroom-ai
```
The TypeScript SDK sends messages to the Headroom proxy over HTTP for compression. The proxy runs the full compression pipeline (Python). Start it before using the SDK:
```bash
pip install "headroom-ai[proxy]"
headroom proxy --port 8787
```
Then point the SDK at it:
```ts
import { compress } from 'headroom-ai';
const result = await compress(messages, {
baseUrl: 'http://localhost:8787',
});
```
### Verify the install [#verify-the-install-1]
```bash
node -e "const h = require('headroom-ai'); console.log('headroom-ai loaded')"
```
## Docker [#docker]
Pre-built images are published to GitHub Container Registry on every release.
```bash
docker pull ghcr.io/headroomlabs-ai/headroom:latest
docker run -p 8787:8787 ghcr.io/headroomlabs-ai/headroom:latest
```
If you want a host `headroom` CLI that keeps Headroom itself inside a container — with mounted state, a one-line installer, and a persistent Docker lifecycle — see [Docker-Native Install](/docs/docker-install).
### Image tags [#image-tags]
| Tag | Extras | Base image | Description |
| ------------------- | -------------------- | ----------- | ----------------------------------------- |
| `latest` | `proxy,bedrock` | Debian slim | Default image, runs as non-root |
| `` | `proxy,bedrock` | Debian slim | Pinned release, same default variant |
| `nonroot` | `proxy,bedrock` | Debian slim | Explicit non-root variant |
| `code` | `proxy,code,bedrock` | Debian slim | Includes tree-sitter for code compression |
| `code-nonroot` | `proxy,code,bedrock` | Debian slim | Code compression, non-root |
| `slim` | `proxy,bedrock` | Distroless | Minimal image, no shell |
| `slim-nonroot` | `proxy,bedrock` | Distroless | Minimal, non-root |
| `code-slim` | `proxy,code,bedrock` | Distroless | Code compression, minimal |
| `code-slim-nonroot` | `proxy,code,bedrock` | Distroless | Code compression, minimal, non-root |
### Build from source [#build-from-source]
Use Docker Bake for multi-variant builds:
```bash
# List all targets
docker buildx bake --list targets
# Build the default runtime image
docker buildx bake runtime-default
# Build a specific variant with custom registry
docker buildx bake runtime-code-slim-nonroot \
--set '*.tags=my-registry/headroom:code-slim-nonroot'
```
## Environment variables [#environment-variables]
These variables configure Headroom at runtime. Set them in your shell, `.env` file, or container environment.
### LLM provider keys [#llm-provider-keys]
| Variable | Description |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `OPENAI_API_KEY` | OpenAI API key (used when proxying to OpenAI) |
| `ANTHROPIC_API_KEY` | Anthropic API key (used when proxying to Anthropic) |
| `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | AWS credentials for Bedrock backend |
| `GOOGLE_APPLICATION_CREDENTIALS` | Google Cloud credentials for Vertex AI backend |
| `VERTEXAI_PROJECT` | GCP project id for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_PROJECT`; set it explicitly to avoid silently billing your ADC default quota project) |
| `VERTEXAI_LOCATION` | GCP region for the Vertex AI backend (LiteLLM-specific — distinct from `GOOGLE_CLOUD_LOCATION`) |
The Vertex AI backend also requires `google-cloud-aiplatform>=1.38`, which is not
included in any extra or Docker image — see
[Google Vertex AI](/docs/proxy#google-vertex-ai) for setup details.
### Proxy configuration [#proxy-configuration]
| Variable | Default | Description |
| -------------------------- | ----------- | ------------------------------------------------------------------- |
| `HEADROOM_PORT` | `8787` | Port the proxy listens on |
| `HEADROOM_HOST` | `127.0.0.1` | Host the proxy binds to |
| `HEADROOM_MODE` | `cache` | Default optimization mode: `token` or `cache` |
| `HEADROOM_TELEMETRY` | `off` | Set to `on` for local-only usage stats (nothing is sent externally) |
| `HEADROOM_REQUEST_TIMEOUT` | `300` | Request timeout in seconds |
### TypeScript SDK [#typescript-sdk]
| Variable | Default | Description |
| ------------------- | ----------------------- | ---------------------------------- |
| `HEADROOM_BASE_URL` | `http://localhost:8787` | Proxy URL for the TypeScript SDK |
| `HEADROOM_API_KEY` | *(none)* | API key if the proxy requires auth |
## Troubleshooting [#troubleshooting]
These are common issues faced during initial setup and how to resolve them.
### Python version error [#python-version-error]
This project requires **Python 3.10+**.
Check your version:
```bash
python3 --version
```
If needed (Mac with Homebrew):
```bash
brew install python@3.13
```
### Editable install fails (`pip install -e`) [#editable-install-fails-pip-install--e]
Upgrade pip to the latest version:
```bash
python3 -m pip install --upgrade pip
```
### Missing `cargo` (Rust error) [#missing-cargo-rust-error]
Some tests require Rust tooling.
The recommended way to install rust is using `rustup`. You can find the official installation instructions [here](https://rust-lang.org/tools/install/).
## Dashboard [#dashboard]
Headroom serves a live savings dashboard while the proxy is running. Open it with:
```bash
headroom dashboard # opens http://localhost:8787/dashboard in your browser
headroom dashboard --no-open # just print the URL
```
Or browse to `http://localhost:8787/dashboard` directly (use `--port` / `HEADROOM_PORT` if you
run the proxy on a different port).
## Next steps [#next-steps]
Headroom plugs into LangChain at four points: the chat model, tool output, retrieved documents, and conversation history. In LangGraph it also works as a graph node, compressing `ToolMessage` content between the tool step and the agent step.
## Install [#install]
```bash
pip install "headroom-ai[langchain]" # langchain-core + langchain-openai
pip install "headroom-ai[langgraph]" # the above, plus langgraph
```
Provider packages are separate, as they are in LangChain itself:
```bash
pip install langchain-anthropic # ChatAnthropic
pip install langchain # create_agent, init_chat_model
pip install langchain-classic # ContextualCompressionRetriever
```
LangChain 1.0 removed several modules that older Headroom docs referenced. `langchain.memory` and `langchain.retrievers` no longer exist, and `create_openai_tools_agent` and `AgentExecutor` are gone from `langchain.agents`. The [migration table](#migrating-from-langchain-0x) at the bottom of this page maps each one to its replacement. Verified against langchain-core 1.6.1, langchain 1.3.18 and langgraph 1.2.11.
## Chat model [#chat-model]
`HeadroomChatModel` wraps any LangChain chat model. Messages are compressed on the way out; everything else about the model is unchanged.
```python
from langchain_openai import ChatOpenAI
from headroom.integrations import HeadroomChatModel
llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
response = llm.invoke("Hello!")
print(llm.get_savings_summary())
# {'total_requests': 1, 'total_tokens_saved': 0, 'average_savings_percent': 0.0,
# 'total_tokens_before': 9, 'total_tokens_after': 9}
```
Short prompts compress to nothing, which is the intended behaviour — savings appear once tool output and history are in the context.
Any provider works:
```python
from langchain_anthropic import ChatAnthropic
from headroom.integrations import HeadroomChatModel
llm = HeadroomChatModel(ChatAnthropic(model="claude-sonnet-4-20250514"))
```
Tool binding is forwarded to the underlying model, so `llm.bind_tools(...)` and the agent runtimes built on it behave the same as the unwrapped model.
## Agents [#agents]
`wrap_tools_with_headroom` compresses tool output before it re-enters the agent's context. The wrapped tool keeps the original argument schema, so the model sees the same parameters.
```python
import json
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langchain.agents import create_agent
from headroom.integrations import HeadroomChatModel, wrap_tools_with_headroom
@tool
def query_database(query: str) -> str:
"""Query the users database. Returns JSON rows."""
return json.dumps({"results": [...], "total": 300})
llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o-mini"))
tools = wrap_tools_with_headroom([query_database], min_chars_to_compress=1000)
agent = create_agent(llm, tools)
result = agent.invoke({
"messages": [("user", "How many users signed up last week?")]
})
```
`create_agent` comes from the `langchain` package. LangGraph's `create_react_agent` still works and is a drop-in substitute, but it is deprecated as of LangGraph 1.0 and slated for removal in 2.0.
Per-tool metrics are collected globally:
```python
from headroom.integrations import get_tool_metrics
print(get_tool_metrics().get_summary())
# {'total_invocations': 1, 'total_compressions': 1, 'total_chars_saved': 4202,
# 'average_compression_ratio': 0.901,
# 'by_tool': {'query_database': {'invocations': 1, 'compressions': 1, 'chars_saved': 4202}}}
```
Async agents work too — the wrapped tool exposes a coroutine, so `await agent.ainvoke(...)` compresses on the async path as well.
## LangGraph compression node [#langgraph-compression-node]
Wrapping tools and compressing in the graph are not equivalent, and the difference is large. On the same 300-row JSON result:
| Path | Result | Saved |
| ------------------------------------ | -----------: | ----: |
| `wrap_tools_with_headroom` | 38,395 chars | 10% |
| `create_compress_tool_messages_node` | 18,390 chars | 57% |
The tool wrapper routes through the MCP compressor, which deliberately keeps its output parseable as JSON for downstream tool consumers. That rules out the schema-hoisting rewrite — the one that turns 300 repeated objects into a single schema line plus CSV rows — so on an array of similar records it mostly removes whitespace. The graph node has no such constraint, because the value only has to be read by the model.
Use the graph node when the output is going to the model and nothing else parses it. Use tool wrapping when something downstream still needs JSON.
```python
from langgraph.graph import StateGraph, MessagesState, START, END
from headroom.integrations.langchain import create_compress_tool_messages_node
graph = StateGraph(MessagesState)
graph.add_node("agent", agent_node)
graph.add_node("tools", tools_node)
graph.add_node("compress", create_compress_tool_messages_node(
min_tokens_to_compress=100,
))
graph.add_edge(START, "agent")
graph.add_edge("tools", "compress")
graph.add_edge("compress", "agent")
app = graph.compile()
```
On a 300-row JSON tool result — 42,597 characters — the node returns 18,390, a 57% reduction, with `tool_call_id` preserved so the graph stays valid.
To call it directly rather than as a node:
```python
from headroom.integrations.langchain import compress_tool_messages
result = compress_tool_messages(state["messages"], min_tokens_to_compress=100)
result.messages # the rewritten message list
result.messages_compressed # 1
result.total_tokens_saved # 6052
```
Messages below the threshold are returned unchanged.
## Memory [#memory]
`HeadroomChatMessageHistory` wraps any `BaseChatMessageHistory` and compresses older turns once the history crosses a token budget.
```python
from langchain_core.chat_history import InMemoryChatMessageHistory
from headroom.integrations import HeadroomChatMessageHistory
history = HeadroomChatMessageHistory(
InMemoryChatMessageHistory(),
compress_threshold_tokens=4000, # start compressing above 4K tokens
keep_recent_turns=5, # never touch the last 5 turns
)
history.add_user_message("...")
history.add_ai_message("...")
print(history.get_compression_stats())
# {'compression_count': 0, 'total_tokens_saved': 0,
# 'threshold_tokens': 4000, 'keep_recent_turns': 5}
```
Use it anywhere a chat history is accepted:
```python
from langchain_core.runnables.history import RunnableWithMessageHistory
chain = RunnableWithMessageHistory(llm, lambda session_id: history)
```
`ConversationBufferMemory` was removed in LangChain 1.0. `RunnableWithMessageHistory`, or a LangGraph checkpointer, replaces it.
## Retriever [#retriever]
`HeadroomDocumentCompressor` scores retrieved documents against the query and keeps the best. Retrieve widely for recall, then narrow for precision.
```python
from headroom.integrations import HeadroomDocumentCompressor
compressor = HeadroomDocumentCompressor(
max_documents=10,
min_relevance=0.3,
prefer_diverse=True, # MMR-style diversity
)
docs = compressor.compress_documents(retrieved_docs, "What is Python?")
```
It is a real `langchain_core.documents.compressor.BaseDocumentCompressor`, so it also drops into the classic retriever pattern:
```python
from langchain_classic.retrievers import ContextualCompressionRetriever
retriever = ContextualCompressionRetriever(
base_compressor=compressor,
base_retriever=vectorstore.as_retriever(search_kwargs={"k": 50}),
)
docs = retriever.invoke("What is Python?") # retrieves 50, returns 10
```
`ContextualCompressionRetriever` moved out of `langchain.retrievers` in 1.0 and now ships in `langchain-classic`.
## Streaming [#streaming]
```python
for chunk in llm.stream("Tell me a story"):
print(chunk.content, end="", flush=True)
response = await llm.ainvoke("Hello!")
async for chunk in llm.astream("Tell me a story"):
print(chunk.content, end="", flush=True)
```
## Configuration [#configuration]
```python
from headroom import HeadroomConfig, HeadroomMode
from headroom.integrations import HeadroomChatModel
from langchain_openai import ChatOpenAI
config = HeadroomConfig(default_mode=HeadroomMode.OPTIMIZE)
llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"), config=config)
```
## Migrating from LangChain 0.x [#migrating-from-langchain-0x]
| Removed in LangChain 1.0 | Use instead | Package |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ------------------- |
| `langchain.memory.ConversationBufferMemory` | `RunnableWithMessageHistory`, or a LangGraph checkpointer | `langchain-core` |
| `langchain_community.chat_message_histories.ChatMessageHistory` | `langchain_core.chat_history.InMemoryChatMessageHistory` | `langchain-core` |
| `langchain.retrievers.ContextualCompressionRetriever` | same class, moved | `langchain-classic` |
| `langchain.agents.create_openai_tools_agent` + `AgentExecutor` | `langchain.agents.create_agent` | `langchain` |
| `langgraph.prebuilt.create_react_agent` | `langchain.agents.create_agent` (the old name still works, deprecated in LangGraph 1.0) | `langchain` |
| `langchain_community.vectorstores.*` | provider packages, or `langchain_core.vectorstores.InMemoryVectorStore` for tests | varies |
Headroom's own API did not change across the LangChain 1.0 boundary. `HeadroomChatModel`, `HeadroomChatMessageHistory`, `HeadroomDocumentCompressor` and `wrap_tools_with_headroom` take the same arguments they always did.
Headroom is designed to compress LLM context without losing accuracy. This page documents when it helps, when it does not, and the safety gates that prevent harmful compression.
## When Headroom Helps vs. Does Not [#when-headroom-helps-vs-does-not]
The compression figures below are not from a published, repeatable
benchmark suite -- we could not trace a prior version of this table to one.
Instead of an unsourced number, each row is a directional single-sample
measurement taken with `compress()` against a synthetic payload of that
shape. For a seeded, committed harness that reproduces figures of this kind
exactly, run `uv run python benchmarks/index_proof_table.py --seed 20260902`.
Your own traffic will
vary; measure it with `client.chat.completions.simulate()` (see
[Troubleshooting](/docs/troubleshooting#use-simulation-to-inspect-transforms))
rather than treating these as guarantees.
| Content Type | Compression (single-sample) | Latency Impact | Best For |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------- | -------------------------------- | --------------------------- |
| **JSON: Arrays of dicts** (search results, API responses, DB rows) | \~52% (200-item sample) | Net latency win on Sonnet/Opus | Primary use case |
| **JSON: Arrays of strings** (file paths, log lines, tags) | \~95% (300-item sample) | Net latency win | String dedup + sampling |
| **JSON: Arrays of numbers** (metrics, time series) | \~97% (500-item sample) | Net latency win | Statistical summary |
| **JSON: Mixed-type arrays** | \~45% (200-item sample) | Net latency win | Group-by-type compression |
| **Structured logs** (as JSON) | Varies widely with duplication -- see methodology note | Net latency win | Log entries in tool outputs |
| **Agentic conversations** (multi-turn) | Not independently measured for this page | Break-even to net win | Multi-tool agent sessions |
| **Plain text** (documentation, articles) | Requires the optional `kompress`/ML text path; not exercised in this measurement pass | Adds latency (cost savings only) | Cost optimization |
| **Code** | Passthrough | Minimal overhead | See below |
| **RAG document contexts** | Passthrough | Minimal overhead | Not compressed |
### Where Headroom Adds the Most Value [#where-headroom-adds-the-most-value]
* Long agent sessions with accumulated tool outputs
* JSON-heavy workflows -- API responses, database queries
* Build and test output
* Multi-tool agents (repeated tool results compound the per-call savings above)
### Where Headroom Adds Little Value [#where-headroom-adds-little-value]
* Short conversational exchanges -- overhead can exceed savings on small payloads
* Code-only sessions (reading/writing files) -- code passes through
* Single-turn requests with no accumulated context
## What Headroom Does NOT Compress [#what-headroom-does-not-compress]
* **Short messages** (below `min_tokens_to_compress`, 50 tokens by default -- `headroom/transforms/content_router.py:4806`) -- overhead exceeds savings
* **Source code** -- passes through unchanged to preserve correctness
* **grep/search results** -- compact structured format, already minimal
* **Images** -- counted at fixed token cost (\~1,600 tokens), not compressed
* **System prompts** -- preserved for prefix cache compatibility
## Code Compression [#code-compression]
Headroom includes an AST-aware CodeCompressor (tree-sitter, 11 languages: Python, JavaScript, TypeScript, Go, Rust, Java, C, C++, Perl, C#, PHP) but it is gated behind safety protections that prevent it from firing in most real-world scenarios. This is intentional.
**Why code mostly passes through:**
1. **Size gates, not a word count**: a message under `min_tokens` (50 tokens by default; savings profiles set this anywhere from 10-250) is skipped, and a block under `min_chars_for_block_compression` (500 characters) is skipped -- both measured in tokens/characters, never words (`headroom/transforms/content_router.py:4806,1596`)
2. **Recent code protection** (`protect_recent_code=4`): Code in the last 4 messages is never compressed
3. **Analysis intent protection** (`protect_analysis_context=True`): If the most recent user message contains keywords like "analyze", "review", "explain", "fix", "debug" -- ALL code in the conversation is protected
**Why this is the right default**: Code is almost always fetched because the user wants to work with it. Compressing function bodies would remove exactly what they need.
**Where code savings come from**: Headroom compresses code in the live zone — the newest tool outputs and content blocks — with the AST-aware CodeCompressor, while keeping recent and analysis-context code fully intact. It never drops messages from the conversation history or strips function bodies.
**Override**: Set `protect_analysis_context=False` in `ContentRouterConfig` for aggressive code compression. Requires `headroom-ai[code]` for tree-sitter.
## JSON Compression Constraints [#json-compression-constraints]
### What Gets Compressed [#what-gets-compressed]
* Arrays of **dicts**: Full statistical analysis with adaptive K (Kneedle algorithm)
* Arrays of **strings**: Dedup + adaptive sampling + error preservation
* Arrays of **numbers**: Statistical summary + outlier/change-point preservation
* **Mixed-type** arrays: Grouped by type, each group compressed independently
* **Nested** objects: Recursed into, arrays within are compressed (up to depth 5)
### What Passes Through [#what-passes-through]
* Arrays below 5 items (`min_items_to_analyze`)
* Content below 200 tokens (`min_tokens_to_crush`)
* Bool-only arrays
* JSON objects without array values
* Malformed JSON (silently passes through, no error)
### Edge Cases [#edge-cases]
* **NaN/Infinity** in numeric fields: Filtered out before statistics are computed
* **Nesting depth > 5**: Inner arrays not examined for compression
* **Mixed-type arrays with small groups**: Groups below `min_items_to_analyze` are kept as-is
## Safety Gates [#safety-gates]
All compressors follow the same principle: **fail gracefully, return original content unchanged**.
* Invalid JSON passes through (no error raised)
* AST parse failure falls back to the original content, unchanged
* Compression that makes output larger returns the original
* A missing optional dependency (tree-sitter) causes a passthrough with a warning log
* Errors are logged at WARNING level and never propagated to callers
LLMLingua is not part of Headroom 0.37.0. The `[llmlingua]` extra was removed
in 0.9.x with no live code path using it (`pyproject.toml:115`); text
compression today goes through the Kompress (ModernBERT ONNX) path instead,
which fails open the same as every other compressor.
## Adaptive K: How Item Retention Works [#adaptive-k-how-item-retention-works]
SmartCrusher does not use fixed K values. It uses information-theoretic sizing:
1. **Kneedle algorithm** on bigram coverage curves finds the point where adding more items stops providing new information
2. **SimHash** fingerprinting detects near-duplicate items
3. **zlib validation** ensures the subset captures the full set's diversity
The resulting K is split: 30% from array start, 15% from end, 55% for importance-scored items.
**Safety guarantees (additive, never dropped):**
* Error items (containing "error", "exception", "failed", "critical") -- across ALL array types
* Numeric anomalies (> 2 standard deviations from mean)
* String length anomalies (> 2 standard deviations from mean length)
* Change points (sudden shifts in running values)
These are kept even if they exceed the K budget.
## Configuration Tuning [#configuration-tuning]
| Parameter | Default | Effect |
| --------------------------- | ------- | ------------------------------------------------------- |
| `min_items_to_analyze` | 5 | Arrays below this pass through |
| `min_tokens_to_crush` | 200 | Content below this passes through |
| `max_items_after_crush` | 15 | Upper bound on retained items |
| `variance_threshold` | 2.0 | Std devs for anomaly detection (lower = more preserved) |
| `protect_analysis_context` | True | Protect code when user asks about it |
| `protect_recent_code` | 4 | Messages from end to protect code in |
| `skip_user_messages` | True | Never compress user messages |
| `toin_confidence_threshold` | 0.3 | Minimum TOIN confidence to apply hints |
## Provider Interactions [#provider-interactions]
* CacheAligner maximizes Anthropic/OpenAI prefix cache hit rates
* Token counting uses model-specific tokenizers (tiktoken for OpenAI, calibrated estimation for Anthropic)
* Compression works with all providers -- no provider-specific limitations
* Compressed content is valid JSON -- downstream tools and parsers work unchanged
## TOIN Cold Start [#toin-cold-start]
The Tool Output Intelligence Network (TOIN) learns compression patterns from usage. For new tool types:
* No learned patterns exist -- falls back to statistical heuristics
* Confidence below `toin_confidence_threshold` (default 0.3) -- TOIN hints ignored
* Patterns build up over time as tools are used repeatedly
* Cross-session learning requires persistence (`TelemetryConfig.storage_path`)
## Telemetry and the Upload Beacon [#telemetry-and-the-upload-beacon]
Headroom ships two independent, separately-defaulted switches
(`headroom/telemetry/beacon.py:1-92`) -- worth knowing about before you treat
this page's "nothing leaves the machine" framing as universal:
* **`HEADROOM_TELEMETRY`** -- off by default, opt-in. Aggregates compression
stats locally for the in-process collector and the `/stats` endpoint.
Never leaves the machine.
* **`HEADROOM_BEACON`** -- **on by default, opt-out.** Uploads an anonymous
summary of how compression behaved, so we can see when a release regresses a
ratio or starts skipping a content type across real workloads instead of only
our own test corpus. It carries counters and identifiers only: token totals,
compression ratios, skip reasons, provider and model ids, failure counts, plus
OS and architecture. Never prompts, completions, code, or file paths -- the
collector allowlists fields server-side and drops the rest. Disable it with
`HEADROOM_BEACON=off`, the cross-tool `DO_NOT_TRACK=1` convention, or by
running offline. `is_beacon_enabled()` is fail-open: an unrecognized value
uploads rather than staying silent. See [Proxy](/docs/proxy) for the full
field list.
These are deliberately separate so that enabling local `/stats` does not
silently start an upload on the next release.
Headroom integrates with [LiteLLM](https://github.com/BerriAI/litellm) as a callback that compresses messages before they reach any provider. One line to enable, works with all 100+ LiteLLM-supported providers.
Looking for the proxy's `--backend vertex_ai` / `bedrock` options
instead? Those make the Headroom **proxy** call cloud providers through LiteLLM — a
different mechanism from the callback documented here. See
[Cloud providers](/docs/proxy#cloud-providers).
## Installation [#installation]
```bash
pip install headroom-ai litellm
```
## Quick start [#quick-start]
```python
import litellm
from headroom.integrations.litellm_callback import HeadroomCallback
litellm.callbacks = [HeadroomCallback()]
# All calls now compressed automatically
response = litellm.completion(model="gpt-4o", messages=[...])
response = litellm.completion(model="bedrock/claude-sonnet", messages=[...])
response = litellm.completion(model="azure/gpt-4o", messages=[...])
```
The callback compresses messages in LiteLLM's `async_pre_call_hook` before they reach the provider.
## How it works [#how-it-works]
1. You call `litellm.completion()` with your messages
2. `HeadroomCallback.async_pre_call_hook` compresses the messages
3. LiteLLM sends the compressed messages to the provider
4. The response comes back unchanged
This works with every provider LiteLLM supports: OpenAI, Anthropic, Bedrock, Azure, Vertex AI, Cohere, Groq, Mistral, Together, Ollama, and more.
## With LiteLLM Proxy [#with-litellm-proxy]
If you run LiteLLM as a proxy server, use the ASGI middleware:
```python
from litellm.proxy.proxy_server import app
from headroom.integrations.asgi import CompressionMiddleware
app.add_middleware(CompressionMiddleware)
```
Or configure via YAML:
```yaml
# litellm_config.yaml
litellm_settings:
callbacks: ["headroom.integrations.litellm_callback.HeadroomCallback"]
```
## Direct compress() with LiteLLM [#direct-compress-with-litellm]
You can also use `compress()` directly instead of the callback:
```python
import litellm
from headroom import compress
messages = [{"role": "user", "content": large_content}]
compressed = compress(messages, model="bedrock/claude-sonnet")
response = litellm.completion(
model="bedrock/claude-sonnet",
messages=compressed.messages,
)
print(f"Saved {compressed.tokens_saved} tokens")
```
## ASGI middleware [#asgi-middleware]
Drop-in middleware for any ASGI application. Intercepts `/v1/messages`, `/v1/chat/completions`, `/v1/responses`, and `/chat/completions`:
```python
from fastapi import FastAPI
from headroom.integrations.asgi import CompressionMiddleware
app = FastAPI()
app.add_middleware(CompressionMiddleware)
```
Response headers include `x-headroom-compressed: true` and `x-headroom-tokens-saved: 1234`.
## Over HTTP (guardrail / gateway) [#over-http-guardrail--gateway]
The options above run Headroom **in** the LiteLLM process. If instead LiteLLM runs as its own proxy and you want it to call Headroom over the network — the guardrail deployment — point it at [`POST /v1/compress`](/docs/proxy#post-v1compress). LiteLLM swaps `messages` for the compressed result and forwards to the provider.
Two things this deployment needs — the proxy extras, and the proxy itself running with remote access opted in:
```bash
pip install "headroom-ai[proxy]"
# Headroom is loopback-only by default and answers remote callers with 404.
HEADROOM_COMPRESS_ALLOW_REMOTE=1 headroom proxy
```
Without `HEADROOM_COMPRESS_ALLOW_REMOTE=1` a remote caller gets `404`, not `403` — so a misconfigured guardrail looks exactly like a wrong URL. If you also set `HEADROOM_PROXY_TOKEN`, send it as `X-Headroom-Proxy-Token` or you get `401`.
Leave `config.mode` unset. The default pipeline is marker-free, which is what a forward-only caller wants: `mode: "ccr"` emits retrieval markers that are a dangling pointer unless you also inject the `headroom_retrieve` tool and can reach `/v1/retrieve`.
Because LiteLLM passes model names through, send the real one — `claude-sonnet-4-6`, `bedrock/anthropic.claude-3-5-sonnet`, `gemini-2.5-pro` — so Headroom resolves the right tokenizer and context limit. Anthropic-shaped messages need no conversion; see [Message format](/docs/proxy#message-format).
For multi-turn agent loops, set `config.frozen_message_count` to the number of messages the provider has already cached, **and send back the messages you previously forwarded rather than the pristine originals**. Getting this wrong silently destroys the provider's prefix cache — see [Multi-turn usage](/docs/proxy#multi-turn-usage-keeping-the-prefix-cache) for the loop.
Local models do not charge per token, but they still pay for every prompt token during prefill. On Apple Silicon and other local inference setups, long coding-agent sessions often bottleneck on prompt processing rather than generation speed. Headroom can help by sending fewer prompt tokens to the local server.
This workflow measures that effect with the proxy dashboard: run the same task once with optimization disabled, reset the agent state, run it again with optimization enabled, and compare token counts.
Joe Maddalone demonstrated this workflow in [Cut Local LLM Prompt Processing 30% on a Mac with Headroom](https://www.youtube.com/watch?v=j6U_kKiMXgo). His June 2026 demo used an OpenAI-compatible local server, a coding-agent refactor task, `--no-optimize` for the baseline, and the dashboard to compare sessions.
## Setup [#setup]
Start your local OpenAI-compatible model server first. Examples include MLX/OMLX, vLLM, LM Studio, Ollama's OpenAI-compatible endpoint, or another server that accepts `/v1/chat/completions` or `/v1/responses`.
For this guide, assume the local server is listening on `http://127.0.0.1:8000`.
```bash
pip install "headroom-ai[proxy]"
```
## 1. Baseline passthrough run [#1-baseline-passthrough-run]
Start Headroom as a transparent proxy with optimization disabled:
```bash
headroom proxy \
--port 8787 \
--openai-api-url http://127.0.0.1:8000 \
--no-optimize
```
Point your agent or app at Headroom, not directly at the local server:
```bash
export OPENAI_BASE_URL=http://127.0.0.1:8787/v1
export OPENAI_API_KEY=local
```
Run a realistic task. Coding-agent refactors are good benchmark candidates because they produce repeated file reads, tool results, lint/test output, and a growing conversation context.
Open the dashboard while the run is active:
```bash
headroom dashboard --port 8787 --no-open
# or open http://127.0.0.1:8787/dashboard
```
Record the baseline session totals. With `--no-optimize`, before and after token counts should match because Headroom is only forwarding traffic.
## 2. Reset the task [#2-reset-the-task]
Before the optimized run, reset the benchmark state so the second run is comparable:
* revert the code or data changes made by the first run
* start a fresh agent session
* use the same model and local server
* use the same prompt
* avoid changing unrelated flags or server settings
For coding-agent tests, a clean git worktree is the simplest reset point.
## 3. Optimized run [#3-optimized-run]
Restart Headroom without `--no-optimize`:
```bash
headroom proxy \
--port 8787 \
--openai-api-url http://127.0.0.1:8000
```
Run the same task again with the same `OPENAI_BASE_URL` and prompt. Watch the dashboard's before/after token counts for the session.
The savings percentage is the prompt-token reduction sent upstream to the local model. That does not make the model's prefill kernel faster; it reduces how much prompt the kernel has to process.
## Optional: traffic learning [#optional-traffic-learning]
After you have a baseline, you can test learning-enabled runs:
```bash
headroom proxy \
--port 8787 \
--openai-api-url http://127.0.0.1:8000 \
--learn
```
`--learn` implies memory and lets Headroom learn recurring traffic patterns from proxy sessions. Treat this as a separate benchmark condition: compare passthrough, optimized, and optimized-with-learning runs independently.
## What to report [#what-to-report]
For a useful local prefill benchmark, include:
| Field | Example |
| ---------------- | ---------------------------------------------------------- |
| Local server | MLX, vLLM, LM Studio, Ollama-compatible endpoint |
| Model | local model name and quantization, if relevant |
| Hardware | Mac model, RAM, or GPU/CPU target |
| Agent/client | coding agent or app name |
| Task | short description of the repeated task |
| Baseline tokens | dashboard before/after total with `--no-optimize` |
| Optimized tokens | dashboard before/after total without `--no-optimize` |
| Savings | dashboard percentage |
| Notes | whether `--learn`, `--memory`, or other flags were enabled |
## Interpreting results [#interpreting-results]
Local inference changes the value proposition:
* Hosted APIs: fewer input tokens usually means lower cost and lower latency.
* Local models: fewer input tokens primarily means less prefill work and lower memory pressure.
Long-running agent sessions tend to show larger gains than short chat turns because repeated file reads, tool outputs, and logs create more compressible context. If a task is mostly short natural-language turns, expect smaller savings.
Do not compare a cold first run against a warmed second run and attribute all improvement to compression. Keep the server, model, prompt, and agent task stable, and use the dashboard token counts as the primary measurement.
Headroom's MCP server exposes compression, retrieval, and observability as tools that any MCP-compatible AI coding tool can call -- Claude Code, Cursor, Codex, and more. No proxy required.
## Installation [#installation]
```bash
# MCP tools only (lightweight)
pip install "headroom-ai[mcp]"
# Or with the proxy
pip install "headroom-ai[proxy]"
```
## Setup for Claude Code [#setup-for-claude-code]
```bash
# Register with Claude Code (one-time)
headroom mcp install
# Start Claude Code — it now has headroom tools
claude
```
Claude Code can now compress content on demand, retrieve originals, and check session stats.
For automatic compression of **all** traffic, also run the proxy:
```bash
# Terminal 1
headroom proxy
# Terminal 2
ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude
```
## Tools [#tools]
### headroom\_compress [#headroom_compress]
Compress content on demand. The LLM calls this when it wants to shrink large content before reasoning over it.
**Parameters:**
* `content` (required) -- text to compress (files, JSON, logs, search results)
**Returns:**
* `compressed` -- compressed text
* `hash` -- key for retrieving the original later
* `original_tokens` / `compressed_tokens` / `savings_percent`
* `transforms` -- which compression algorithms were applied
Example flow:
```
Claude: Let me compress this large output to save context space.
-> headroom_compress(content="[5000 lines of grep results...]")
<- {
"compressed": "[key matches with context...]",
"hash": "a1b2c3d4e5f6...",
"original_tokens": 12000,
"compressed_tokens": 3200,
"savings_percent": 73.3,
"transforms": ["router:search:0.27"]
}
```
The original is stored locally for 1 hour. If the LLM needs the full content later, it calls `headroom_retrieve`.
### headroom\_retrieve [#headroom_retrieve]
Retrieve original uncompressed content by hash.
**Parameters:**
* `hash` (required) -- hash key from a previous compression
* `query` (optional) -- search within the original to return only matching items
**Returns:**
* `original_content` (full retrieval) or `results` (filtered search)
* `source` -- `"local"` or `"proxy"`
Retrieval checks the local store first, then falls back to the proxy's store. Hashes from either source work transparently.
### headroom\_stats [#headroom_stats]
Session compression statistics.
**Returns:**
* `compressions`, `retrievals`, `tokens_saved`, `savings_percent`
* `estimated_cost_saved_usd`
* `recent_events` -- last 10 compression/retrieval events
* `sub_agents` -- stats from sub-agent MCP instances
* `combined` -- main + sub-agent totals
* `proxy` -- request count, cache hits, cost saved (if proxy is running)
Sub-agent stats are aggregated via a shared stats file at `~/.headroom/session_stats.jsonl`.
## CLI commands [#cli-commands]
```bash
# Install (registers with Claude Code)
headroom mcp install
headroom mcp install --proxy-url http://host:9000 # Custom proxy URL
headroom mcp install --force # Overwrite existing
# Check status
headroom mcp status
# Uninstall
headroom mcp uninstall
# Debug mode
headroom mcp serve --debug
# Streamable HTTP mode
headroom mcp serve --transport http --host 127.0.0.1 --port 8788 --path /mcp
```
## MCP host configuration [#mcp-host-configuration]
For MCP hosts that let you configure a local stdio server, point them at `headroom mcp serve`. If you also run the proxy, pass the proxy URL explicitly so retrieval and stats come from the intended proxy instance.
If you are publishing or consuming Headroom through an MCP registry, use the canonical descriptor at `https://github.com/headroomlabs-ai/headroom/blob/main/server.json`. It captures the package form for `headroom-ai[mcp]` and the current `headroom mcp serve` launch contract in one file.
```json
{
"mcpServers": {
"headroom": {
"type": "stdio",
"command": "headroom",
"args": ["mcp", "serve", "--proxy-url", "http://127.0.0.1:8787"]
}
}
}
```
For multiple proxy instances, register one stdio MCP server per proxy URL:
```json
{
"mcpServers": {
"headroom": {
"type": "stdio",
"command": "headroom",
"args": ["mcp", "serve", "--proxy-url", "http://127.0.0.1:8787"]
},
"headroom-azure": {
"type": "stdio",
"command": "headroom",
"args": ["mcp", "serve", "--proxy-url", "http://127.0.0.1:8788"]
}
}
}
```
If you need Streamable HTTP instead of stdio, run `headroom mcp serve --transport http` and point your MCP host at that endpoint. The default path is `/mcp`, and the default port is `8788`.
Do not assume that a running proxy exposes an HTTP MCP endpoint at `/mcp`. The proxy serves its own API; it does not automatically provide the MCP HTTP transport.
### `command: "headroom"` fails to start [#command-headroom-fails-to-start]
The configurations above use `"command": "headroom"`, which only works if the `headroom`
executable is on the PATH your MCP host (Codex, etc.) sees at startup. If you installed Headroom
into a project virtualenv — for example with `uv add headroom-ai` — the CLI lives only inside
that venv, and the host fails at launch with:
```text
MCP client for `headroom` failed to start: MCP startup failed: No such file or directory (os error 2)
```
Install Headroom so it's globally on PATH — `uv tool install "headroom-ai[mcp]"` (or
`pipx install "headroom-ai[mcp]"`) — or replace `"headroom"` with the absolute path to the binary
(`command -v headroom`, or `where headroom` on Windows).
## Cross-tool compatibility [#cross-tool-compatibility]
| Tool | MCP Support | Setup |
| ------------ | ------------ | ----------------------------- |
| Claude Code | Native | `headroom mcp install` |
| Cursor | Supported | Add to Cursor MCP settings |
| Codex | If supported | Configure MCP server |
| Any MCP host | Yes | Point to `headroom mcp serve` |
## Architecture [#architecture]
For user-managed Serena drift, run `headroom mcp reconcile` to inspect the current recommendation. Add `--adopt` only when you want Headroom to replace the Serena entry.
### MCP only (no proxy) [#mcp-only-no-proxy]
The LLM calls `headroom_compress` on demand. Compression happens locally in the MCP process. Originals are stored in a local `CompressionStore` with 1-hour TTL.
### MCP + Proxy (full setup) [#mcp--proxy-full-setup]
The proxy compresses all traffic at the HTTP level (before the LLM sees content). MCP tools operate after the LLM receives content. They handle different data and do not double-compress.
`headroom_retrieve` checks the local store first, then falls back to the proxy's store.
## Troubleshooting [#troubleshooting]
**"MCP SDK not installed"** -- Run `pip install "headroom-ai[mcp]"`.
**"Proxy not running"** -- Start the proxy with `headroom proxy` in another terminal. Only needed for proxy-backed retrieval.
**"Entry not found or expired"** -- Local content expires after 1 hour, proxy content after 30 minutes (configurable via `HEADROOM_CCR_TTL_SECONDS`).
**Claude doesn't see headroom tools** -- Run `headroom mcp status`, restart Claude Code, and verify with `/mcp` inside Claude Code.
### Claude Code `/usage` attributes a large share to `headroom` MCP [#claude-code-usage-attributes-a-large-share-to-headroom-mcp]
Claude Code counts MCP tool calls and MCP tool results as session context. If a
long-running workflow or a subagent-heavy command calls `headroom_compress`,
`headroom_retrieve`, or `headroom_stats` many times, `/usage` can show a visible
share under the `headroom` MCP server even when Headroom is saving tokens inside
individual tool results.
That number is not a direct "Headroom overhead" bill. It means Claude Code kept
Headroom MCP interactions in the conversation context. Deep research workflows
can amplify this because each subagent has its own requests and may keep its own
MCP results in context.
Use these checks when the MCP share looks high:
* Run `headroom_stats` and compare `tokens_saved` with the number of MCP calls.
* Use `/compact` after large MCP-backed investigation steps so old MCP tool
results stop occupying the active context window.
* Prefer the proxy path for automatic compression of normal Claude Code traffic:
`headroom proxy` plus `ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude`.
* Disable the MCP server for sessions where you only want proxy-level
compression and do not need on-demand `headroom_compress` or
`headroom_retrieve`.
* For deep research or custom subagent workflows, reduce unnecessary subagent
fan-out first; subagent traffic usually dominates the usage picture before
MCP overhead does.
LLMs have two fundamental limitations: context windows overflow with too much history, and every conversation starts from zero. Persistent Memory solves both by extracting key facts, persisting them, and injecting them when relevant.
This is **temporal compression** -- instead of carrying 10,000 tokens of conversation history, carry 100 tokens of extracted memories.
## Quick Start [#quick-start]
```python
from openai import OpenAI
from headroom import with_memory
# One line -- that's it
client = with_memory(OpenAI(), user_id="alice")
# Use exactly like normal
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "I prefer Python for backend work"}]
)
# Memory extracted INLINE -- zero extra latency
# Later, in a new conversation...
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What language should I use?"}]
)
# Response uses the Python preference from memory
```
## How It Works [#how-it-works]
The `with_memory()` wrapper intercepts every chat completion call:
1. **Inject** -- Semantic search finds relevant memories and prepends them to the user message
2. **Instruct** -- Adds a memory extraction instruction to the system prompt
3. **Call** -- Forwards the request to the LLM
4. **Parse** -- Extracts the `` block from the response
5. **Store** -- Saves with embeddings, vector index, and full-text search index
6. **Return** -- Cleans the response (strips the memory block before returning)
Memory extraction happens **inline** as part of the LLM response. No extra API calls, no extra latency.
## Hierarchical Scoping [#hierarchical-scoping]
Memories exist at four scope levels, from broadest to narrowest:
| Scope | Persists Across | Use Case |
| ----------- | ------------------------ | ------------------------------- |
| **User** | All sessions, all time | Long-term preferences, identity |
| **Session** | Current session only | Current task context |
| **Agent** | Current agent in session | Agent-specific context |
| **Turn** | Single turn only | Ephemeral working memory |
```python
from openai import OpenAI
from headroom import with_memory
# Session 1: Morning
client1 = with_memory(
OpenAI(),
user_id="bob",
session_id="morning-session",
)
response = client1.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "I prefer Go for performance-critical code"}]
)
# Memory stored at USER level (persists across sessions)
# Session 2: Afternoon (different session, same user)
client2 = with_memory(
OpenAI(),
user_id="bob",
session_id="afternoon-session",
)
response = client2.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "What language for my new microservice?"}]
)
# Recalls Go preference from morning session
```
## Memory Categories [#memory-categories]
There is no dedicated `category` field on `Memory` or parameter on `add()`
(`headroom/memory/models.py`, `headroom/memory/core.py`) -- store a
free-form category string in the `metadata` dict instead, e.g.
`metadata={"category": "fact"}`. These are the conventional category values
used in Headroom's own examples for organization and retrieval:
| Category | Description | Examples |
| -------------- | ------------------------------------- | ------------------------------------------------- |
| `"preference"` | Likes, dislikes, preferred approaches | "Prefers Python", "Likes dark mode" |
| `"fact"` | Identity, role, constraints | "Works at fintech startup", "Senior engineer" |
| `"context"` | Current goals, ongoing tasks | "Migrating to microservices", "Working on auth" |
| `"entity"` | Information about entities | "Project Apollo uses React", "Team lead is Sarah" |
| `"decision"` | Decisions made | "Chose PostgreSQL over MySQL" |
| `"insight"` | Derived insights | "User tends to prefer typed languages" |
## Memory API [#memory-api]
The `with_memory()` wrapper exposes a `.memory` attribute for direct access:
```python
client = with_memory(OpenAI(), user_id="alice")
# Search memories (semantic)
results = client.memory.search("python preferences", top_k=5)
for memory in results:
print(f"{memory.content}")
# Add a memory manually
client.memory.add(
"User is a senior engineer",
importance=0.9,
)
# Get all memories for this user
all_memories = client.memory.get_all()
# Clear all memories
client.memory.clear()
# Get stats
# `stats()` currently returns only {"total": N} -- there is no
# per-category breakdown (`headroom/memory/wrapper.py:342-348`).
stats = client.memory.stats()
print(f"Total memories: {stats['total']}")
```
## Temporal Versioning [#temporal-versioning]
When facts change, Headroom creates a **supersession chain** that preserves history:
```python
from headroom.memory import HierarchicalMemory, MemoryFilter
memory = await HierarchicalMemory.create()
# Original fact
# `category` is a convention, not a keyword argument -- `add()` has no
# `category` param (`headroom/memory/core.py`); store it in `metadata`.
orig = await memory.add(
content="User works at Google",
user_id="alice",
metadata={"category": "fact"},
)
# User changes jobs -- supersede the old memory
new = await memory.supersede(
old_memory_id=orig.id,
new_content="User now works at Anthropic",
)
# Query current state (excludes superseded by default)
current = await memory.query(MemoryFilter(
user_id="alice",
include_superseded=False,
))
# Returns only "User now works at Anthropic"
# Get the full chain
chain = await memory.get_history(new.id)
# [
# Memory(content="User works at Google", is_current=False),
# Memory(content="User now works at Anthropic", is_current=True),
# ]
```
This gives you an audit trail, the ability to debug why the LLM made certain decisions, and rollback if needed.
## Backends [#backends]
### Embedder Backends [#embedder-backends]
```python
from headroom.memory import MemoryConfig, EmbedderBackend
# ONNX embeddings (recommended -- fast, free, private; ~30 MB int8-quantized)
# Note: EmbedderBackend.LOCAL requires PyTorch (~2 GB); use ONNX instead.
config = MemoryConfig(
embedder_backend=EmbedderBackend.ONNX,
embedder_model="BAAI/bge-small-en-v1.5",
)
# OpenAI embeddings (higher quality, costs money)
config = MemoryConfig(
embedder_backend=EmbedderBackend.OPENAI,
openai_api_key="sk-...",
embedder_model="text-embedding-3-small",
)
# Ollama embeddings (local server, many models)
config = MemoryConfig(
embedder_backend=EmbedderBackend.OLLAMA,
ollama_base_url="http://localhost:11434",
embedder_model="nomic-embed-text",
)
```
### Embedding Runtime / GPU Offload (Apple Silicon) [#embedding-runtime--gpu-offload-apple-silicon]
By default the proxy's memory embedder runs on the **ONNX CPU** backend -- fast
and dependency-light, but CPU-only. Under sustained load the embedding step can
saturate the CPU and make the proxy less responsive.
On Apple Silicon you can opt in to running the embedder on the **Apple GPU
(MPS)** instead, which offloads that work off the CPU and keeps the proxy
responsive. Install the extra and set the env var:
```bash
pip install "headroom-ai[pytorch-mps]" # also works as [pytorch_mps]
export HEADROOM_EMBEDDER_RUNTIME=pytorch_mps
```
When set, the embedder runs via the torch sentence-transformers backend on the
Apple GPU instead of the default ONNX CPU embedder. Notes:
* **Strictly opt-in.** `pytorch_mps` is the only accepted value; anything else
(or unset) keeps the default ONNX CPU embedder. Default behavior is unchanged,
and there is no CLI flag -- it is env-var only.
* **Auto-fallback.** It only activates when Apple MPS is actually available
(Apple Silicon + torch). If MPS is unavailable or torch/sentence-transformers
is not installed, it logs a warning and uses the existing default embedder
selection path: ONNX when available, then the pre-existing local
sentence-transformers fallback.
* **MPS serialization.** torch-MPS is not thread-safe, so the embedder
serializes MPS encode calls internally via a single-worker executor. This is
automatic -- there is nothing to configure.
### Storage [#storage]
Storage uses **SQLite** for CRUD and filtering, **HNSW** for vector similarity search, and **FTS5** for full-text keyword search. All embedded -- no external services required.
```python
config = MemoryConfig(
db_path="memory.db",
vector_dimension=384,
hnsw_ef_construction=200,
hnsw_m=16,
hnsw_ef_search=50,
cache_enabled=True,
cache_max_size=1000,
)
```
## Provider Compatibility [#provider-compatibility]
Memory works with any OpenAI-compatible client:
```python
from openai import OpenAI
from headroom import with_memory
# OpenAI
client = with_memory(OpenAI(), user_id="alice")
# Azure OpenAI
client = with_memory(
OpenAI(base_url="https://your-resource.openai.azure.com/..."),
user_id="alice",
)
# Groq
from groq import Groq
client = with_memory(Groq(), user_id="alice")
```
## Performance [#performance]
| Operation | Latency | Notes |
| ----------------- | -------------- | ------------------------------ |
| Memory injection | \<50ms | Local embeddings + HNSW search |
| Memory extraction | +50-100 tokens | Part of LLM response (inline) |
| Memory storage | \<10ms | SQLite + HNSW + FTS5 indexing |
| Cache hit | \<1ms | LRU cache lookup |
Headroom provides comprehensive metrics for monitoring compression performance, cost savings, and system health through both the proxy server and the SDK.
## Proxy Endpoints [#proxy-endpoints]
### Stats Endpoint [#stats-endpoint]
```bash
curl http://localhost:8787/stats
```
```json
{
"persistent_savings": {
"lifetime": {
"tokens_saved": 12500,
"compression_savings_usd": 0.04
}
},
"requests": {
"total": 42,
"cached": 5,
"rate_limited": 0,
"failed": 0
},
"tokens": {
"input": 50000,
"output": 8000,
"saved": 12500,
"savings_percent": 25.0
},
"cost": {
"total_cost_usd": 0.15,
"total_savings_usd": 0.04
},
"cache": {
"entries": 10,
"total_hits": 5
}
}
```
Persistent savings are stored at `~/.headroom/proxy_savings.json` and survive proxy restarts. Override the path with `HEADROOM_SAVINGS_PATH`.
### Historical Savings [#historical-savings]
```bash
curl http://localhost:8787/stats-history
```
Returns durable compression history with hourly, daily, weekly, and monthly rollups. Supports CSV export:
```bash
curl "http://localhost:8787/stats-history?format=csv&series=daily"
curl "http://localhost:8787/stats-history?format=csv&series=monthly"
```
### Prometheus Metrics [#prometheus-metrics]
```bash
curl http://localhost:8787/metrics
```
```
# HELP headroom_requests_total Total requests processed
headroom_requests_total 1234
# HELP headroom_tokens_saved_total Total tokens saved
headroom_tokens_saved_total 5678900
# HELP headroom_persistent_savings_tokens_saved_total Durable lifetime input tokens saved by proxy compression
headroom_persistent_savings_tokens_saved_total 5678900
# HELP headroom_latency_ms_sum Sum of request latency in milliseconds
headroom_latency_ms_sum 152300
# HELP headroom_latency_ms_count Count of latency samples
headroom_latency_ms_count 1234
# HELP headroom_provider_cache_hit_requests_total Requests that read from the provider's prompt cache
headroom_provider_cache_hit_requests_total{provider="anthropic"} 456
```
There are no Prometheus histogram buckets for compression ratio or latency — build percentiles or averages from `_sum` / `_count` pairs (`headroom_latency_ms_sum`, `headroom_latency_ms_count`, and so on), or use the `headroom perf` CLI for real p95/p99.
### OpenTelemetry (OTLP) Export [#opentelemetry-otlp-export]
The proxy can also push its counters to any OTLP/HTTP endpoint. Install the extra and set four variables:
```bash
pip install "headroom-ai[proxy,otel]"
```
```bash
HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT=http://127.0.0.1:4318/v1/metrics
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy
HEADROOM_OTEL_RESOURCE_ATTRIBUTES=deployment.environment=prod
```
| Variable | Default | Purpose |
| ------------------------------------------ | ---------------- | -------------------------------------------------------------------------- |
| `HEADROOM_OTEL_METRICS_ENABLED` | `0` | Enable Headroom-managed OTLP metric export |
| `HEADROOM_OTEL_METRICS_EXPORTER` | `otlp_http` | `otlp_http` or `console` (local debugging) |
| `HEADROOM_OTEL_METRICS_ENDPOINT` | unset | Full OTLP metrics URL — Headroom does **not** append `/v1/metrics` for you |
| `HEADROOM_OTEL_METRICS_HEADERS` | unset | Comma-separated `key=value` auth headers |
| `HEADROOM_OTEL_METRICS_EXPORT_INTERVAL_MS` | `10000` | Export interval |
| `HEADROOM_OTEL_SERVICE_NAME` | `headroom-proxy` | OTEL `service.name` |
| `HEADROOM_OTEL_RESOURCE_ATTRIBUTES` | unset | Comma-separated resource attributes |
Exported counters include `headroom.proxy.requests`, `headroom.proxy.tokens.input`, and
`headroom.proxy.tokens.output`. `headroom.proxy.tokens.saved` is the all-layer total:
message/compression savings plus tool-schema deferral savings. The component counter
`headroom.proxy.tokens.tool_schema_saved` exposes the deferral portion separately;
`headroom.compression.tokens.saved` remains the compression-pipeline component.
Confirm the exporter is live with `curl -s http://localhost:8787/stats | jq .otel`.
If your application already configures a global OTEL meter provider, leave `HEADROOM_OTEL_*` unset — Headroom records into the ambient provider automatically.
### Dynatrace [#dynatrace]
Point the exporter at your environment's OTLP API and add the API token as a header. The token needs the `metrics.ingest` scope.
```bash
HEADROOM_OTEL_METRICS_ENABLED=1
HEADROOM_OTEL_METRICS_ENDPOINT="https://.live.dynatrace.com/api/v2/otlp/v1/metrics"
HEADROOM_OTEL_METRICS_HEADERS="Authorization=Api-Token dt0c01.XXXX"
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
HEADROOM_OTEL_SERVICE_NAME=headroom-proxy
```
`OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA` is required. Dynatrace [only ingests delta counters](https://docs.dynatrace.com/docs/ingest-from/opentelemetry/getting-started/metrics/limitations) and rejects cumulative ones with `UNSUPPORTED_METRIC_TYPE_MONOTONIC_CUMULATIVE_SUM`, while the OTEL SDK default is cumulative. Without this line, every Headroom metric is dropped at ingest and the proxy logs no error.
Restart the proxy, then search the Dynatrace metric explorer for `headroom.proxy.tokens.saved` — data appears within \~30s.
For an ActiveGate deployment, swap the base URL for `https://:9999/e//api/v2/otlp/v1/metrics`. If you already run an OpenTelemetry Collector, send Headroom to it instead and add the `cumulativetodelta` processor — then the temporality variable is unnecessary and the collector holds the token.
Trace export is separate: Headroom's self-configured tracing targets Langfuse only. To land its spans in Dynatrace, leave `HEADROOM_LANGFUSE_*` unset and run the proxy under `opentelemetry-instrument` with the standard `OTEL_EXPORTER_OTLP_ENDPOINT` / `OTEL_EXPORTER_OTLP_HEADERS` / `OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf` variables; Headroom records into the ambient tracer provider.
### Health Check [#health-check]
```bash
curl http://localhost:8787/health
```
```json
{
"status": "healthy",
"version": "0.37.0",
"uptime_seconds": 3600
}
```
## SDK Metrics [#sdk-metrics]
### Proxy Stats [#proxy-stats]
The TypeScript SDK queries the proxy for stats:
```ts twoslash
import { HeadroomClient } from 'headroom-ai';
const client = new HeadroomClient();
// Get proxy stats
const stats = await client.proxyStats();
console.log(`Tokens saved: ${stats.tokens.saved}`);
console.log(`Savings: ${stats.tokens.savingsPercent}%`);
```
### Compression Result Metrics [#compression-result-metrics]
Every `compress()` call returns metrics:
```ts twoslash
import { compress } from 'headroom-ai';
const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Tokens: ${result.tokensBefore} -> ${result.tokensAfter}`);
console.log(`Saved: ${result.tokensSaved} (${(result.compressionRatio * 100).toFixed(1)}%)`);
console.log(`Transforms: ${result.transformsApplied.join(', ')}`);
```
### Session Stats [#session-stats]
Quick stats for the current session (no database query):
```python
stats = client.get_stats()
print(f"Mode: {stats['config']['mode']}")
print(f"Tokens saved: {stats['session']['tokens_saved_total']}")
print(f"Requests optimized: {stats['session']['requests_optimized']}")
```
Returns:
```python
{
"session": {
"requests_total": 10,
"requests_optimized": 8,
"requests_audit": 2,
"tokens_saved_total": 15000,
"cache_hits": 3,
},
"config": {
"mode": "optimize",
"provider": "openai",
"cache_optimizer": "openai-prefix-stabilizer",
"semantic_cache": False,
},
"transforms": {
"smart_crusher_enabled": True,
"cache_aligner_enabled": True,
},
}
```
### Historical Metrics [#historical-metrics]
Query stored metrics from the database:
```python
from datetime import datetime, timedelta
metrics = client.get_metrics(
start_time=datetime.utcnow() - timedelta(hours=1),
limit=100,
)
for m in metrics:
print(f"{m.timestamp}: {m.tokens_input_before} -> {m.tokens_input_after}")
```
### Summary Statistics [#summary-statistics]
Aggregate statistics across all stored metrics:
```python
summary = client.get_summary()
print(f"Total requests: {summary['total_requests']}")
print(f"Total tokens saved: {summary['total_tokens_saved']}")
print(f"Avg tokens saved per request: {summary['avg_tokens_saved']:.0f}")
```
## Logging [#logging]
```python
import logging
# INFO level shows compression summaries
logging.basicConfig(level=logging.INFO)
# DEBUG level shows detailed transform decisions
logging.basicConfig(level=logging.DEBUG)
```
Example output:
```
INFO:headroom.transforms.pipeline:Pipeline complete: 45000 -> 4500 tokens (saved 40500, 90.0% reduction)
INFO:headroom.transforms.smart_crusher:SmartCrusher applied top_n strategy: kept 15 of 1000 items
DEBUG:headroom.transforms.smart_crusher:Kept items: [0,1,2,42,77,97,98,99] (errors at 42, warnings at 77)
```
```bash
# Log to file
headroom proxy --log-file headroom.jsonl
# Increase verbosity (no --log-level flag; use the env var)
HEADROOM_LOG_LEVEL=debug headroom proxy
```
## Cost Tracking [#cost-tracking]
### Budget Alerts [#budget-alerts]
Set a budget limit in the proxy:
```bash
headroom proxy --budget 10.00
```
When the budget is exceeded, requests return a budget exceeded error, the `/stats` endpoint shows budget status, and logs indicate the budget state.
### Measured vs Estimated Spend [#measured-vs-estimated-spend]
Every cost record carries a *basis* — where its input-token count came from. When a provider response includes a usage breakdown, the basis is `measured`. When it doesn't, Headroom substitutes its own `tokens_sent` count so input cost isn't dropped from the budget, and the record's basis is `estimated`. Headroom logs one warning per model the first time this happens.
`/stats` keeps the two separable under `cost.budget_basis`:
```json
{
"cost": {
"budget_limit_usd": 10.0,
"budget_period": "daily",
"budget_estimated_basis": "count",
"budget_basis": {
"total_usd": 3.1400,
"measured_usd": 2.9000,
"estimated_usd": 0.2400,
"estimated_pct": 7.6,
"records": 412,
"estimated_records": 31
}
}
}
```
An estimate can drift in either direction, so you choose what it does to the hard limit:
```bash
headroom proxy --budget 10.00 --budget-estimated-basis count # default
```
| Value | Effect |
| -------- | -------------------------------------------------------------------------------------------------------------- |
| `count` | Estimated spend consumes the budget like measured spend. The default; matches historical behavior. |
| `ignore` | Estimated spend is still booked and reported, but only provider-reported spend consumes the budget. |
| `block` | Refuse requests once the period holds any estimated spend, rather than enforcing a hard limit against a guess. |
Env: `HEADROOM_BUDGET_ESTIMATED_BASIS`. `headroom doctor` reports the estimated share alongside the budget check.
## Key Metrics to Monitor [#key-metrics-to-monitor]
| Metric | What It Tells You | Target |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ---------------- |
| `headroom_tokens_saved_total` | Runtime tokens saved since this proxy process started | Higher is better |
| `headroom_persistent_savings_tokens_saved_total` | Durable lifetime tokens saved from `/stats.persistent_savings` | Higher is better |
| `headroom_overhead_ms_sum` / `headroom_overhead_ms_count` | Mean latency Headroom itself adds (there are no percentile buckets — see the callout above) | Low is better |
| `headroom_provider_cache_hit_requests_total` / `headroom_provider_cache_requests_total` | Cache effectiveness, by provider | >20% is good |
| `headroom_requests_failed_total` | Reliability (upstream 5xx errors) | 0 |
## Grafana Dashboard [#grafana-dashboard]
A dashboard template ships in
[`examples/grafana/headroom-dashboard.json`](https://github.com/headroomlabs-ai/headroom/blob/main/examples/grafana/headroom-dashboard.json).
Its metric names match `/metrics`, but every panel query filters on a `pool` label
(e.g. `headroom_tokens_saved_total{pool=~"..."}`) that no metric in
`headroom/proxy/prometheus_metrics.py` actually emits, so the panel dropdowns come up
empty on import. Edit the queries to drop the `pool=~"..."` filter, or use the ad-hoc
queries below instead.
Example ad-hoc queries against the same metric family:
| Panel | PromQL |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| Runtime Tokens Saved | `headroom_tokens_saved_total` |
| Lifetime Tokens Saved | `headroom_persistent_savings_tokens_saved_total` |
| Mean Headroom Overhead | `rate(headroom_overhead_ms_sum[5m]) / rate(headroom_overhead_ms_count[5m])` |
| Cache Hit Rate (by provider) | `sum by (provider) (rate(headroom_provider_cache_hit_requests_total[5m])) / sum by (provider) (rate(headroom_provider_cache_requests_total[5m]))` |
Headroom wraps the OpenAI Node.js SDK to automatically compress messages before every `chat.completions.create()` call. All other methods (embeddings, images, audio) pass through unchanged.
## Installation [#installation]
```bash
npm install headroom-ai openai
```
The TypeScript SDK sends messages to a local Headroom proxy for compression. Start the proxy before using the SDK:
```bash
pip install "headroom-ai[proxy]"
headroom proxy
```
## Quick start [#quick-start]
```ts twoslash
import { withHeadroom } from 'headroom-ai/openai';
import OpenAI from 'openai';
const client = withHeadroom(new OpenAI());
// Messages are compressed automatically before sending
const response = await client.chat.completions.create({
model: 'gpt-4o',
messages: longConversation,
});
```
That's it. Every call to `client.chat.completions.create()` compresses the messages first. The response format is identical to the unwrapped client.
## How it works [#how-it-works]
`withHeadroom()` returns a proxy around your OpenAI client that intercepts `chat.completions.create()`:
1. Extracts `messages` from the request params
2. Sends them to the Headroom proxy's [`POST /v1/compress`](/docs/proxy#post-v1compress) endpoint
3. Replaces the original messages with the compressed result
4. Forwards the request to OpenAI as normal
The SDK talks to a **local** proxy, which is why no extra configuration is needed: `/v1/compress` is loopback-only by default. If you move the proxy to another host, set `HEADROOM_COMPRESS_ALLOW_REMOTE=1` on it or requests come back `404`.
All other client methods are untouched:
```ts twoslash
import { withHeadroom } from 'headroom-ai/openai';
import OpenAI from 'openai';
const client = withHeadroom(new OpenAI());
// These pass through unchanged
const embedding = await client.embeddings.create({
model: 'text-embedding-3-small',
input: 'Hello world',
});
```
## Options [#options]
Pass compression options as the second argument:
```ts twoslash
import { withHeadroom } from 'headroom-ai/openai';
import OpenAI from 'openai';
const client = withHeadroom(new OpenAI(), {
model: 'gpt-4o',
baseUrl: 'http://localhost:8787',
});
```
## Streaming [#streaming]
Streaming works normally. Compression happens before the request is sent:
```ts twoslash
import { withHeadroom } from 'headroom-ai/openai';
import OpenAI from 'openai';
const client = withHeadroom(new OpenAI());
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: longConversation,
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}
```
## Tool calling [#tool-calling]
Tool call messages and tool results are compressed like any other message content. Large tool outputs (JSON arrays, logs) see the biggest savings:
```ts twoslash
import { withHeadroom } from 'headroom-ai/openai';
import OpenAI from 'openai';
const client = withHeadroom(new OpenAI());
const response = await client.chat.completions.create({
model: 'gpt-4o',
messages: [
{ role: 'user', content: 'Search for recent errors' },
{
role: 'assistant',
content: null,
tool_calls: [{ id: 'call_1', type: 'function', function: { name: 'search', arguments: '{"q":"errors"}' } }],
},
{
role: 'tool',
tool_call_id: 'call_1',
content: hugeJsonResult, // Compressed automatically
},
],
tools: [{ type: 'function', function: { name: 'search', parameters: {} } }],
});
```
Cut DeepSeek API costs with Headroom's context compression proxy: tool outputs,
logs, and search results are compressed before they reach the model, and
responses can be shaped to be more concise.
## How it works [#how-it-works]
```
OpenCode → Headroom Proxy (:8787) → DeepSeek API
↑ compresses input
+ shapes output
```
The proxy sits between OpenCode and DeepSeek. It compresses tool outputs, logs,
and search results before they reach the model, then shapes responses to be
concise. DeepSeek's API is OpenAI-compatible — one flag and you're running.
***
## 1. Install Headroom [#1-install-headroom]
```bash
pip install "headroom-ai[proxy]"
# or via uv:
uv tool install "headroom-ai[proxy]"
```
You get SmartCrusher (structural compression), the proxy, output shaping, and
the MCP server — everything you need.
***
## 2. Get your DeepSeek API key [#2-get-your-deepseek-api-key]
Sign up at [platform.deepseek.com](https://platform.deepseek.com) and generate
an API key.
Store it somewhere safe:
```bash
export DEEPSEEK_API_KEY="sk-your-deepseek-key-here"
```
***
## 3. Start the proxy [#3-start-the-proxy]
```bash
headroom proxy \
--port 8787 \
--openai-api-url https://api.deepseek.com/v1
```
The proxy auto-detects `api.deepseek.com` and labels itself "DeepSeek" on the
dashboard. Verify it's running:
```bash
curl http://127.0.0.1:8787/health
# → "status": "healthy"
```
To see which models the proxy exposes:
```bash
curl -s http://127.0.0.1:8787/v1/models \
-H "Authorization: Bearer sk-your-key" | jq '.data[].id'
```
### With output shaping (optional) [#with-output-shaping-optional]
Output shaping makes the model's responses shorter — fewer tokens, lower cost:
```bash
HEADROOM_OUTPUT_SHAPER=1 HEADROOM_VERBOSITY_LEVEL=2 \
headroom proxy --port 8787 --openai-api-url https://api.deepseek.com/v1
```
Verbosity levels:
| Level | Behavior |
| ----- | ---------------------------------------------------------------------- |
| `1` | Skip preambles/postambles |
| `2` | + Don't restate code/file content already in context (**recommended**) |
| `3` | + Omit rationale unless asked |
| `4` | Maximum — fragments, zero fluff |
***
## 4. Configure OpenCode [#4-configure-opencode]
**Note:** If you have an existing `~/.config/opencode/opencode.json` (for MCP
servers, etc.), merge the provider section into that file. Having both `.json`
and `.jsonc` in the same directory can cause conflicts.
Edit `~/.config/opencode/opencode.json`:
```jsonc
{
"$schema": "https://opencode.ai/config.json",
"model": "headroom/deepseek-v4-pro",
"provider": {
"headroom": {
"npm": "@ai-sdk/openai-compatible",
"name": "Headroom Proxy",
"options": {
"baseURL": "http://127.0.0.1:8787/v1",
"apiKey": "sk-your-deepseek-key"
},
"models": {
"deepseek-v4-pro": {
"name": "DeepSeek V4 Pro",
"limit": { "context": 1000000, "output": 384000 }
},
"deepseek-v4-flash": {
"name": "DeepSeek V4 Flash",
"limit": { "context": 1000000, "output": 384000 }
}
}
}
},
"mcp": {
"headroom": {
"type": "local",
"command": ["headroom", "mcp", "serve"],
"enabled": true
}
}
}
```
**Important:** Only include model IDs that appear in the proxy's `/v1/models`
response. OpenCode validates config models against the proxy's model list.
The current DeepSeek model names are `deepseek-v4-pro` and `deepseek-v4-flash`.
`deepseek-chat` and `deepseek-reasoner` are deprecated compatibility aliases.
### Model comparison [#model-comparison]
| Model | Input / Output (per 1M) | Context | Max Output |
| ------------------- | ----------------------- | ------- | ---------- |
| `deepseek-v4-pro` | $0.435 / $0.87 | 1M | 384K |
| `deepseek-v4-flash` | $0.14 / $0.28 | 1M | 384K |
Both models support **thinking mode** for step-by-step reasoning (see below).
Switch models at any time with `/model` in OpenCode.
***
## 5. Start OpenCode [#5-start-opencode]
```bash
opencode
```
Run `/models` to confirm both DeepSeek models appear under "Headroom Proxy".
Select one with `/model deepseek-v4-flash` or `/model deepseek-v4-pro`.
***
## 6. Check savings [#6-check-savings]
```bash
curl http://127.0.0.1:8787/stats | python3 -m json.tool | grep -A5 compression
```
Or open the dashboard at [http://127.0.0.1:8787/dashboard](http://127.0.0.1:8787/dashboard).
***
## Thinking mode (reasoning) [#thinking-mode-reasoning]
Both models support thinking mode natively, and DeepSeek enables it by default.
This replaces the deprecated `deepseek-reasoner` (R1) model.
See [DeepSeek's thinking mode docs](https://api-docs.deepseek.com/guides/thinking_mode)
for details on switching between thinking and non-thinking modes.
***
## Common issues [#common-issues]
### "Authentication Fails" / Unauthorized [#authentication-fails--unauthorized]
The `apiKey` in OpenCode's config is missing or wrong. OpenCode must send the
API key to the proxy, and the proxy forwards it to DeepSeek. Make sure
`"apiKey": "sk-..."` is set under `options`.
### Models don't appear under "Headroom Proxy" [#models-dont-appear-under-headroom-proxy]
1. Verify the proxy is running: `curl http://127.0.0.1:8787/health`
2. Check which models the proxy exposes: `curl -s http://127.0.0.1:8787/v1/models -H "Authorization: Bearer sk-your-key"`
3. Make sure your config model IDs match **exactly** what the proxy returns
4. Don't use both `opencode.json` and `opencode.jsonc` in the same config directory — use one file
### Models appear but requests fail [#models-appear-but-requests-fail]
You ran `headroom wrap opencode`. That command replaces your config with Claude
and GPT models. **Do not use `headroom wrap`.** Configure OpenCode manually as
shown above, and launch OpenCode directly with `opencode`.
### "headroom" command not found [#headroom-command-not-found]
`uv tool install` puts binaries in `~/.local/bin/`. Add it to your PATH:
```bash
export PATH="$HOME/.local/bin:$PATH"
```
### Output shaping shows no savings [#output-shaping-shows-no-savings]
Output savings are measured against a learned baseline (it compares "what the
model actually emitted" vs "what it would have emitted unshaped"). After a few
sessions, run:
```bash
headroom learn --verbosity --apply
```
This builds the baseline, and `/stats` will show output savings numbers. The
shaper is active immediately — the numbers just need calibration.
***
## What's NOT in this guide [#whats-not-in-this-guide]
* **Claude or GPT models** — this setup uses DeepSeek exclusively
* **`headroom wrap`** — do not use it; it overrides the config
* **Deprecated model names** — `deepseek-chat` and `deepseek-reasoner` are
compatibility aliases slated for removal; use `deepseek-v4-pro` and
`deepseek-v4-flash` instead
* **Kompress (ML compression)** — requires extra dependencies; SmartCrusher
handles the majority of use cases
* **Any code changes** — headroom ships full DeepSeek support natively
(model tables, pricing, tokenizers, domain detection)
Use `headroom wrap opencode` to route OpenCode LLM traffic through the Headroom proxy with a single command. The wrapper starts or reuses the proxy, writes OpenCode config, injects Headroom MCP tools, and launches OpenCode with the generated config.
The `headroom-opencode` npm package also exports a native OpenCode plugin. The plugin can be used directly from OpenCode config when you want in-process transport interception plus the Headroom retrieve tool.
## Quick Start [#quick-start]
```bash
headroom wrap opencode
```
When you are done:
```bash
headroom unwrap opencode
```
## What `wrap opencode` Does [#what-wrap-opencode-does]
| Step | What happens |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Proxy | Starts the Headroom proxy unless `--no-proxy` is set |
| Provider injection | Writes a `headroom` provider using `@ai-sdk/openai-compatible` into `opencode.json`, pointing at `http://127.0.0.1:/v1` |
| Runtime env | Sets `OPENCODE_CONFIG_CONTENT` with provider, plugin, and optional local MCP config so OpenCode picks up Headroom at launch |
| Provider compatibility | Leaves `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` untouched so OpenCode `/connect` providers keep their own routing |
| MCP setup | Registers the Headroom MCP server (`headroom_compress`, `headroom_retrieve`, `headroom_stats`) |
| Serena MCP | Optionally registers Serena code graph tools (`--no-serena` to skip) |
| Backup | Snapshots `opencode.json` to `opencode.json.headroom-backup` before making any changes |
| Launch | Starts the `opencode` binary through the proxy |
## Options [#options]
```bash
headroom wrap opencode \
--port 8787 \
--no-mcp \
--no-serena \
--code-graph \
--no-proxy \
--learn \
--memory \
--backend anthropic \
--anyllm-provider ... \
--region ... \
--
```
## Provider Model Mapping [#provider-model-mapping]
The generated `headroom` provider exposes these models through the proxy:
| Provider model | Upstream model |
| ------------------------------------ | ------------------------------------------- |
| `headroom/claude-sonnet-4-6` | Claude Sonnet 4.6, 200K context, 16K output |
| `headroom/claude-opus-4-6` | Claude Opus 4.6, 200K context, 16K output |
| `headroom/claude-haiku-4-5-20251001` | Claude Haiku 4.5, 200K context, 8K output |
| `headroom/gpt-4o` | GPT-4o, 128K context, 16K output |
| `headroom/gpt-4.1` | GPT-4.1, 1M context, 32K output |
The default model is `headroom/claude-sonnet-4-6`. Change it in `opencode.json` or in the generated `OPENCODE_CONFIG_CONTENT` payload.
## Environment Variables [#environment-variables]
| Variable | Description |
| ------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `OPENCODE_CONFIG_CONTENT` | JSON payload with provider, plugin, and optional local MCP config injected by `wrap` |
| `HEADROOM_PROXY_URL` | Proxy URL passed to Headroom MCP when a non-default port is used, and to the native plugin when configured |
## Failure Learning [#failure-learning]
`headroom learn` supports OpenCode as a scan target. It reads past sessions from the newer of `~/.local/share/opencode/opencode-local.db` and `~/.local/share/opencode/opencode.db`, or from `HEADROOM_OPENCODE_DB` when you set an explicit override, and writes corrections to your project's `AGENTS.md`.
```bash
headroom learn --agent opencode --apply
```
See [Failure Learning](/docs/failure-learning) for details on the learn system.
## Persistent Installs [#persistent-installs]
`headroom install` supports OpenCode as a target for persistent provider wiring. Use provider scope when you want Headroom to edit `opencode.json` directly:
```bash
headroom install apply --preset persistent-service --scope provider --providers manual --target opencode
```
This writes the Headroom provider into `~/.config/opencode/opencode.json` and keeps the proxy running on port 8787.
The default user scope only writes shell environment configuration. For OpenCode, direct provider config requires `--scope provider`.
## Native OpenCode Plugin [#native-opencode-plugin]
The `headroom-opencode` package exports `HeadroomPlugin` for direct OpenCode plugin registration. The plugin installs Headroom transport interception inside OpenCode, exposes the `headroom_retrieve` tool, and publishes Headroom metadata through the OpenCode plugin output env.
Example:
```ts
import { HeadroomPlugin } from "headroom-opencode";
export default async function plugin(input) {
return HeadroomPlugin(input, {
proxyUrl: process.env.HEADROOM_PROXY_URL ?? "http://127.0.0.1:8787",
});
}
```
Use this plugin when OpenCode should intercept provider traffic in-process. Use `headroom wrap opencode` when you want the CLI to manage the proxy, config injection, MCP registration, backups, and unwrap behavior.
## Programmatic Config Helpers [#programmatic-config-helpers]
The package also exports helpers for custom integrations:
```ts
import {
buildOpencodeConfigContent,
createHeadroomProvider,
createHeadroomRetrieveTool,
} from "headroom-opencode";
const provider = createHeadroomProvider({ proxyPort: 8787 });
const config = buildOpencodeConfigContent({
proxyPort: 8787,
defaultModel: "claude-sonnet-4-6",
});
const retrieve = createHeadroomRetrieveTool({
proxyBaseUrl: "http://127.0.0.1:8787",
});
```
## How It Works Under The Hood [#how-it-works-under-the-hood]
1. **Config injection**. The wrapper writes a `provider.headroom` block into `opencode.json`. The provider uses `@ai-sdk/openai-compatible`, which OpenCode supports natively. Model mappings route requests through `http://127.0.0.1:/v1`.
2. **Runtime config**. `OPENCODE_CONFIG_CONTENT` is set as an env var containing provider, plugin, and optional local MCP JSON. OpenCode reads it at startup and merges it with on-disk config.
3. **MCP tools**. Headroom registers `headroom_compress`, `headroom_retrieve`, and `headroom_stats` through `headroom mcp serve` unless `--no-mcp` is set.
4. **Native plugin path**. `HeadroomPlugin` installs Headroom transport interception and uses `HEADROOM_PROXY_URL` or `http://127.0.0.1:8787` to reach the proxy.
5. **Unwrap**. `headroom unwrap opencode` restores `opencode.json` from the pre-wrap backup when present, strips Headroom marker blocks when no backup exists, and unregisters Headroom MCP servers.
## Troubleshooting [#troubleshooting]
**OpenCode does not use the headroom provider.**
Check that `OPENCODE_CONFIG_CONTENT` is set and contains the `provider.headroom` block. The wrap command prints the env vars it sets.
**The native plugin cannot reach Headroom.**
Set `HEADROOM_PROXY_URL` to the running proxy URL, for example `http://127.0.0.1:8787`.
**Provider not found after unwrap.**
If unwrap left the provider configured, run `headroom unwrap opencode` again, or manually restore from `~/.config/opencode/opencode.json.headroom-backup`.
**Proxy port conflict.**
Use `--port` to select a specific port, or let the proxy auto-select an available one.
Headroom can be installed as a durable local runtime instead of only being started ad hoc with `headroom proxy` or `headroom wrap ...`.
Use `headroom deploy` when you want the one-line path: Headroom chooses the best local runtime it can run on the current host, configures detected tools, starts the proxy at `http://127.0.0.1:8787`, and stores the deployment profile for later lifecycle commands.
Use the lower-level Python-native `headroom install` CLI when you need to force a specific service, task, Docker, scope, or provider target. Both surfaces create profiles that `wrap` can reuse or recover instead of starting a second ephemeral proxy.
## Runtime matrix [#runtime-matrix]
| Mode | What stays running | Primary entrypoint |
| ---------------------- | ---------------------------------------------------------------------- | ---------------------------------------------------- |
| Turnkey Deploy | Docker, scheduled watchdog, or detached runtime selected for this host | `headroom deploy` |
| Persistent Service | Native background service | `headroom install apply --preset persistent-service` |
| Persistent Task | Scheduled watchdog + on-demand runner | `headroom install apply --preset persistent-task` |
| Persistent Docker | Restartable Docker container | `headroom install apply --preset persistent-docker` |
| On-Demand CLI (Python) | Nothing after command exits | `headroom proxy` |
| On-Demand CLI (Docker) | Nothing after container exits | Docker-native wrapper / compose CLI |
| Wrapped (Python) | Proxy lasts for wrapped session | `headroom wrap ...` |
| Wrapped (Docker) | Containerized proxy + host tool session | Docker-native wrapper |
## Quick examples [#quick-examples]
### Turnkey local deployment [#turnkey-local-deployment]
```bash
headroom deploy
headroom install status
```
`deploy` prefers the most capable restartable path it can prove is available. On NVIDIA workstations with Docker GPU support, such as an RTX 4090 host with the NVIDIA container runtime installed, it uses Docker GPU passthrough. Otherwise it prefers plain Docker, then the native scheduled watchdog for the host when available (`launchd`, Task Scheduler, or cron). If no supported supervisor is present, it still starts a managed detached Python runtime instead of failing on missing platform services.
### Persistent service on the local machine [#persistent-service-on-the-local-machine]
```bash
headroom install apply --preset persistent-service --providers auto
headroom install status
```
This installs a background service on the current machine, applies persistent tool wiring, and keeps the proxy healthy on port `8787`.
### Persistent watchdog task [#persistent-watchdog-task]
```bash
headroom install apply --preset persistent-task --providers manual --target claude --target codex
```
This installs a scheduled recovery path instead of a traditional always-running service.
### Persistent Docker [#persistent-docker]
```bash
headroom install apply --preset persistent-docker --scope user --providers auto
```
This uses Docker's restart policy instead of an OS supervisor.
If you are using the Docker-native host wrapper instead of a Python install, you can use `headroom install apply|status|start|stop|restart|remove` for the `persistent-docker` preset directly from the installed wrapper. Service/task installs and provider/user/system mutation flows still belong to the Python-native CLI.
## Command surface [#command-surface]
```text
headroom install apply
headroom install status
headroom install start
headroom install stop
headroom install restart
headroom install remove
```
`apply` creates or updates a named deployment profile, stores its manifest under `~/.headroom/deploy//manifest.json`, applies reversible configuration changes, and starts the selected runtime.
## Presets and runtime kinds [#presets-and-runtime-kinds]
### Presets [#presets]
* `persistent-service` → native service supervisor
* `persistent-task` → scheduled watchdog / recovery supervisor
* `persistent-docker` → Docker restart policy with no extra OS supervisor
### Runtime kinds [#runtime-kinds]
* `--runtime python` runs `headroom proxy` directly
* `--runtime docker` runs Headroom inside Docker while keeping the deployment managed locally
For `persistent-docker`, the runtime is always Docker.
## Configuration scopes [#configuration-scopes]
| Scope | What changes |
| ---------- | ------------------------------------------------------------------------------- |
| `provider` | Tool-specific config surfaces where Headroom can make a precise reversible edit |
| `user` | User-level shell or environment surfaces |
| `system` | Machine-wide shell or environment surfaces |
### Provider scope today [#provider-scope-today]
Provider scope is intentionally conservative. The current direct adapters are:
* Claude Code → `~/.claude/settings.json` `env`
* Codex → managed block in `~/.codex/config.toml`
* OpenClaw → existing `wrap openclaw` / `unwrap openclaw` flow
* OpenCode → managed block in `~/.config/opencode/opencode.json`
For Copilot, Aider, Cursor, and broader env-driven setups, prefer `--scope user` or `--scope system`.
## Provider selection [#provider-selection]
| Option | Meaning |
| --------------------------------- | ---------------------------------------------------------------------------- |
| `--providers auto` | Detect supported tools on the host and configure the best available defaults |
| `--providers all` | Configure all known targets |
| `--providers manual --target ...` | Configure only the named tools |
Examples:
```bash
headroom install apply --providers auto
headroom install apply --providers all --scope user
headroom install apply --providers manual --target claude --target copilot
```
## Health and wrap behavior [#health-and-wrap-behavior]
Persistent deployments publish the same `readyz` and `health` endpoints as ad hoc proxy runs.
`/health` also exposes deployment metadata when the proxy was launched through the install subsystem:
```json
{
"deployment": {
"profile": "default",
"preset": "persistent-service",
"runtime": "python",
"supervisor": "service",
"scope": "user"
}
}
```
The Python-native `headroom wrap ...` flow checks for a matching persistent deployment on the requested port before it starts a new ephemeral proxy. If an installed deployment exists but is stopped or unhealthy, it attempts to recover it first.
The Docker-native host wrapper does **not** yet reuse or recover persistent profiles automatically; it still starts a fresh proxy container unless you opt into `--no-proxy`.
## Claude Code VSCode extension caveat [#claude-code-vscode-extension-caveat]
Persistent Claude deployments default to `ENABLE_TOOL_SEARCH=true` because the
standalone Claude CLI benefits from deferred tool schemas.
Anthropic's VSCode extension currently does not render those deferred-tool content
blocks correctly through Headroom and can show `unsupported content type` in the
webview. If your persistent install targets Claude Code inside VSCode, edit
`~/.headroom/deploy//manifest.json`, set
`tool_envs.claude.ENABLE_TOOL_SEARCH` to `"false"`, then restart the deployment.
Keep `ENABLE_TOOL_SEARCH=true` for the standalone `claude` CLI unless you hit the
same renderer limitation there.
## Docker-native relationship [#docker-native-relationship]
The Docker-native host wrapper and the Python install CLI solve different layers of the runtime story:
* [Docker-Native Install](/docs/docker-install) → containerized on-demand CLI, wrapped host-tool flows, and Docker-native `persistent-docker` lifecycle commands
* `headroom install ...` → full persistent service, task, and Docker lifecycle management, including provider/user/system mutation
For a no-Python persistent Docker workflow, use the compose-managed proxy path from `docker/docker-compose.native.yml`:
```bash
export HEADROOM_HOST_HOME="$HOME"
export HEADROOM_WORKSPACE="$PWD"
docker compose -f docker/docker-compose.native.yml up -d proxy
```
That keeps `localhost:8787` stable and restarts the proxy automatically.
`HEADROOM_WORKSPACE` (the host-side bind-mount source used by the compose file) is **not** the same variable as `HEADROOM_WORKSPACE_DIR` (the canonical Headroom state root inside the container). Both are retained; the compose file sets the latter automatically. See [Filesystem Contract](/docs/filesystem-contract) for the full bucket model.
## Related guides [#related-guides]
Headroom emits lifecycle events at every stage of the canonical request pipeline. Third-party packages can hook these events — without forking Headroom — by registering a **pipeline extension** under the `headroom.pipeline_extension` entry-point group. Extensions can mutate `messages`, `tools`, `headers`, or `metadata` in place before the request is forwarded upstream.
Both the SDK client and the proxy dispatch the same events, so one extension covers both deployments.
## Lifecycle stages [#lifecycle-stages]
Extensions receive a `PipelineEvent` for each stage in `headroom.pipeline.PipelineStage`:
| Stage | When |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `SETUP`, `PRE_START`, `POST_START` | Process/pipeline startup |
| `INPUT_RECEIVED` | Raw request accepted |
| `INPUT_CACHED`, `INPUT_ROUTED`, `INPUT_COMPRESSED`, `INPUT_REMEMBERED` | Cache, routing, compression, memory stages |
| `PRE_SEND` | Last hook before the request is forwarded upstream |
| `POST_SEND`, `RESPONSE_RECEIVED` | After forwarding / on response |
| `OUTCOME_OBSERVED` | Emits a read-only `OutcomeSnapshot` (tokens, stop reason, transforms applied) once the response is fully processed |
`PRE_SEND` is the right stage for normalizing requests to fit a quirky upstream: compression and caching are done, and whatever you write into `event.messages` is exactly what the provider receives.
## Recipe: normalize requests for a quirky upstream provider [#recipe-normalize-requests-for-a-quirky-upstream-provider]
Some OpenAI-compatible gateways reject valid OpenAI-spec payloads. A real example: an upstream returns `400 "Message content is null"` for assistant messages that carry `content: null` alongside `tool_calls` — a combination the OpenAI spec explicitly produces when the model returns only tool calls. The provider-recommended workaround is to send `content: ""` instead.
An extension that rewrites those messages at `PRE_SEND`:
```python
# my_headroom_ext/normalize.py
from headroom.pipeline import PipelineEvent, PipelineStage
class NullContentNormalizer:
"""Rewrite `content: null` + tool_calls to `content: ""` before send."""
def on_pipeline_event(self, event: PipelineEvent) -> PipelineEvent | None:
if event.stage is not PipelineStage.PRE_SEND or not event.messages:
return None
for message in event.messages:
if (
message.get("role") == "assistant"
and message.get("content") is None
and message.get("tool_calls")
):
message["content"] = ""
return None # mutated in place; returning None keeps the event
```
Register it as an entry point in your extension package:
```toml
# pyproject.toml of your extension package
[project.entry-points."headroom.pipeline_extension"]
null-content-normalizer = "my_headroom_ext.normalize:NullContentNormalizer"
```
Install the package into the same environment as Headroom (`pip install my-headroom-ext`) and it is discovered automatically — entry points are loaded on startup, and a failing extension is isolated and logged rather than breaking the pipeline.
Notes on the contract:
* An extension is either an object with an `on_pipeline_event(event)` method or a class Headroom instantiates with no arguments.
* Return `None` (mutate in place) or return a replacement `PipelineEvent`.
* Exceptions raised by an extension are caught and logged (`fail-open`); the request proceeds unmodified.
* Discovery can be disabled with the SDK config flag `discover_pipeline_extensions=False`, and explicit instances can be passed via `pipeline_extensions=[...]` (SDK `HeadroomConfig` and proxy `ProxyConfig` both expose these fields).
## Per-request upstream routing with `x-headroom-base-url` [#per-request-upstream-routing-with-x-headroom-base-url]
To route different models through one Headroom instance to different OpenAI-compatible upstream bases — instead of one global `OPENAI_API_URL` / `OPENAI_TARGET_API_URL` per proxy process — send the `x-headroom-base-url` request header. The dedicated OpenAI handlers (`/v1/chat/completions`, `/v1/responses`) and the generic passthrough route all honor it, falling back to the configured upstream when absent:
```bash
curl http://localhost:8787/v1/chat/completions \
-H "content-type: application/json" \
-H "x-headroom-base-url: https://api.example-gateway.ai/gemini-3-flash" \
-d '{"model": "gemini-3-flash", "messages": [{"role": "user", "content": "hi"}]}'
```
Internal `x-headroom-*` headers (including this one) are stripped before the request is forwarded upstream by default — see `HEADROOM_STRIP_INTERNAL_HEADERS` in [Configuration](/docs/configuration).
Because this header is client-driven, operator-configured secret headers (`OPENAI_TARGET_API_HEADERS` / `ANTHROPIC_TARGET_API_HEADERS`) are only attached when the resolved upstream host is one you designated — a configured provider target, or a host in `HEADROOM_UPSTREAM_ALLOWED_HOSTS`. Other upstreams are still routed to, just without those headers. See [Configuration](/docs/configuration) for details.
## Per-request model routing with `request.state.headroom_route` [#per-request-model-routing-with-requeststateheadroom_route]
`x-headroom-base-url` is client-driven and points at one OpenAI-compatible base. When the choice of model belongs to an extension instead of the caller — a router that picks a cheaper model per turn, say — publish it on the request state and Headroom serves that one request from a backend that speaks the target provider:
```python
# middleware or an extension holding the request
request.state.headroom_route = SimpleNamespace(
model="moonshot/kimi-k2", # required
provider="moonshot", # optional; inferred from the model id if absent
reason="cheaper at this prefix length",
)
```
The contract, in `headroom/proxy/route_advice.py`:
* **Absent means unchanged.** No advice — or advice that is malformed, names an unknown provider, or fails to build a backend — and the request takes exactly the path it took before. A routing preference can never take traffic down.
* **Duck-typed**, so an extension does not import Headroom to publish one.
* A **native** provider (`anthropic`) needs no backend switch — rewrite `body["model"]` yourself. A foreign one is translated by a `LiteLLMBackend` built for it, and Headroom writes the model id.
* Backends are **built once per provider** and cached; a provider that fails to build is not retried per request.
* Honored on `/v1/messages` and `/v1/chat/completions`, streaming and non-streaming alike. (Not the Responses API, which does not use the backend abstraction.)
`routemegood` is the reference consumer of this seam: it decides, Headroom routes.
The Headroom proxy is a standalone HTTP server that compresses all LLM traffic passing through it. Point any client at the proxy and get automatic context optimization.
Running a local OpenAI-compatible model? See [Local LLM prefill benchmarking](/docs/local-llm-prefill) for a baseline-vs-optimized workflow that measures prompt-processing savings with the dashboard.
## Starting the proxy [#starting-the-proxy]
```bash
# Basic usage
headroom proxy
# Custom host and port
headroom proxy --host 0.0.0.0 --port 8080
# With logging and budget
headroom proxy \
--log-file /var/log/headroom.jsonl \
--budget 100.0
```
Two independent telemetry switches exist. `HEADROOM_TELEMETRY=on` (or `--telemetry`) turns on **local-only** in-process usage stats that power your own `/stats`, `/metrics`, and dashboard — off by default, and nothing under this switch leaves the machine.
Separately, an **anonymous usage beacon is on by default** (opt-out). Once a session goes idle it POSTs a small summary of how compression behaved to a Headroom Labs collector.
**Why it exists.** Compression quality is the product. The beacon tells us when a release changes it — a ratio that regresses, a content type that starts getting skipped, a provider path that begins failing — across real workloads rather than only our own test corpus. That is the whole purpose; there is no other use.
**What it sends.** Counters and identifiers, never free text. The collector applies an allowlist server-side and drops everything else before storage ([`deploy/beacon/worker.js`](https://github.com/headroomlabs-ai/headroom/blob/main/deploy/beacon/worker.js)): token totals, compression ratios and rates, skip reasons, source and provider names, model ids, failure counts and status codes, and a routing summary. Alongside those, seven resource attributes: service name and version, install id, install mode, stack, OS type and CPU architecture. A typical event is about 2KB.
**What it never sends.** Your prompts, your completions, your code, file paths, file contents, environment variables, or anything else derived from request or response bodies. None of these are on the allowlist, so even if a future client sent one the collector would discard it.
The install id is a random UUID generated on first run and stored in your config directory. It is deliberately not derived from hostname, MAC address, or any hardware property — delete the file and you get a new one.
Disable it with `HEADROOM_BEACON=off`, the `DO_NOT_TRACK=1` convention, or `--offline` (which also disables update checks and license reporting).
## CLI options [#cli-options]
### Core [#core]
| Option | Default | Description |
| --------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--host` | `127.0.0.1` | Host to bind to |
| `--port` | `8787` | Port to bind to |
| `--workers` | `1` | Number of Uvicorn worker processes |
| `--limit-concurrency` | `1000` | Maximum concurrent connections before Uvicorn returns 503 |
| `--max-connections` | `500` | Maximum upstream HTTP connections |
| `--max-keepalive` | `100` | Maximum upstream keep-alive connections |
| `--http-proxy` | None | HTTP proxy URL for upstream provider requests only; HTTPS provider APIs use CONNECT |
| `--mode` | `cache` | Optimization mode: `token` prioritizes compression, `cache` preserves provider prefix-cache stability. Default is `cache` (see [Savings profiles](#savings-profiles)) |
| `--no-optimize` | `false` | Disable optimization (passthrough mode) |
| `--no-cache` | `false` | Disable semantic caching |
| `--no-rate-limit` | `false` | Disable rate limiting |
| `--log-file` | None | Path to JSONL log file |
| `--log-messages` | `false` | Store full request/response content for the live feed |
| `--budget` | None | Daily budget limit in USD |
| `--openai-api-url` | `https://api.openai.com` | Custom OpenAI API URL |
| `--provider-name` | Detected from `--openai-api-url` | Display name for the OpenAI-compatible upstream on the dashboard (e.g. `OpenRouter`). Well-known hosts (OpenRouter, Groq, Together, Azure OpenAI, …) are detected automatically; this overrides them. Routing and pricing are unaffected. |
| `--anthropic-api-url` | Anthropic default | Custom Anthropic API URL |
| `--gemini-api-url` | Gemini default | Custom Gemini API URL |
| `--backend` | `anthropic` | Backend: `anthropic`, `bedrock`, `openrouter`, `anyllm`, or `litellm-` |
| `--bedrock-api-url` | None | Bedrock InvokeModel upstream for the `/model/{id}/invoke` passthrough routes (see [Bedrock via a local gateway](#bedrock-via-a-local-gateway)) |
| `--telemetry` | `false` | Enable local, in-process usage stats (for your own `/stats` and dashboard; nothing leaves the machine) |
| `--no-telemetry` | `false` | Force local telemetry off (already the default) |
| `--stateless` | `false` | Disable filesystem writes and keep runtime state in memory |
Use `--http-proxy` or `HEADROOM_HTTP_PROXY` when only provider API traffic should go through a proxy:
```bash
headroom proxy --http-proxy http://proxy.internal:8080
```
Avoid setting process-wide variables such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY` for this use case. HTTPX reads those variables too, but Headroom also inherits them into tool executions, so they can proxy unrelated tool traffic.
### Context management [#context-management]
| Option | Default | Description |
| ---------------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `--mode token` | | Prioritize token compression; prior turns may be rewritten for maximum savings. |
| `--mode cache` | default | Freeze prior turns to maximize provider prefix-cache hit rate. This is the effective default (see [Savings profiles](#savings-profiles)). |
| `--intercept-tool-results` | `false` | Opt into canary tool-result interceptors such as ast-grep Read outlining. Requires `HEADROOM_ROLLOUT_CHANNEL=canary` (or `dev`). |
| `--no-read-lifecycle` | `false` | Disable stale/superseded Read-output compression. |
| `--code-aware` / `--no-code-aware` | disabled | Enable or disable AST-based code compression. Requires `headroom-ai[code]`. |
| `--code-graph` | `false` | Enable the proxy's live code-graph file watcher for the current project. |
#### Code-memory MCP (Serena) [#code-memory-mcp-serena]
`headroom wrap` registers **[Serena](https://github.com/oraios/serena)** as the code-memory MCP for semantic, symbol-level code navigation. Serena runs on demand via `uvx` — Headroom downloads and executes no binary of its own — and indexes the current project locally. Pass `--code-memory none` to register no code-memory MCP.
> **Upgrading from tokensave?** Earlier releases registered a `tokensave` MCP server (a downloaded Rust binary). tokensave has been retired in favour of Serena. On your next `headroom wrap` / `headroom unwrap`, Headroom removes the `tokensave` MCP entry it installed and switches you to Serena — nothing to migrate, since both are just indexes rebuilt from your source. The leftover `tokensave` binary in `~/.local/bin` and any `.tokensave/` folders are unused and safe to delete.
By default, the proxy uses the shared **ContentRouter** pipeline. It routes text, logs, JSON, code, images, and tool outputs through the currently enabled compressors and preserves reversible CCR markers where applicable.
```bash
# Maximize compression
headroom proxy --mode token
# Preserve provider prefix cache stability
headroom proxy --mode cache
```
### Savings profiles [#savings-profiles]
`HEADROOM_SAVINGS_PROFILE` selects a named profile that seeds Headroom's whole compression posture — proxy mode, keep-ratio, which messages are compressed, and `force_kompress` — at proxy startup. It is read by `headroom proxy` and by the `headroom wrap` subprocesses. When unset, the default profile is `coding`.
| Profile | Target savings | Mode | Notes |
| ---------- | ---------------- | ------- | ------------------------------------------------------------------------------------------------- |
| `coding` | emergent (\~50%) | `cache` | **Default.** Delta-only compression at \~0 prefix-cache busts; never lossy-compresses file reads. |
| `balanced` | \~70% | `token` | Moderate compression with structural compaction. |
| `agent-90` | \~90% | `token` | Aggressive; pins a `0.10` keep-ratio and forces Kompress. |
| `general` | emergent (\~60%) | `token` | Non-coding workloads. |
An unrecognized `HEADROOM_SAVINGS_PROFILE` value logs a warning and falls back to `coding` (the same default an unset variable resolves to) — the proxy never fails to start over a bad profile name. `balanced` is used only as a last-resort fallback when the runtime has no `coding` entry at all (old-runtime/new-client version skew). See `headroom/agent_savings.py` for each profile's full set of knobs.
Because the default `coding` profile uses **cache** mode (and the proxy's own default mode is also `cache`), Headroom runs in cache mode out of the box. Mode precedence: an explicit `--mode` wins, otherwise `HEADROOM_MODE` (which a profile seeds), otherwise the `cache` default. To run token mode, pass `--mode token` or choose a token-mode profile:
```bash
# Aggressive ~90% token-savings profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --port 8787
```
### Optional features [#optional-features]
| Option | Default | Description |
| ------------------------------- | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--memory` | `false` | Enable persistent user memory and provider-appropriate memory tools |
| `--memory-db-path` | `{cwd}/.headroom/memory.db` | Override the memory SQLite path |
| `--no-memory-tools` | `false` | Disable automatic memory tool injection |
| `--no-memory-context` | `false` | Disable automatic memory context injection |
| `--memory-top-k` | `10` | Number of memories to inject as context |
| `--learn` | `false` | Enable live traffic learning; implies `--memory` |
| `--no-learn` | `false` | Explicitly disable traffic learning |
| `--min-evidence` | `5` | Minimum observations before a learned pattern is persisted |
| `--codex-wire-debug` | `false` | Write local Codex wire snapshots and matching proxy log traces |
| `HEADROOM_COMPRESS_PASSTHROUGH` | `0` | Also compress custom proxy paths that fall through to the catch-all handler (OpenAI Responses-shaped bodies, path ends in `/responses`). No `--compress-passthrough` flag on `headroom proxy`; the direct module entry point (`python -m headroom.proxy.server`) does accept one |
```bash
headroom proxy --memory
headroom proxy --learn --min-evidence 3
headroom proxy --codex-wire-debug
HEADROOM_COMPRESS_PASSTHROUGH=1 headroom proxy
```
The old LLMLingua proxy toggles are no longer part of the CLI. Headroom's proxy compression path uses ContentRouter plus the current built-in compressors, including Kompress where applicable.
## Savings profiles [#savings-profiles-1]
The proxy uses a **savings profile** to control compression behavior — which messages get compressed, how aggressively, and whether to prioritize provider prefix-cache stability or raw savings. Only the env var survives across related tools (`headroom wrap` passes it to the proxy it launches).
```bash
# Switch to a different profile
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy
```
### Built-in profiles [#built-in-profiles]
| Profile | Target savings | proxy\_mode | force\_kompress | Best for |
| ------------------ | ---------------- | ----------- | --------------- | ---------------------------------------------------------- |
| `coding` (default) | \~50% (emergent) | `cache` | No | Coding agents — preserves Anthropic prefix-cache stability |
| `agent-90` | 90% | `token` | Yes | Non-coding, cost-sensitive, or high-volume workloads |
| `balanced` | 70% | `token` | No | General-purpose moderate compression |
| `general` | \~60% (emergent) | `token` | No | Non-coding chat, little code in context |
**`coding` (default)** — Optimizes for coding-agent workloads with Anthropic. Uses **cache mode** (`proxy_mode="cache"`): compresses only the newest delta in each turn so the provider's prefix-cache is never busted. User messages are compressed, system prompts preserved (hottest cache). `protect_recent` is `0` — in cache mode the delta already *is* the newest turn, so a turn-count guard would suppress the only thing cache mode compresses; byte-exact fidelity instead comes from `protect_reads=True` (file reads are never lossy-compressed). Lossless-first with lossy fallback; tool search and cross-turn dedup enabled. This is the profile that `headroom wrap` uses.
**`agent-90`** — Forces ML-based (Kompress) compression with a 10% keep-ratio, ignoring the lossless path. Compresses both user and system messages. Designed for non-coding or cost-sensitive workloads where maximum compression is the goal.
**`balanced`** — Token-mode compression with a 30% keep-ratio. Uses the standard lossless pipeline (does not force Kompress). Protects 4 recent turns. A safe general-purpose profile.
**`general`** — Token-mode compression for non-coding conversations. No turn protection (`protect_recent=0` — nothing code-positional to preserve) and does not compress user or system messages. Uses the standard lossless pipeline.
### Profiles override CLI flags [#profiles-override-cli-flags]
A profile's `proxy_mode` setting overrides the `--mode` flag. The `coding` profile sets `proxy_mode="cache"`, so `--mode token` has **no effect** when coding is active:
```bash
# These are equivalent — coding's cache mode always wins
headroom proxy
headroom proxy --mode token # --mode token is silently overridden
```
To run in token mode, switch to a profile that uses it:
```bash
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy --mode token
```
### Extending a profile with env overrides [#extending-a-profile-with-env-overrides]
Profile defaults are applied only when the corresponding env var is not already set. You can start from a named profile and override individual settings:
```bash
# Start from coding but force Kompress on
HEADROOM_SAVINGS_PROFILE=coding HEADROOM_FORCE_KOMPRESS=1 headroom proxy
# Start from balanced but lower the keep-ratio
HEADROOM_SAVINGS_PROFILE=balanced HEADROOM_TARGET_RATIO=0.15 headroom proxy
```
### Custom profiles [#custom-profiles]
For permanent custom profiles, see the profile definitions in `headroom/agent_savings.py`. Each profile is an `AgentSavingsProfile` dataclass with fields for compression mode, target ratio, turn protection, and pipeline toggles.
## Configuration in depth [#configuration-in-depth]
Proxy behavior is set by three layers, **each overriding the one before**:
1. **Savings profile** (`HEADROOM_SAVINGS_PROFILE`) — seeds a whole posture (mode, keep-ratio, which roles get compressed, Kompress on/off). Default `coding`. See [Savings profiles](#savings-profiles).
2. **Environment variables** — nearly every CLI flag has an `HEADROOM_*` twin, which is what you'll use in Docker, systemd, or CI.
3. **CLI flags** — the most explicit; they win over env and profile.
The tables below group the knobs by what they control. They aren't exhaustive (`headroom proxy --help` prints the full list), but they cover what real deployments actually touch. Unless noted, every option is off/unset by default and safe to ignore.
### Compression tuning [#compression-tuning]
Fine-grained control over what gets compressed and how hard. Most users pick a [profile](#savings-profiles) instead and never touch these.
| Flag / env | Default | Effect |
| ---------------------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--mode` / `HEADROOM_MODE` | `cache` | `cache` compresses only the newest delta (prefix-cache safe); `token` maximizes removal. |
| `--target-ratio` / `HEADROOM_TARGET_RATIO` | unset | Keep-ratio for ML text compression; lower = more aggressive (e.g. `0.10`). |
| `HEADROOM_MIN_TOKENS` | `500` | Minimum block size before a tool output is compressed. |
| `HEADROOM_COMPRESS_USER_MESSAGES` | `false` | Compress content inside user-role messages (tool results live there). The `coding` profile turns this on. No `--compress-user-messages` flag on `headroom proxy`; the direct module entry point (`python -m headroom.proxy.server`) does accept one. |
| `HEADROOM_COMPRESS_SYSTEM_MESSAGES` | unset | Compress system prompts. Off by default to keep the hottest cache prefix stable. |
| `HEADROOM_PROTECT_RECENT` | profile | Never compress the N most recent turns. |
| `--protect-tool-results` / `HEADROOM_PROTECT_TOOL_RESULTS` | empty | Comma-separated tool names whose output is never lossy-compressed. |
| `--compressor` (repeatable) / `HEADROOM_COMPRESSORS` | all | Restrict to specific compressors: `smart_crusher,kompress,code_aware,search,log,tabular,config,html,image`. |
| `--code-aware` / `--no-code-aware` | off | AST-based [code compression](/docs/code-compression). Requires `headroom-ai[code]`. |
### Kompress (ML compression) [#kompress-ml-compression]
Kompress is the ModernBERT/ONNX compressor that ContentRouter falls back to for prose and unstructured text. It can run in-process or be offloaded to a hosted endpoint.
| Flag / env | Default | Effect |
| ------------------------------------------------------------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--disable-kompress` / `HEADROOM_DISABLE_KOMPRESS` | `false` | Turn off ML compression; keep the structural compressors. |
| `--disable-kompress-anthropic` / `--disable-kompress-openai` | inherit | Per-provider override. |
| `HEADROOM_FORCE_KOMPRESS_ALL` | `false` | Route *all* content through Kompress, bypassing per-type selection. No `--force-kompress-all` flag on `headroom proxy`; the direct module entry point (`python -m headroom.proxy.server`) does accept one. |
| `HEADROOM_KOMPRESS_ENDPOINT` | none | Offload ML compression to a remote `/compress` endpoint (e.g. a Modal deployment) instead of running the model locally. |
| `HEADROOM_KOMPRESS_ENDPOINT_TOKEN` | none | Bearer token for the remote endpoint. |
| `HEADROOM_KOMPRESS_BACKEND` | `auto` | Compute backend: `auto`, `onnx_cpu`, `onnx_coreml`, `pytorch`, `pytorch_mps`. |
### Reversible compression (CCR) and lossless mode [#reversible-compression-ccr-and-lossless-mode]
By default Headroom stores originals so the model can recover them via `headroom_retrieve`. See [Reversible Compression](/docs/ccr).
| Flag / env | Default | Effect |
| ---------------------------------- | ------------ | ----------------------------------------------------------------------------------- |
| `--no-ccr` / `HEADROOM_NO_CCR` | CCR on | Disable retrieval markers **and** the injected `headroom_retrieve` tool. |
| `--lossless` / `HEADROOM_LOSSLESS` | `false` | Format-native lossless compaction only — no CCR marker, no retrieval tool. |
| `--no-ccr-proactive-expansion` | expansion on | Stop proactively re-expanding compressed content when the model appears to need it. |
### File-read handling [#file-read-handling]
Coding agents re-read the same files repeatedly; these control how stale reads are handled without busting the prefix cache.
| Flag / env | Default | Effect |
| ------------------------------------------------ | ------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
| `--no-read-lifecycle` | lifecycle on | Stop replacing stale/superseded file reads with CCR markers. |
| `--read-maturation` / `HEADROOM_READ_MATURATION` | `false` | *(Beta)* Hold freshly-read files out of the prefix cache until the file quiesces. Requires `HEADROOM_ROLLOUT_CHANNEL=beta` (or `dev`). |
| `--read-maturation-quiesce-turns` | `5` | Turns of no change before a held read is admitted. |
### Reliability: timeouts, retries, limits [#reliability-timeouts-retries-limits]
| Flag / env | Default | Effect |
| -------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------- |
| `--request-timeout-seconds` / `HEADROOM_REQUEST_TIMEOUT` | `300` | Upstream request timeout (seconds). |
| `--connect-timeout-seconds` | `10` | Upstream connect timeout (seconds). |
| `--retry-max-attempts` | `3` | Upstream retries on transient failure. |
| `--limit-concurrency` | `1000` | Concurrent connections before returning 503. |
| `--rpm` / `--tpm` | `60` / `100000` | Requests- and tokens-per-minute rate limits (disable with `--no-rate-limit`). |
| `--budget` / `--budget-period` | none / `daily` | Spend cap in USD per period; over-budget requests get 429. |
| `--workers` / `HEADROOM_WORKERS` | `1` | Uvicorn worker processes. |
### Tool search and MCP [#tool-search-and-mcp]
Defers large tool schemas so they don't sit in every request. See [MCP](/docs/mcp).
| Env | Scope | Effect |
| ---------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `HEADROOM_TOOL_SEARCH` | proxy (server-side) | Defer MCP/system tool schemas behind a `search_tools` tool. **On by default** for Anthropic requests carrying enough tools to be worth it; set `HEADROOM_TOOL_SEARCH=0` to opt out. |
| `ENABLE_TOOL_SEARCH` | client (Claude Code) | Keep Claude Code's own deferred tool-loading active behind a custom base URL ([issue #746](https://github.com/headroomlabs-ai/headroom/issues/746)). Set automatically by `headroom wrap`. |
### Cost-aware model routing [#cost-aware-model-routing]
Rewrite the upstream model per request — for example, send small, tool-free calls to a cheaper model. Opt-in and off by default. Configure with `HEADROOM_MODEL_ROUTER_ENABLED` plus `HEADROOM_MODEL_ROUTES`; see [Cost-aware model routing](/docs/configuration#cost-aware-model-routing).
### Observability [#observability]
| Flag / env | Default | Effect |
| ------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--telemetry` / `HEADROOM_TELEMETRY` | off | **Local-only** usage stats for your own `/stats`, `/metrics`, and dashboard. Nothing leaves the machine. |
| `--log-file` / `HEADROOM_LOG_FILE` | none | JSONL request/response log. |
| `--log-messages` | `false` | Include full message bodies in the log (may contain sensitive data). |
| `HEADROOM_LOG_LEVEL` | `warning` | uvicorn's log level (`critical`, `error`, `warning`, `info`, `debug`, `trace`). Raise to `info` for the per-request access log when diagnosing a deployed proxy. An unrecognized value warns and falls back to `warning`. |
| `HEADROOM_OTEL_METRICS_ENABLED` | `false` | Export OpenTelemetry metrics (`HEADROOM_OTEL_METRICS_ENDPOINT`, …). See [OTLP export](/docs/metrics#opentelemetry-otlp-export). |
| `HEADROOM_LANGFUSE_ENABLED` | `false` | Emit Langfuse traces (`LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY`). |
See [Metrics](/docs/metrics) for the Prometheus and Grafana setup.
### Security and networking [#security-and-networking]
| Flag / env | Default | Effect |
| ------------------------------------ | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `HEADROOM_PROXY_TOKEN` | none | Require a bearer token (`X-Headroom-Proxy-Token`) from non-loopback callers. |
| `HEADROOM_COMPRESS_ALLOW_REMOTE` | `false` | Allow non-loopback callers to reach [`POST /v1/compress`](#post-v1compress). Required to run Headroom as a gateway/sidecar; without it remote callers get `404`. |
| `--offline` / `HEADROOM_OFFLINE` | `false` | Air-gap mode: hard-disable **all** egress (telemetry, update checks, license reporting, model downloads). |
| `--stateless` / `HEADROOM_STATELESS` | `false` | Keep all state in memory; no filesystem writes (disables logs, memory, TOIN). |
| `HEADROOM_STRIP_INTERNAL_HEADERS` | `enabled` | Strip internal `x-headroom-*` headers before forwarding upstream. |
| `HEADROOM_TLS_STRICT` | strict | Set `0` to relax CA-constraint checks behind a corporate TLS-inspection proxy. |
### Performance [#performance]
| Flag / env | Default | Effect |
| ---------------------------------------------------------------- | --------- | ------------------------------------------------------------ |
| `--embedding-server` / `HEADROOM_EMBEDDING_SERVER` | off | Share one ONNX embedder across workers (\~600 MB RSS saved). |
| `--compression-max-workers` / `HEADROOM_COMPRESSION_MAX_WORKERS` | CPU count | Bound the CPU-bound compression threadpool. |
For programmatic deployment you can pass an entire proxy config as JSON via `HEADROOM_PROXY_CONFIG_JSON`, or point `HEADROOM_CONFIG_DIR` / `HEADROOM_WORKSPACE_DIR` at custom roots (see [Filesystem Contract](/docs/filesystem-contract)).
## API endpoints [#api-endpoints]
### `GET /health` [#get-health]
```bash
curl http://localhost:8787/health
```
```json
{
"status": "healthy",
"optimize": true,
"stats": {
"total_requests": 42,
"tokens_saved": 15000,
"savings_percent": 45.2
}
}
```
### `GET /stats` [#get-stats]
Live session statistics plus durable `persistent_savings` totals. Stored at `~/.headroom/proxy_savings.json` (override with `HEADROOM_SAVINGS_PATH`).
```bash
curl http://localhost:8787/stats
```
### `GET /stats-history` [#get-stats-history]
Durable history with hourly, daily, weekly, and monthly rollups. Powers the `/dashboard` view.
```bash
curl http://localhost:8787/stats-history
curl "http://localhost:8787/stats-history?format=csv&series=weekly"
```
### `GET /metrics` [#get-metrics]
Prometheus-format metrics for monitoring.
```bash
curl http://localhost:8787/metrics
```
```
headroom_requests_total{mode="optimize"} 1234
headroom_tokens_saved_total 5678900
headroom_persistent_savings_tokens_saved_total 5678900
headroom_compression_ratio_bucket{le="0.5"} 890
headroom_latency_seconds_bucket{le="0.01"} 800
headroom_cache_hits_total 456
```
`headroom_tokens_saved_total` is the runtime counter for the current proxy process. Use `headroom_persistent_savings_tokens_saved_total` for durable lifetime savings that match `/stats.persistent_savings`.
### `POST /v1/messages` [#post-v1messages]
Anthropic API format. The proxy compresses messages, forwards to Anthropic, and returns the response.
### `POST /v1/chat/completions` [#post-v1chatcompletions]
OpenAI API format. The proxy compresses messages, forwards to OpenAI, and returns the response.
### `POST /v1/responses` [#post-v1responses]
OpenAI Responses API format. The proxy compresses `input` payloads where applicable, forwards the request, and returns the response.
For Codex-compatible clients, the proxy also accepts these alias paths and routes them through the same handler:
* `POST /v1/codex/responses`
* `POST /backend-api/responses`
* `POST /backend-api/codex/responses`
Matching WebSocket and subpath aliases are also supported for Codex flows.
### Codex Live voice WebSocket [#codex-live-voice-websocket]
The proxy relays Codex Live voice frames without parsing or transforming them.
These paths use the same transparent transport:
* `ws://localhost:8787/v1/live`
* `ws://localhost:8787/v1/codex/live`
* `ws://localhost:8787/backend-api/live`
* `ws://localhost:8787/backend-api/codex/live`
Subscription authentication uses the derived ChatGPT backend path. API-key
authentication preserves the selected OpenAI-compatible base URL and inbound
path. The backend Live suffix defaults to `/live` and can be corrected with
`HEADROOM_CODEX_LIVE_WS_PATH` if the upstream contract changes. The exact
ChatGPT backend path is not confirmed by this proxy documentation.
### `POST /v1internal:streamGenerateContent` [#post-v1internalstreamgeneratecontent]
Google Cloud Code Assist / Antigravity compatibility endpoint used by Pi-style `google-gemini-cli` and `google-antigravity` providers.
The proxy also accepts:
* `POST /v1/v1internal:streamGenerateContent`
### `POST /v1/compress` [#post-v1compress]
Compression-only endpoint. Compresses messages and returns them without ever making a **completion request to an LLM provider** — no generation, no provider API key, no upstream chat call. Used by the TypeScript SDK, by LiteLLM's `headroom` guardrail, and by API gateways running Headroom as a sidecar.
"No LLM call" means no *generative* request to a provider. Compression itself is ML-backed: **Kompress** is a ModernBERT encoder that scores tokens for retention (classification, not generation), and Magika classifies content types. Both run in-process by default, so budget CPU and memory for the sidecar accordingly.
If `HEADROOM_KOMPRESS_ENDPOINT` is set, Kompress inference is offloaded over HTTP to that model server — **real egress from the sidecar**, which matters if you deployed it expecting none. Only inference goes remote: the CCR store and retrieval markers stay proxy-local, and original content never persists off-box. Leave the variable unset to keep everything in-process, or run with `HEADROOM_DISABLE_KOMPRESS=1` for structural compression only.
This route is restricted to loopback callers and answers everyone else with **`404`**, not `403` — deliberately, so it stays invisible to external scanners. A gateway calling it from another host or pod therefore sees what looks like a missing route.
Both the client IP and the inbound `Host:` header must name loopback. To allow remote callers, set `HEADROOM_COMPRESS_ALLOW_REMOTE=1`. `HEADROOM_PROXY_TOKEN` still applies if set.
#### Message format [#message-format]
The endpoint does **no format conversion**. Whatever shape you send in `messages` is the shape you get back, and both wire formats are compressed natively:
* **OpenAI shape** — `role: "tool"` messages with `tool_call_id`, assistant `tool_calls`
* **Anthropic shape** — content-block lists with `tool_use` / `tool_result` / `thinking` blocks
So an Anthropic-native caller does not need to convert to OpenAI format first. Block types, `tool_use_id`s and message order are all preserved.
`model` selects the tokenizer (per-model, from Headroom's tokenizer registry) and the context limit. Send the real model name — including gateway-prefixed forms like `bedrock/anthropic.claude-3-5-sonnet` or `vertex_ai/claude-sonnet-4@20250514` — so token counts and compression aggressiveness are right.
#### Request [#request]
| Field | Type | Required | Description |
| -------------- | ------- | -------- | ----------------------------------------------------------------------------------------------------------------- |
| `messages` | array | yes | Messages to compress, in either wire format. `400` if missing. Empty array returns immediately with zero metrics. |
| `model` | string | yes | Model name. Drives tokenizer + context-limit resolution. `400` if missing. |
| `token_budget` | integer | no | Overrides the model's context limit. Used by callers that need to fit a tighter budget. |
| `config` | object | no | Compression options, below. A non-object value is ignored rather than rejected. |
Only the four fields above are read. Anthropic sends `system` and `tools` **out of band**, alongside `messages` — this endpoint accepts them without complaint (you get a `200`, no warning) and returns neither, so neither is compressed.
Keep carrying both yourself and send them upstream unchanged. Two consequences worth knowing:
* An Anthropic system prompt is not compressed here, even though it is resent on every request.
* Tool-schema compaction and tool-search deferral are not reachable through this endpoint — on tool-heavy traffic those can be the largest share of available savings. Run Headroom as the proxy (rather than calling `/v1/compress`) if you need them.
`config` fields:
| Field | Type | Default | Description |
| -------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `mode` | string | unset | `ccr`, `lossy_inline`, or `lossless_then_lossy`. Unset selects the default marker-free pipeline. Any other value is a `400`. |
| `frozen_message_count` | integer | unset | Pin a prefix: the first N messages are returned byte-for-byte unchanged while staying visible to cross-message transforms like dedup. Set it to the number of messages the provider has already cached so compression cannot rewrite the prefix and bust that cache. Must be a non-negative integer; anything else is a `400`. |
| `compress_user_messages` | boolean | `false` | Also compress user-role messages. |
| `target_ratio` | number | unset | Target compression ratio. |
| `protect_recent` | integer | unset | Leave the last N messages uncompressed. |
| `protect_analysis_context` | boolean | unset | Preserve analysis context blocks. |
**`config.mode` values:**
* **unset (default)** — marker-free. Emits no `<>` retrieval markers and writes nothing to the CCR store, so you can forward the returned messages straight to a provider. This is the right mode for a gateway or guardrail that just swaps `messages` and forwards.
* **`ccr`** — emits CCR markers and writes to the store. Only for callers that also inject the `headroom_retrieve` tool *and* can reach `/v1/retrieve` (itself loopback-only). Markers are a dangling pointer for the model otherwise.
* **`lossy_inline`** (alias `lossless_then_lossy`) — runs the lossless byte/data fold first, then compresses the folded remainder. Marker-free.
#### Response [#response]
| Field | Type | Description |
| -------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `messages` | array | Compressed messages, in the shape you sent. |
| `tokens_before` | integer | Token count before compression. |
| `tokens_after` | integer | Token count after compression. |
| `tokens_saved` | integer | `tokens_before - tokens_after`. |
| `compression_ratio` | number | `tokens_after / tokens_before` — so **lower is better**. A ratio of `0.23` means a 77% reduction, not 23%. `1.0` when nothing was compressed. |
| `transforms_applied` | array | Transform labels that ran. |
| `transforms_summary` | object | Per-transform counts. |
| `ccr_hashes` | array | Retrieval hashes for markers inserted (empty unless `mode: "ccr"`). |
```json
{
"messages": [{ "role": "user", "content": "..." }],
"tokens_before": 15000,
"tokens_after": 3500,
"tokens_saved": 11500,
"compression_ratio": 0.23,
"transforms_applied": ["router:smart_crusher:0.35"],
"transforms_summary": { "router:smart_crusher:0.35": 1 },
"ccr_hashes": []
}
```
#### Headers [#headers]
`x-headroom-bypass: true` (case-insensitive) skips compression entirely and echoes your messages back with zeroed metrics. The bypass and empty-messages responses omit `transforms_summary`.
#### Errors and fail-open [#errors-and-fail-open]
| Status | Body | When |
| ------ | ---------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `400` | `error.type = "invalid_request"` | Missing `messages` or `model`, malformed JSON, invalid `config.mode`, or invalid `config.frozen_message_count`. |
| `401` | — | `HEADROOM_PROXY_TOKEN` is set and the bearer token is missing or wrong. |
| `404` | — | Non-loopback caller without `HEADROOM_COMPRESS_ALLOW_REMOTE=1`. |
| `503` | `error.type = "compression_error"` | Compression failed unexpectedly. |
Compression **fails open on timeout**: you get `200` with your original messages, zeroed metrics, plus `compression_skipped: true` and `skip_reason: "compression_timeout"`. Always check `compression_skipped` if you need to know whether compression actually ran.
Requests are recorded under `provider="compress"` in `/stats` and `/metrics`.
#### Multi-turn usage: keeping the prefix cache [#multi-turn-usage-keeping-the-prefix-cache]
This is the single most important thing to get right, and the default is not safe for an agent loop.
When Headroom proxies a request itself it watches the provider's cache hit rate turn over turn and freezes the already-cached prefix. `/v1/compress` **cannot do that — it is stateless.** It sees one isolated call and has no idea what the provider already cached.
The provider caches the bytes you **forwarded**. Compression changed those bytes, so your original messages and the ones the provider cached are no longer the same thing — and it is the forwarded version you have to keep reproducing. Send the pristine originals again next turn and the provider sees a different prefix and re-reads it from scratch. On Anthropic a cache read is \~90% cheaper than fresh input, so that can easily cost more than the compression saves.
Compression is also not uniform over a conversation: how hard a message is compressed depends partly on how far it now sits from the end, so an older tool result can fall outside the recent-read protection window as the conversation grows and be compressed harder than it was last turn. Another reason not to rely on re-compression reproducing earlier output.
Two rules:
1. **Pass `config.frozen_message_count`** — how many leading messages the provider has already cached.
2. **Send back your own previous output, not the original messages.** `frozen_message_count` returns those leading messages *exactly as you passed them in* — it pins whatever you hand it. Hand it pristine originals and you get pristine originals back, which is precisely the prefix the provider does not have.
```python
# Keep what you FORWARDED, not what you started with.
forwarded: list[dict] = []
def next_turn(new_messages: list[dict]) -> list[dict]:
body = {
"messages": forwarded + new_messages,
"model": "claude-sonnet-4-6",
# Everything already forwarded is already cached upstream — pin it.
"config": {"frozen_message_count": len(forwarded)},
}
result = requests.post(f"{proxy}/v1/compress", json=body).json()
forwarded[:] = result["messages"] # becomes next turn's frozen prefix
return forwarded
```
Compressing the full original conversation on each turn looks correct — you get a `200` and a positive `tokens_saved` — but the leading messages come back different from the ones the provider cached. You pay for compression *and* for a cache miss. Nothing in the response tells you this happened; watch your provider's cache-read tokens.
Also for multi-turn callers:
* **Leave `config.mode` unset.** The default is marker-free, which is what a forward-only caller wants.
* **Send the real model name** so the tokenizer and context limit resolve correctly — including gateway-prefixed forms.
* **`protect_recent` is not a substitute.** It guards the newest messages; `frozen_message_count` guards the oldest, which is the cached end.
`HEADROOM_KOMPRESS_ENDPOINT` points *outbound* at a remote Kompress ML model server that happens to expose a `/compress` path. It is unrelated to this inbound endpoint.
## Agent wrapping [#agent-wrapping]
Use `headroom wrap` to launch supported CLI agents through the local proxy:
```bash
# Claude Code
headroom wrap claude
# Claude Code extension in VS Code (configures settings, then starts the proxy)
headroom wrap vscode-claude
# OpenAI Codex
headroom wrap codex
# Aider
headroom wrap aider
# Cursor (starts the proxy and prints settings to paste into Cursor)
headroom wrap cursor
# Grok Build (updates ~/.grok/config.toml and starts the proxy)
headroom wrap grok-build
```
Cursor reads model endpoints from its settings UI, so `headroom wrap cursor`
does not rewrite Cursor configuration or launch the app. After it starts the
proxy, copy the printed base URL into Cursor's model settings.
Grok Build reads model endpoints from `~/.grok/config.toml`. `headroom wrap grok-build`
injects or updates `[model.grok-build] base_url` to point at the local proxy, then
run `grok` from the same project directory. See [Grok Build Integration](/docs/grok-build).
The official Claude Code extension reads Claude Code's user settings rather than
the terminal environment. Use `headroom wrap vscode-claude`, reload VS Code after
the first run, and keep the wrapper running. See the
[VS Code Claude Code guide](/docs/vscode-claude-code) for verification and undo
steps.
For environment-driven clients, you can also set the base URL manually:
```bash
# Claude Code
ANTHROPIC_BASE_URL=http://localhost:8787 claude
# Any OpenAI-compatible CLI client that reads OPENAI_BASE_URL
OPENAI_BASE_URL=http://localhost:8787/v1 your-client
```
## Cloud providers [#cloud-providers]
```bash
# AWS Bedrock
headroom proxy --backend bedrock --region us-east-1
# Google Vertex AI
headroom proxy --backend vertex_ai --region us-central1
# Azure OpenAI
headroom proxy --backend azure
# OpenRouter (400+ models)
OPENROUTER_API_KEY=sk-or-... headroom proxy --backend openrouter
```
### Google Vertex AI [#google-vertex-ai]
`--backend vertex_ai` delegates to [LiteLLM](https://docs.litellm.ai/docs/providers/vertex),
which brings two requirements that are easy to miss:
**1. Install the Vertex SDK.** `google-cloud-aiplatform` is not included in any
Headroom extra (`[proxy]`, `[all]`, …) or Docker image variant, so install it
alongside Headroom:
```bash
pip install "headroom-ai[proxy]" "google-cloud-aiplatform>=1.38"
```
Without it, the first Vertex request fails with
`litellm.BadRequestError: … vertexai import failed … No module named 'vertexai'`.
**2. Set the LiteLLM project/location variables.** LiteLLM reads the GCP project
and region from `VERTEXAI_PROJECT` / `VERTEXAI_LOCATION` — these are **not** the
standard Google Cloud variables (`GOOGLE_CLOUD_PROJECT` / `GOOGLE_CLOUD_LOCATION`)
used by `gcloud` and current Google SDKs:
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json # or use ADC
export VERTEXAI_PROJECT=
export VERTEXAI_LOCATION=us-central1
headroom proxy --backend vertex_ai --region us-central1
```
If `VERTEXAI_PROJECT` is unset, requests do not fail loudly — they can silently
resolve against your Application Default Credentials' default quota project,
billing a different GCP project than you intended. Set it explicitly even if
`GOOGLE_CLOUD_PROJECT` is already exported.
**Backend name aliases.** `vertex_ai`, `vertex`, `google-vertex`, `googlevertex`,
`litellm-vertex`, and `litellm-vertex_ai` are all normalized to the same
LiteLLM-backed `vertex_ai` backend — CLI help text and older docs use these
spellings interchangeably.
Running Claude Code against Claude models on Vertex? See
[Claude Code on Vertex AI](/docs/claude-code-vertex) for the recommended
native Vertex-mode flow that reuses Claude Code's own GCP auth.
`--backend vertex_ai` runs Headroom **as a proxy that itself calls Vertex via
LiteLLM**. The [LiteLLM integration page](/docs/litellm) documents the inverse:
adding Headroom as a compression callback inside your own LiteLLM app. Despite
the shared name, they are different mechanisms.
### Native Vertex passthrough routes [#native-vertex-passthrough-routes]
Separately from `--backend vertex_ai`, the proxy always registers routes that
mirror Vertex's native REST shape verbatim — no backend flag needed:
```text
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:generateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamGenerateContent
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:countTokens
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:rawPredict
/{api_version}/projects/{project}/locations/{location}/publishers/{publisher}/models/{model}:streamRawPredict
```
Requests with `publisher=google` (Gemini models) go through Headroom's full
Gemini optimization handler; `publisher=anthropic` (Claude on Vertex) routes
through the same LiteLLM-Vertex path as `--backend vertex_ai`. Any client that
already speaks the native Vertex REST API can simply point its endpoint at the
proxy — this is the mechanism `headroom wrap claude` uses in
[Vertex mode](/docs/claude-code-vertex).
### Bedrock via a local gateway [#bedrock-via-a-local-gateway]
`--backend bedrock` accepts **Anthropic** input (`/v1/messages`) and re-signs to AWS. Some setups are the other way around: the client already speaks **Bedrock** (e.g. Claude Code with `CLAUDE_CODE_USE_BEDROCK=1`, or any AWS SDK pointed at a custom endpoint), sending `POST /model/{id}/invoke` to a local gateway that re-signs and forwards to AWS (LiteLLM, LocalStack, a corporate Bedrock proxy).
`--bedrock-api-url` lets Headroom sit in that chain. It registers passthrough routes for `/model/{id}/invoke` and `/model/{id}/invoke-with-response-stream`, compresses the request body with the same pipeline as `/v1/messages`, and forwards to the gateway:
```bash
headroom proxy --bedrock-api-url http://127.0.0.1:4000
# then point the client's Bedrock endpoint at Headroom:
AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://127.0.0.1:8787 your-bedrock-client
```
The routes are registered **only** when `--bedrock-api-url` (or `BEDROCK_TARGET_API_URL`) is set — otherwise Bedrock requests fall through unchanged.
Rewriting the request body invalidates the caller's **SigV4** signature (it covers a hash of the body). Point `--bedrock-api-url` at a gateway that re-signs or does not verify the inbound signature — **never raw AWS**, which would reject the request with 403. For direct-to-AWS compression, use `--backend bedrock` (which re-signs). The two are complementary.
## Environment variables [#environment-variables]
```bash
export HEADROOM_HOST=0.0.0.0
export HEADROOM_PORT=8787
export HEADROOM_BUDGET=100.0
# Route OpenAI passthrough requests to a custom endpoint
export OPENAI_TARGET_API_URL=https://custom.openai.endpoint.com
# Route Anthropic passthrough requests to a custom endpoint
export ANTHROPIC_TARGET_API_URL=https://litellm.company.internal
# Compress Bedrock InvokeModel traffic, forwarding to a re-signing gateway
export BEDROCK_TARGET_API_URL=http://127.0.0.1:4000
headroom proxy
```
## Production deployment [#production-deployment]
### gunicorn [#gunicorn]
```bash
pip install gunicorn
gunicorn "headroom.proxy.server:create_app()" \
--workers 4 \
--bind 0.0.0.0:8787 \
--worker-class uvicorn.workers.UvicornWorker
```
`headroom.proxy.server` has no module-level `app` object — it exposes a `create_app(config: ProxyConfig | None = None) -> FastAPI` factory, so gunicorn's import string must call it.
### Docker [#docker]
```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
&& pip install "headroom-ai[proxy]" \
&& apt-get purge -y build-essential && apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
EXPOSE 8787
CMD ["headroom", "proxy", "--host", "0.0.0.0"]
```
`build-essential` is required at install time because `headroom-ai` includes `hnswlib`, a C++ extension compiled from source. It is removed after installation to keep the image slim.
This guide gets you from zero to compressed LLM calls in under 5 minutes.
## 1. Install [#1-install]
```bash
npm install headroom-ai
```
```bash
# CLI/proxy/wrap on your machine
uv tool install --python 3.13 "headroom-ai[all]"
# Python project or virtualenv
pip install "headroom-ai[all]"
```
The TypeScript SDK sends messages to a local Headroom proxy for compression. Start the proxy before using the TS SDK:
```bash
uv tool install --python 3.13 "headroom-ai[proxy]"
# or, inside a Python project: pip install "headroom-ai[proxy]"
headroom proxy --port 8787
```
The proxy runs the compression pipeline (Python) and exposes an HTTP API that the TS SDK calls.
## 2. Compress messages [#2-compress-messages]
```ts twoslash
import { compress } from 'headroom-ai';
const messages = [
{ role: 'system' as const, content: 'You analyze search results.' },
{ role: 'user' as const, content: 'Search for Python tutorials.' },
{
role: 'assistant' as const,
content: null,
tool_calls: [{
id: 'call_1',
type: 'function' as const,
function: { name: 'search', arguments: '{"q": "python"}' },
}],
},
{
role: 'tool' as const,
tool_call_id: 'call_1',
content: JSON.stringify({
results: Array.from({ length: 500 }, (_, i) => ({
title: `Result ${i}`,
snippet: `Description ${i}`,
score: 100 - i,
})),
}),
},
{ role: 'user' as const, content: 'What are the top 3 results?' },
];
const result = await compress(messages, {
model: 'gpt-4o',
baseUrl: 'http://localhost:8787',
});
```
```python
from headroom import compress
import json
messages = [
{"role": "system", "content": "You analyze search results."},
{"role": "user", "content": "Search for Python tutorials."},
{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {"name": "search", "arguments": '{"q": "python"}'},
}],
},
{
"role": "tool",
"tool_call_id": "call_1",
"content": json.dumps({
"results": [
{"title": f"Result {i}", "snippet": f"Description {i}", "score": 100 - i}
for i in range(500)
]
}),
},
{"role": "user", "content": "What are the top 3 results?"},
]
result = compress(messages, model="gpt-4o")
```
## 3. Send to your LLM [#3-send-to-your-llm]
Use the compressed messages exactly like the originals:
```ts twoslash
import OpenAI from 'openai';
const client = new OpenAI();
// result.messages from the previous step
const messages: any[] = [];
const response = await client.chat.completions.create({
model: 'gpt-4o',
messages,
});
console.log(response.choices[0].message.content);
```
```python
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=result.messages,
)
print(response.choices[0].message.content)
```
## 4. Check your savings [#4-check-your-savings]
```ts twoslash
const result = {
tokensBefore: 45000,
tokensAfter: 4500,
tokensSaved: 40500,
// compressionRatio is tokensAfter / tokensBefore, so savings is 1 - ratio.
compressionRatio: 0.1,
transformsApplied: ['smart_crusher', 'cache_aligner'],
messages: [],
ccrHashes: [],
compressed: true,
};
// ---cut---
console.log(`Tokens before: ${result.tokensBefore}`);
console.log(`Tokens after: ${result.tokensAfter}`);
console.log(`Tokens saved: ${result.tokensSaved}`);
console.log(`Compression: ${((1 - result.compressionRatio) * 100).toFixed(0)}%`);
console.log(`Transforms: ${result.transformsApplied.join(', ')}`);
```
Example output:
```
Tokens before: 45000
Tokens after: 4500
Tokens saved: 40500
Compression: 90%
Transforms: smart_crusher, cache_aligner
```
```python
print(f"Tokens before: {result.tokens_before}")
print(f"Tokens after: {result.tokens_after}")
print(f"Tokens saved: {result.tokens_saved}")
print(f"Compression: {result.compression_ratio:.0%}")
print(f"Transforms: {result.transforms_applied}")
```
Example output:
```
Tokens before: 45000
Tokens after: 4500
Tokens saved: 40500
Compression: 90%
Transforms: ['smart_crusher', 'cache_aligner']
```
## Alternative: proxy mode (zero code changes) [#alternative-proxy-mode-zero-code-changes]
Run `headroom wrap vscode-claude`, reload the VS Code window once, and keep the
wrapper running while you use the official Claude Code extension. See the
[complete VS Code Claude Code guide](/docs/vscode-claude-code) for verification,
undo steps, custom profiles, and remote development.
If you do not want to change any code, run Headroom as a proxy and point your existing client at it:
```bash
# Start the proxy
headroom proxy --port 8787
# Point Claude Code at it
ANTHROPIC_BASE_URL=http://localhost:8787 claude
# Or any OpenAI-compatible client
OPENAI_BASE_URL=http://localhost:8787/v1 your-app
```
All requests flow through Headroom automatically. Check savings at any time:
```bash
curl http://localhost:8787/stats
# {"requests_total": 42, "tokens_saved_total": 125000, ...}
```
## What gets compressed [#what-gets-compressed]
The biggest savings come from tool outputs -- search results, database rows, log files, API responses. Headroom auto-detects the content type and routes it to the best compressor. No configuration needed.
| Content type | Compressor | Typical savings |
| --------------- | ---------------- | ------------------------------------- |
| JSON arrays | SmartCrusher | varies with array size and redundancy |
| Source code | CodeCompressor | opt-in; disabled by default |
| Build/test logs | LogCompressor | varies with log verbosity |
| Search results | SearchCompressor | varies with result set size |
| Plain text | Kompress | varies with redundancy |
Savings depend heavily on how repetitive the content is -- see
[Benchmarks](/docs/benchmarks) for measured numbers on real workloads.
## Next steps [#next-steps]
## Overview [#overview]
Headroom uses `release-please` to maintain a release PR from conventional commits on `main`. Merging that release PR creates the release tag and GitHub Release, which triggers `.github/workflows/release.yml` to publish all packages, build version-matched Docker images, and attach release assets.
The release workflow also calls `.github/workflows/docker.yml` as a reusable workflow so GHCR images are published in the same release run with the exact same synced version as PyPI, npm, and GitHub release assets.
For the end-to-end visual flow, see [CI/CD Flow Diagrams](/docs/ci-cd-flows).
## Packages & Registries [#packages--registries]
| Package | Type | Registry | Environment Variable |
| ------------------------------------------------------------------------- | ---------------------------- | ----------------------------------- | ---------------------- |
| `headroom-ai` | Python | PyPI | `PYPI_PACKAGE` |
| `headroom-ai` | TypeScript SDK | npmjs.org | `NPM_SDK_PACKAGE` |
| `headroom-openclaw` | TypeScript plugin | npmjs.org | `NPM_OPENCLAW_PACKAGE` |
| `headroom-opencode` | TypeScript plugin | npmjs.org | `NPM_OPENCODE_PACKAGE` |
| `@{owner}/headroom-ai` | TypeScript SDK | GitHub Package Registry | — |
| `@{owner}/headroom-openclaw` | TypeScript plugin | GitHub Package Registry | — |
| `headroom-ai-{version}.tar.gz` / `headroom_ai-{version}-py3-none-any.whl` | Python package distributions | GitHub Release (`{owner}/headroom`) | — |
| `headroom-ai-{version}.tgz` / `headroom-openclaw-{version}.tgz` | Node release assets | GitHub Release (`{owner}/headroom`) | — |
| `ghcr.io/{owner}/headroom` | Docker image | GitHub Container Registry | — |
## Version Strategy [#version-strategy]
Release Please calculates the release version from conventional commits and the release manifest. The release workflow still computes and verifies the version it is about to publish:
1. `.release-please-config.json` defines release-please behavior.
2. `.release-please-manifest.json` tracks current package versions.
3. The release PR updates versions and changelog content.
4. Merging the release PR publishes a GitHub Release tagged `vX.Y.Z`.
5. `release.yml` uses that tag as the manual version for the publish run.
### Version Files [#version-files]
* `.release-please-config.json` - release-please package configuration
* `.release-please-manifest.json` - release-please version manifest
* `pyproject.toml` - `[project].version`
* `headroom/_version.py` - `__version__`, synced at build time
* `plugins/openclaw/package.json` - `version`, synced at build time
* `plugins/opencode/package.json` - `version`, synced at build time
* `sdk/typescript/package.json` - `version`, synced at build time
`release.yml` does not commit back to the repo. Version synchronization happens inside the release build workspace.
## Conventional Commits & Semantic Bumping [#conventional-commits--semantic-bumping]
Release Please analyzes unreleased conventional commits and applies the highest required bump level:
| Commit | Bump |
| --------------------------------------------------------------------------------- | -------------------------------------------- |
| `fix:` | patch |
| `feat:` | minor |
| Any conventional commit with `!` or any commit with `BREAKING CHANGE` in the body | major |
| `docs:`, `ci:`, `chore:`, `refactor:` | no release note by default unless configured |
Commits are linted in CI via `commitlint` using `@commitlint/config-conventional`.
The release PR is the place where version and changelog changes are reviewed before publishing.
## Release Workflow [#release-workflow]
The `release.yml` workflow runs when a GitHub Release is published, which normally happens when the release-please PR is merged. It also supports manual `workflow_dispatch` and PR dry-runs for release-critical workflow/package changes.
```
detect-version → build → build-wheels ─┬→ collect-dist ─┐
│ └→ smoke-import-wheels ─┬→ publish-pypi ─┐
├──────────────────────────────────────┼→ publish-docker ┤
├→ publish-npm ──────────────────────────────────────────┤
└→ publish-github-packages ──────────────────────────────┴→ create-release
```
`build-wheels` fans out to `collect-dist` and `smoke-import-wheels` in
parallel (both only need the wheel artifacts, not each other).
`publish-npm` and `publish-github-packages` only need `detect-version` +
`build` -- they do not wait on the wheel matrix at all. `publish-pypi` and
`publish-docker` both gate on `smoke-import-wheels`. `create-release` is the
only job that waits on everything (`.github/workflows/release.yml:78-1004`).
The workflow never commits back to the repo.
### detect-version [#detect-version]
Resolves the release version from the trigger. On `release: published`, it uses the published tag (`vX.Y.Z`) as the manual version for the run. On `workflow_dispatch`, it uses the optional `version` input when provided. PR dry-runs compute a version without publishing.
### build [#build]
1. Syncs version across package files via `scripts/version-sync.py --version {npm_version}`
2. Verifies package versions with `scripts/verify-versions.py`
3. Generates the changelog artifact
4. Builds npm release packages for the TypeScript SDK and OpenClaw plugin
5. Uploads release asset artifacts for downstream publish jobs
### build-wheels [#build-wheels]
Builds the Python wheel matrix for Linux x86\_64, Linux arm64, Apple Silicon macOS, Intel macOS, and Windows x86\_64, plus one source distribution (`.github/workflows/release.yml:214-264`). Linux wheels are audited for glibc symbol compatibility.
### collect-dist [#collect-dist]
Collects the wheel matrix, source distribution, and npm tarballs into the canonical artifacts used by publishing and GitHub Release asset upload.
### smoke-import-wheels [#smoke-import-wheels]
Installs the built wheels into representative customer environments and imports `headroom._core`. This blocks publishing if a wheel builds successfully but cannot import on its promised platform floor.
### publish-pypi [#publish-pypi]
Downloads the Python dist artifact and publishes to PyPI via `pypa/gh-action-pypi-publish@v1.13.0` (trusted publisher).
### publish-npm [#publish-npm]
Publishes all npm packages to npmjs.org:
* `sdk/typescript/` as `headroom-ai`
* `plugins/openclaw/` as `headroom-openclaw`
* `plugins/opencode/` as `headroom-opencode`
### publish-github-packages [#publish-github-packages]
Publishes both Node packages to GitHub Package Registry (`npm.pkg.github.com`) using the current repository owner as the npm scope:
* `sdk/typescript/` as `@{owner}/headroom-ai`
* `plugins/openclaw/` as `@{owner}/headroom-openclaw`
### GitHub release assets [#github-release-assets]
Uploads the built Python distributions and both npm tarballs to the GitHub Release created in the current repository. GitHub Packages does not provide a PyPI-compatible package registry, so the workflow publishes Python wheels and sdists to GitHub as release assets while npm packages go to GitHub Package Registry and Docker images go to GHCR.
### publish-docker [#publish-docker]
Calls the reusable Docker workflow to publish GHCR images with the same semantic version and synced package metadata as the rest of the release.
### create-release [#create-release]
Creates or updates the GitHub Release in the current repo and uploads the built Python distributions and npm tarballs as release assets. PyPI publish is a hard gate unless `PYPI_SKIP=true`, so release notes and assets do not advertise a version that failed to publish to PyPI.
## Configuration [#configuration]
All package names, registry URLs, and environment names are defined as top-level `env` constants:
```yaml
env:
PYPI_PACKAGE: headroom-ai
PYPI_ENVIRONMENT: pypi
NPM_REGISTRY_URL: https://registry.npmjs.org
NPM_SDK_PACKAGE: headroom-ai
NPM_OPENCLAW_PACKAGE: headroom-openclaw
NPM_OPENCODE_PACKAGE: headroom-opencode
GITHUB_PACKAGES_REGISTRY_URL: https://npm.pkg.github.com
```
To rename a package, update the corresponding constant — all references throughout the workflow update automatically.
## Safety Gates [#safety-gates]
Each publish job requires all of the following to be true (`.github/workflows/release.yml:768`, PyPI shown; npm/GitHub Packages mirror it with their own skip variable):
```yaml
if: github.event_name != 'pull_request' && github.event.inputs.dry_run != 'true' && vars.PYPI_SKIP != 'true'
```
To skip a publish target, set the corresponding GitHub Actions variable:
| Variable | Effect |
| ----------------------- | ------------------------------------ |
| `PYPI_SKIP=true` | Skip PyPI publish |
| `NPM_SKIP=true` | Skip both npm publishes |
| `GH_PACKAGES_SKIP=true` | Skip GitHub Package Registry publish |
Set these in: **GitHub repo → Settings → Variables → Actions Variables → New repository variable**.
PyPI publishing is a hard gate for GitHub Releases unless `PYPI_SKIP=true`.
If the PyPI upload fails, the workflow stops before creating or updating the
GitHub Release, so release notes cannot advertise a version that was not
published to PyPI.
Before release artifacts are built, the workflow runs:
```bash
python scripts/verify-versions.py
```
That gate fails on cross-package version drift. The sdist build is also checked
for a top-level `LICENSE` file before any publish job can consume it.
## Workflow Triggers [#workflow-triggers]
Release Please runs on pushes to `main` and maintains the release PR:
```yaml
on:
push:
branches: [main]
```
The publish workflow runs when a GitHub Release is published, on PR dry-runs for release-critical paths, and by manual dispatch:
```yaml
on:
release:
types: [published]
pull_request:
paths:
- ".github/workflows/release.yml"
- ".github/workflows/docker.yml"
- "crates/headroom-py/**"
- "pyproject.toml"
- "scripts/verify-versions.py"
- "scripts/version-sync.py"
- "Cargo.toml"
- "Cargo.lock"
workflow_dispatch:
inputs:
version:
description: "Manual version override"
required: false
dry_run:
description: "Skip publish"
type: boolean
default: false
```
* **Normal release:** merge the release-please PR; the bot publishes a GitHub Release, which triggers `release.yml`.
* **PR dry-run:** release-critical PRs build and smoke-import wheels before merge, but do not publish.
* **Manual dispatch:** use `version` to override the release version and `dry_run: true` to skip publish steps.
## Local Testing with `act` [#local-testing-with-act]
### Prerequisites [#prerequisites]
```bash
# macOS
brew install act actionlint
# Windows
winget install act
winget install actionlint
```
### Dry-run Test [#dry-run-test]
```bash
act workflow_dispatch -W .github/workflows/release.yml -e .github/act/dry-run.json
```
This runs the full workflow end-to-end with `dry_run=true`, skipping all publish steps.
### Simulate Release Please [#simulate-release-please]
```bash
act push -W .github/workflows/release-please.yml -e .github/act/push-feat.json
```
The `push-feat.json` event file simulates a `feat:` commit on `main` so the release-please workflow can be validated locally.
### Simulate a Published Release [#simulate-a-published-release]
```bash
act release -W .github/workflows/release.yml -e .github/act/release-published.json -n
```
The `release-published.json` event file simulates the event emitted when the release-please PR is merged.
### Validate the Release and Docker Workflows [#validate-the-release-and-docker-workflows]
```bash
bash scripts/validate-workflows.sh
```
This runs `actionlint` plus `act -n` against the release and Docker workflows using the checked-in `.github/act/*.json` event fixtures. CI runs the same script in the `workflow-validation` job so branch changes to release automation are validated before merge.
### Local Secrets [#local-secrets]
```bash
cp .env.act.example .env.act
# Edit .env.act and add your test tokens
```
`act` automatically reads `.env` and passes values as workflow secrets.
## Workflow Files Reference [#workflow-files-reference]
| File | Purpose |
| -------------------------------------- | --------------------------------------------------------- |
| `.github/workflows/release.yml` | Main release pipeline |
| `.github/workflows/release-please.yml` | Release PR aggregation from conventional commits |
| `.github/workflows/ci.yml` | CI — lint, test, commitlint |
| `.github/workflows/publish.yml` | Manual-only PyPI fallback (superseded by `release.yml`) |
| `.commitlintrc.json` | Conventional commit rules |
| `scripts/version-sync.py` | Sync version across all packages |
| `scripts/changelog-gen.py` | Generate changelog from git log |
| `scripts/verify-versions.py` | Pre-release version alignment check |
| `.github/act/dry-run.json` | `act` event file for dry-run testing |
| `.github/act/push-feat.json` | `act` event file for feat commit testing |
| `.github/act/release-published.json` | `act` event file for release publish simulation |
| `.github/act/docker-version.json` | `act` event file for Docker workflow validation |
| `scripts/validate-workflows.sh` | Shared `actionlint` + `act -n` workflow validation script |
| `.actrc` | Default `act` flags (Ubuntu runner, reuse, quiet) |
| `.actrc.local.example` | Local `act` override template |
## Required GitHub Secrets [#required-github-secrets]
| Secret | Purpose | Where to Get |
| -------------- | --------------------------------------- | ----------------------------------------- |
| `NPM_TOKEN` | Publishing to npmjs.org | npmjs.com → Account → Access Tokens |
| `GITHUB_TOKEN` | GitHub Package Registry (auto-provided) | Automatically available in GitHub Actions |
The PyPI publish uses trusted publisher OIDC — no secret required, only the `pypi` GitHub Environment must be configured with your PyPI project.
## Release Cadence [#release-cadence]
Day to day:
1. Merge regular PRs to `main`.
2. Release Please updates the open release PR when releasable conventional commits land.
3. Review the release PR changelog and version bump.
4. Merge the release PR when ready to ship.
5. Watch `release.yml` publish PyPI, npm, GitHub Packages, Docker, and GitHub Release assets.
Runtime rollout answers one question: **which behaviors may this already-built
Headroom artifact expose in this process?** It is separate from the source and
distribution lifecycle, which decides which commit/artifact is qualified,
released, packaged, and published.
```bash
HEADROOM_ROLLOUT_CHANNEL=canary headroom proxy
```
This runs the installed artifact with canary-eligible runtime features available
according to that artifact's rollout policy. It does **not** install, select, or
run a canary release/version of Headroom.
## Channels and feature policy [#channels-and-feature-policy]
Channels are ordered `stable < beta < canary < dev`.
| Channel | Purpose |
| -------- | -------------------------------------------------------------------- |
| `stable` | Default; behavior eligible for normal production use. |
| `beta` | Opt-in behavior backed by automated and limited production evidence. |
| `canary` | Early dogfood behavior still gathering evidence. |
| `dev` | Local development and maintainer experiments. |
Availability and default enablement are separate registry fields. A feature can
be available in `canary` but remain off until explicitly requested; another can
be available and default-enabled in `stable`.
Request a named feature:
```bash
HEADROOM_ROLLOUT_CHANNEL=canary \
HEADROOM_FEATURES=tool_result_interceptors \
headroom proxy --intercept-tool-results
```
Force it off with the kill switch:
```bash
HEADROOM_DISABLE_FEATURES=tool_result_interceptors headroom proxy
```
## Resolution and precedence [#resolution-and-precedence]
CLI arguments, environment variables, and typed configuration are resolved once
at configuration construction. The immutable snapshot is injected into the
proxy and transform pipelines; changing the process environment afterward does
not alter a running proxy.
The existing loopback-only `/admin/runtime-env` endpoint is one narrow
exception: hot-reloading the legacy `HEADROOM_OUTPUT_SHAPER` alias replaces the
proxy's immutable snapshot with a newly resolved snapshot. Channel bounds and
`HEADROOM_DISABLE_FEATURES` still win, and `/stats.rollout` changes with the
effective running decision. Because these overrides are process-local, the
endpoint rejects updates when the built-in server uses multiple workers; restart
the proxy with the desired environment instead. Ambient environment mutation
remains ignored.
Precedence is deterministic:
| Condition | Result |
| ---------------------------------------------------------------- | ------------------------------------------------------------------------- |
| Explicit disable | Off, even if defaulted, requested, aliased, or unsafe override is active. |
| Requested below its availability channel, unsafe override active | On with `unsafe_override`. |
| Requested below its availability channel | Off with `blocked_by_channel`. |
| Explicit request in an allowed channel | On with `explicit`. |
| Enabled legacy alias in an allowed channel | On with `legacy_alias`. |
| Default-enabled in the active channel | On with `default`. |
| Otherwise | Off with `not_requested`. |
Legacy feature-specific variables are narrow compatibility aliases only. They
obey channel bounds and explicit disable precedence.
## Unsafe override and invalid input [#unsafe-override-and-invalid-input]
`HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES=1` is a break-glass mechanism. It can
cross a channel boundary for a requested feature, but cannot beat an explicit
disable. The runtime remains usable for debugging and emergency reproduction,
while its snapshot reports:
```json
{
"unsafe_override": true,
"qualification_eligible": false,
"qualification_ineligible_reason": "unsafe_rollout_override_active"
}
```
The Python resolver logs a warning and falls back to `stable` for an unknown
channel; unknown feature names are warned and ignored (fail-closed). Explicit
Python diagnostics (`headroom rollout status`) and the Rust front proxy's typed
CLI/environment parser reject unknown channels/features and list valid values
before startup.
## Machine-readable status and provenance [#machine-readable-status-and-provenance]
Inspect a supplied configuration without starting the proxy:
```bash
headroom rollout status --json
```
Inspect the actual running process through the supported black-box endpoint:
```bash
curl http://127.0.0.1:8787/stats
```
The Python proxy publishes the object at `/stats.rollout`. The Rust front proxy,
when deployed, publishes its own effective snapshot at `/rollout/status`; this
keeps each process's distinct feature registry and decisions independently
observable.
The `/stats.rollout` object and CLI output contain no secrets. They include:
```json
{
"schema_version": 1,
"policy_version": "1",
"channel": "stable",
"unsafe_override": false,
"registry_digest": "sha256:...",
"snapshot_digest": "sha256:...",
"qualification_eligible": true,
"features": [
{
"name": "tool_result_interceptors",
"available_in": "canary",
"default_enabled_in": null,
"requested": false,
"disabled": false,
"enabled": false,
"decision": "not_requested"
}
]
}
```
`schema_version` versions the external JSON contract. `policy_version` versions
the rollout rules. `registry_digest` is SHA-256 over canonical, ordered feature
definitions. `snapshot_digest` identifies the complete effective runtime state.
Equivalent policies/configurations produce equal digests; material policy or
decision changes do not.
These identities deliberately remain separate from source SHA, artifact SHA-256,
runtime payload SHA-256, and future qualification-policy identities. An external
benchmark can compare `/stats.rollout.registry_digest` and `snapshot_digest`
between A1 passthrough and B Headroom arms without importing Headroom internals.
A mismatch makes the future experiment invalid; benchmark logic itself is out of
scope for runtime rollout.
## Evidence-backed graduation and rollback [#evidence-backed-graduation-and-rollback]
Features progress from canary through beta toward stable only with linked
deterministic, integration, and benchmark evidence. **Bake time is evidence, not
qualification by itself.** Stable eligibility is followed by release
qualification before behavior becomes a stable default.
Every rollout-managed behavior must have a fast disable path. Operational
rollback uses `HEADROOM_DISABLE_FEATURES`; source rollback reverts the defining
change. The unsafe override is for diagnostics, not promotion or passing release
evidence.
Contributors should add named registry entries and tests for default behavior,
explicit request, channel blocking, disable precedence, unsafe behavior,
decision reasons, and provenance rather than reading rollout variables inside
implementation components. Python and Rust registries contain features relevant
to their own runtimes, but share channel ordering, precedence, decision reasons,
fail-closed invalid-input semantics, and deterministic identity semantics.
`headroom savings` shows how much Headroom has saved you over time — cost avoided, token counts, and breakdowns by model and client. Unlike `headroom_stats` (a single in-memory session snapshot), it reads a **durable ledger** that survives proxy and agent restarts.
## Usage [#usage]
```bash
headroom savings # human-readable summary
headroom savings --json # machine-readable report
headroom savings --days 7 # restrict the lookback window (1-30, default 30)
headroom savings --reset # delete the ledger and start fresh
```
### Example [#example]
```text
Today ███████████░░░░░ 67.9% saved 19,000 / 28,000 tokens $0.0850
Last 7 days ███████████░░░░░ 67.1% saved 47,000 / 70,000 tokens $0.2250
Last 30 days ██████████░░░░░░ 65.0% saved 78,000 / 120,000 tokens $0.2680
Cost avoided per model:
claude-opus-4-8 $0.1750
gpt-5.5 $0.0350
unknown $0.0330
claude-haiku-4-5 $0.0250
Savings by client:
claude-code 4 calls · 60,000 tokens saved
codex 2 calls · 18,000 tokens saved
```
## How it works [#how-it-works]
Every compression appends one line to an **append-only, file-locked event ledger** at `~/.headroom/savings_events.jsonl`, and `headroom savings` aggregates it on read. This design is:
* **Durable** — the ledger is on disk, so totals survive proxy and agent restarts.
* **Accurate under concurrency** — Headroom's MCP server runs as multiple processes (the main agent plus each subagent), and the proxy is a separate process. An append-only, locked log lets every writer contribute without the lost-update races a single shared mutable file would suffer.
* **Self-pruning** — events older than the retention window (30 days, which is also the hard maximum for `--days`) are dropped on read, and the file is compacted once it grows large.
Both compression paths feed the same ledger:
* **MCP tool** — each `headroom_compress` call records its client (the MCP client name) and tokens saved.
* **Proxy** — each request records its real upstream model, so cost is priced accurately.
### Cost basis [#cost-basis]
Cost avoided is the dollar value of the saved **input** tokens. Headroom uses [litellm](https://docs.headroomlabs.ai/docs/litellm) list pricing where the model is known (proxy traffic). MCP-tool compressions don't know the agent's upstream model, so they record `model="unknown"` and fall back to a blended per-token rate rather than reporting `$0`.
## Configuration [#configuration]
| Variable | Purpose |
| ------------------------------ | --------------------------------------------------------------------------------------------------------- |
| `HEADROOM_SAVINGS_EVENTS_PATH` | Override the ledger location (default `~/.headroom/savings_events.jsonl`). |
| `HEADROOM_MCP_CLIENT` | Override the client label recorded by the MCP tool path. |
| `HEADROOM_MCP_MODEL` | Optional model hint so MCP-tool compressions price against a known model instead of the blended fallback. |
`headroom savings` is distinct from `headroom_stats` (a per-session, in-memory snapshot) and from the proxy's live `/stats` endpoint (backed by `proxy_savings.json`). The savings ledger is the durable, cross-process source of truth.
When agents hand off to each other, context gets replayed in full. SharedContext compresses what moves between agents using Headroom's compression pipeline, typically saving **\~80% of tokens** on agent handoffs.
## Quick Start [#quick-start]
```ts twoslash
import { SharedContext } from "headroom-ai";
const ctx = new SharedContext();
// Agent A stores large output
const entry = await ctx.put("research", bigResearchOutput, {
agent: "researcher",
});
// Agent B gets compressed version (~80% smaller)
const summary = ctx.get("research");
// Agent B needs full details on demand
const full = ctx.get("research", { full: true });
```
```python
from headroom import SharedContext
ctx = SharedContext()
# Agent A stores large output
ctx.put("research", big_research_output, agent="researcher")
# Agent B gets compressed version (~80% smaller)
summary = ctx.get("research")
# Agent B needs full details on demand
full = ctx.get("research", full=True)
```
## API [#api]
### `put(key, content, agent?)` [#putkey-content-agent]
Store content under a key. Compresses automatically using Headroom's full pipeline (SmartCrusher for JSON, CodeCompressor for code, Kompress for text).
```ts twoslash
import { SharedContext } from "headroom-ai";
const ctx = new SharedContext();
// ---cut---
const entry = await ctx.put("findings", bigJsonOutput, {
agent: "researcher",
});
entry.originalTokens; // 20000
entry.compressedTokens; // 4000
entry.savingsPercent; // 80.0
entry.transforms; // ["router:json:0.20"]
```
```python
entry = ctx.put("findings", big_json_output, agent="researcher")
entry.original_tokens # 20,000
entry.compressed_tokens # 4,000
entry.savings_percent # 80.0
entry.transforms # ["router:json:0.20"]
```
### `get(key, full?)` [#getkey-full]
Retrieve content. Returns the compressed version by default, or the original with `full=True`.
```ts twoslash
import { SharedContext } from "headroom-ai";
const ctx = new SharedContext();
// ---cut---
const compressed = ctx.get("findings"); // 4K tokens
const original = ctx.get("findings", { full: true }); // 20K tokens
const missing = ctx.get("nonexistent"); // null
```
```python
compressed = ctx.get("findings") # 4K tokens
original = ctx.get("findings", full=True) # 20K tokens
missing = ctx.get("nonexistent") # None
```
### `stats()` [#stats]
Aggregated statistics across all entries.
```ts twoslash
import { SharedContext } from "headroom-ai";
const ctx = new SharedContext();
// ---cut---
const stats = ctx.stats();
stats.entries; // 3
stats.totalOriginalTokens; // 60000
stats.totalCompressedTokens; // 12000
stats.totalTokensSaved; // 48000
stats.savingsPercent; // 80.0
```
```python
stats = ctx.stats()
stats.entries # 3
stats.total_original_tokens # 60000
stats.total_compressed_tokens # 12000
stats.total_tokens_saved # 48000
stats.savings_percent # 80.0
```
### `keys()` and `clear()` [#keys-and-clear]
`keys()` lists all non-expired keys. `clear()` removes all entries.
## Configuration [#configuration]
```ts twoslash
import { SharedContext } from "headroom-ai";
// ---cut---
const ctx = new SharedContext({
model: "claude-sonnet-4-5-20250929", // For token counting
ttl: 3600, // 1 hour (default)
maxEntries: 100, // Evicts oldest when full
});
```
```python
ctx = SharedContext(
model="claude-sonnet-4-5-20250929", # For token counting
ttl=3600, # 1 hour (default)
max_entries=100, # Evicts oldest when full
)
```
Entries expire after `ttl` seconds. When `maxEntries` is reached, the oldest entry is evicted.
## Framework Examples [#framework-examples]
SharedContext is framework-agnostic. It works anywhere context moves between agents.
### CrewAI [#crewai]
```python
from headroom import SharedContext
ctx = SharedContext()
# After researcher task completes
ctx.put("findings", researcher_task.output.raw)
# Coder task gets compressed context
coder_context = ctx.get("findings")
```
### LangGraph [#langgraph]
```python
from headroom import SharedContext
ctx = SharedContext()
def researcher_node(state):
result = do_research()
ctx.put("research", result)
return {"research_summary": ctx.get("research")}
def coder_node(state):
# Compressed summary in state, full details on demand
full = ctx.get("research", full=True)
return {"code": write_code(full)}
```
### OpenAI Agents SDK [#openai-agents-sdk]
```python
from headroom import SharedContext
ctx = SharedContext()
def compress_handoff(messages):
for msg in messages:
if len(msg.content) > 1000:
ctx.put(msg.id, msg.content)
msg.content = ctx.get(msg.id)
return messages
handoff(agent=coder, input_filter=compress_handoff)
```
## How It Works [#how-it-works]
Under the hood, `put()` calls `headroom.compress()` -- the same pipeline used by the Headroom proxy -- and stores the original in memory. `get()` returns the compressed version. `get(full=True)` returns the original.
The compression pipeline routes content to the best compressor:
* **JSON arrays** -- SmartCrusher (70-95% compression)
* **Code** -- CodeCompressor (AST-aware)
* **Text** -- Kompress (ModernBERT-based) or passthrough
Simulation mode lets you preview what Headroom would do to your messages without sending them to an LLM. This is useful for cost estimation, debugging compression behavior, and understanding where token waste comes from.
## Basic Usage [#basic-usage]
```ts twoslash
import { compress } from 'headroom-ai';
// compress() returns the same result structure —
// use it without sending to your LLM to simulate
const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Would save: ${result.tokensSaved} tokens`);
console.log(`Compression ratio: ${(result.compressionRatio * 100).toFixed(1)}%`);
console.log(`Transforms: ${result.transformsApplied.join(', ')}`);
```
```python
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=large_conversation,
)
print(f"Tokens before: {plan.tokens_before}")
print(f"Tokens after: {plan.tokens_after}")
print(f"Would save: {plan.tokens_saved} tokens ({plan.tokens_saved/plan.tokens_before*100:.1f}%)")
print(f"Transforms: {plan.transforms}")
```
## Waste Signals [#waste-signals]
Simulation reports where token waste comes from in your messages:
```python
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=messages,
)
waste = plan.waste_signals # dict[str, int]
print(f"JSON bloat: {waste['json_bloat']} tokens")
print(f"HTML noise: {waste['html_noise']} tokens")
print(f"Whitespace: {waste['whitespace']} tokens")
print(f"Dynamic dates: {waste['dynamic_date']} tokens")
print(f"Repetition: {waste['repetition']} tokens")
```
Waste signals help you understand which parts of your input are contributing the most unnecessary tokens.
## Block Breakdown [#block-breakdown]
The parser breaks your conversation into blocks so you can see where tokens are concentrated:
```python
# Block types: system, user, assistant, tool_call, tool_result, rag
# The breakdown shows token counts per block type
```
| Block Kind | Description |
| ------------- | ------------------------------------- |
| `system` | System prompt instructions |
| `user` | User messages |
| `assistant` | Model responses |
| `tool_call` | Function call requests |
| `tool_result` | Tool output (largest source of waste) |
| `rag` | Retrieved document context |
## Use Cases [#use-cases]
### Cost Estimation [#cost-estimation]
Run simulation on a representative sample of your workload to estimate savings before enabling `optimize` mode:
```python
import json
total_before = 0
total_after = 0
for messages in sample_conversations:
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=messages,
)
total_before += plan.tokens_before
total_after += plan.tokens_after
savings_pct = (1 - total_after / total_before) * 100
print(f"Estimated savings: {savings_pct:.1f}%")
print(f"Tokens saved: {total_before - total_after:,}")
```
### Debugging Compression [#debugging-compression]
Use simulation to understand why a particular conversation is or is not being compressed:
```python
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=messages,
)
if plan.tokens_saved == 0:
print("No compression applied. Possible reasons:")
print("- Messages are too short (< 200 tokens per tool output)")
print("- No tool outputs with compressible JSON arrays")
print("- Content is already compact (code, grep results)")
else:
print(f"Transforms applied: {plan.transforms}")
# See the optimized messages
print(json.dumps(plan.messages_optimized, indent=2))
```
### Comparing Configurations [#comparing-configurations]
Test different configurations to find the best settings for your workload:
```python
from headroom import HeadroomClient, HeadroomConfig, OpenAIProvider
from headroom.transforms import SmartCrusherConfig
configs = [
SmartCrusherConfig(max_items_after_crush=10),
SmartCrusherConfig(max_items_after_crush=25),
SmartCrusherConfig(max_items_after_crush=50),
]
for smart_crusher_config in configs:
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
config=HeadroomConfig(smart_crusher=smart_crusher_config),
)
plan = client.chat.completions.simulate(model="gpt-4o", messages=messages)
print(f"max_items={smart_crusher_config.max_items_after_crush}: "
f"{plan.tokens_saved} tokens saved ({plan.tokens_saved/plan.tokens_before*100:.1f}%)")
```
Simulation never calls the LLM API. It runs the full transform pipeline locally and returns the results, so there is no cost and no latency from the provider.
SmartCrusher is Headroom's compressor for JSON tool outputs. This is the compressor that fires automatically when ContentRouter detects JSON arrays.
By default SmartCrusher tries a **lossless** compaction first: if the array is cleanly tabular (a consistent set of keys across items), it re-encodes every item into a compact `csv-schema` form and keeps it *if that saves at least 30% of the bytes* -- no items are dropped, they're just represented more compactly. Only when that lossless fold doesn't clear the 30% bar (or the array isn't cleanly tabular) does SmartCrusher fall back to the scored keep/drop behavior described below. See `headroom/transforms/smart_crusher.py:415-419`. Set `with_compaction=False` on `SmartCrusher(...)` to force the pre-lossless keep/drop-only path (this is also what Headroom's retention-property tests use).
## How It Works [#how-it-works]
When the lossy keep/drop path runs, SmartCrusher doesn't blindly truncate arrays -- it scores each item across five dimensions:
1. **First/Last items** -- Context for pagination and recency
2. **Error items** -- 100% preservation of error states (never dropped)
3. **Anomalies** -- Statistical outliers (> 2 standard deviations from the mean)
4. **Relevant items** -- Matches to the user's query via BM25/embeddings
5. **Change points** -- Significant transitions in data
On that path, the target size is `max_items_after_crush` (15 by default): a 1,000-item array becomes \~15 items with all the information the LLM actually needs (measured: a 1,000-item array with `with_compaction=False` compressed to 16 items).
## What Gets Preserved [#what-gets-preserved]
| Category | Preserved | Why |
| --------- | --------- | -------------------------- |
| Errors | 100% | Critical for debugging |
| First N | 100% | Context and pagination |
| Last N | 100% | Recency |
| Anomalies | All | Unusual values matter |
| Relevant | Top K | Match user's query |
| Others | Sampled | Statistical representation |
## Quick Start [#quick-start]
```ts twoslash
import { compress } from "headroom-ai";
// SmartCrusher fires automatically for JSON tool outputs
const messages = [
{ role: "system" as const, content: "You are a helpful assistant." },
{ role: "user" as const, content: "Find errors in the last 24 hours" },
{
role: "tool" as const,
content: JSON.stringify({ results: new Array(1000).fill({ status: "ok" }) }),
tool_call_id: "call_1",
},
];
const result = await compress(messages);
console.log(`Tokens saved: ${result.tokensSaved}`);
// SmartCrusher keeps errors, anomalies, and relevant items
```
```python
import json
from headroom import SmartCrusher
crusher = SmartCrusher()
# crush() takes a JSON string, not a dict
tool_output = json.dumps({"results": ["...1000 items..."]})
# Returns a CrushResult; the compressed JSON string is `.compressed`
result = crusher.crush(tool_output, query="user's question")
print(result.compressed)
print(result.strategy) # e.g. "lossless:table(...)" or "smart_sample(1000->15)"
```
## Configuration [#configuration]
```ts twoslash
import { compress } from "headroom-ai";
// Configure via the Headroom proxy or HeadroomClient
const result = await compress(messages, {
model: "gpt-4o",
tokenBudget: 10000, // SmartCrusher will reduce JSON to fit
});
console.log(`Transforms: ${result.transformsApplied}`);
// ["smart_crusher", "cache_aligner"]
```
```python
from headroom import SmartCrusher, SmartCrusherConfig
config = SmartCrusherConfig(
min_tokens_to_crush=200, # Only compress if > 200 tokens
max_items_after_crush=15, # Keep at most 15 items (lossy path only)
first_fraction=0.3, # Keep first 30% of items
last_fraction=0.15, # Keep last 15% of items
variance_threshold=2.0, # Statistical variance threshold
preserve_change_points=True, # Keep significant transitions
)
crusher = SmartCrusher(config)
result = crusher.crush(tool_output, query="find payment failures")
```
## Configuration Options [#configuration-options]
| Option | Default | Description |
| ------------------------ | ------- | ---------------------------------------------------- |
| `min_tokens_to_crush` | `200` | Only compress arrays with more than this many tokens |
| `min_items_to_analyze` | `5` | Minimum items before analyzing for compression |
| `max_items_after_crush` | `15` | Maximum items to keep after compression |
| `variance_threshold` | `2.0` | Statistical variance threshold for analysis |
| `uniqueness_threshold` | `0.1` | Uniqueness threshold for deduplication |
| `similarity_threshold` | `0.8` | Similarity threshold for grouping |
| `preserve_change_points` | `True` | Preserve significant transitions in data |
| `first_fraction` | `0.3` | Fraction of items always kept from the start |
| `last_fraction` | `0.15` | Fraction of items always kept from the end |
| `dedup_identical_items` | `True` | Deduplicate identical items |
| `use_feedback_hints` | `True` | Use TOIN feedback hints for scoring |
## Example: Before and After [#example-before-and-after]
Consider a tool that returns 1,000 search results:
```python
# Before compression: 45,000 tokens
{
"results": [
{"id": 1, "status": "ok", "message": "Success", "timestamp": "..."},
{"id": 2, "status": "ok", "message": "Success", "timestamp": "..."},
# ... 995 more "ok" results ...
{"id": 998, "status": "error", "message": "Connection timeout", "timestamp": "..."},
{"id": 999, "status": "ok", "message": "Success", "timestamp": "..."},
{"id": 1000, "status": "ok", "message": "Success", "timestamp": "..."},
]
}
# After SmartCrusher's lossy keep/drop path (with_compaction=False):
# ~15-16 items kept -- first items, last items, and the error at id=998
```
The LLM sees the structure, the error, and a representative sample -- everything it needs to answer "find errors in the last 24 hours" without wading through 1,000 identical success responses. Note this specific shape (mostly-identical items, one error) is also exactly what the lossless table fold favors (see the callout above), so with default settings this array is more likely to be re-encoded compactly with every item still present than reduced to \~15 items -- the numbers above illustrate the scored keep/drop path, not a guarantee of which path a given array takes.
You don't need to call SmartCrusher directly. The ContentRouter detects JSON arrays and routes them to SmartCrusher automatically. Direct usage is available when you want fine-grained control over the configuration.
Headroom integrates with [Strands Agents](https://github.com/strands-agents/sdk-python) through two patterns: wrap the model for full conversation compression, or hook into tool calls for targeted tool output compression.
## Installation [#installation]
```bash
pip install headroom-ai strands-agents
```
## Quick start [#quick-start]
```python
from strands import Agent
from strands.models.bedrock import BedrockModel
from headroom.integrations.strands import HeadroomStrandsModel
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
optimized = HeadroomStrandsModel(wrapped_model=model)
agent = Agent(model=optimized)
response = agent("Investigate the production incident")
print(f"Tokens saved: {optimized.total_tokens_saved}")
```
## Model wrapping [#model-wrapping]
Wraps the Strands `Model` interface. Every call to `stream()` compresses messages before they reach the provider:
```python
from headroom import HeadroomConfig
from headroom.integrations.strands import HeadroomStrandsModel
optimized = HeadroomStrandsModel(
wrapped_model=model,
config=HeadroomConfig(),
)
agent = Agent(model=optimized)
response = agent("Analyze these logs")
```
## Hook provider (tool output compression) [#hook-provider-tool-output-compression]
Compresses tool call results via Strands' hook system. Uses SmartCrusher on JSON arrays returned by tools:
```python
from strands import Agent
from strands.models.bedrock import BedrockModel
from headroom.integrations.strands import HeadroomHookProvider
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
hooks = HeadroomHookProvider(
compress_tool_outputs=True,
min_tokens_to_compress=200,
preserve_errors=True,
)
agent = Agent(model=model, hooks=[hooks])
response = agent("Search the database for recent failures")
print(f"Tokens saved by hooks: {hooks.total_tokens_saved}")
```
The hook preserves error items, anomalous values (statistical outliers), items matching the query context, and boundary items (first/last).
## Both together [#both-together]
Model wrapping compresses conversation history. Hooks compress individual tool results. Use both for maximum savings:
```python
from headroom.integrations.strands import HeadroomStrandsModel, HeadroomHookProvider
optimized = HeadroomStrandsModel(wrapped_model=model)
hooks = HeadroomHookProvider(compress_tool_outputs=True)
agent = Agent(model=optimized, hooks=[hooks])
```
## How it works [#how-it-works]
```
Agent decides to call tool
|
v
Tool executes, returns result
|
v
HeadroomHookProvider (optional)
compresses tool result JSON
|
v
Agent builds next API request
|
v
HeadroomStrandsModel.stream()
compresses full message list
|
v
Provider API (Bedrock, etc.)
```
The model wrapper uses the full Headroom pipeline (CacheAligner, ContentRouter). The hook provider uses SmartCrusher directly for fast JSON compression.
## Structured output [#structured-output]
`structured_output()` is an async generator (it delegates to the wrapped model's own `structured_output()`), so it must be consumed with `async for`:
```python
from pydantic import BaseModel
class Analysis(BaseModel):
severity: str
root_cause: str
recommendation: str
async for event in optimized.structured_output(Analysis, messages):
result = event
```
## Metrics [#metrics]
```python
for m in optimized.metrics_history:
print(f" {m.tokens_before} -> {m.tokens_after} ({m.tokens_saved} saved)")
print(f"Total saved: {optimized.total_tokens_saved}")
```
## Supported providers [#supported-providers]
| Strands Model | Provider Detected |
| -------------- | ------------------------ |
| `BedrockModel` | Anthropic (via Bedrock) |
| `OllamaModel` | OpenAI-compatible |
| Custom `Model` | Falls back to estimation |
Headroom provides specialized compressors for text-based content that isn't JSON or source code. Each one understands the structure of its content type and preserves what the LLM needs while dropping the noise.
| Compressor | Input Type | What It Preserves | Typical Savings |
| -------------------- | --------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SearchCompressor` | grep/ripgrep output | Relevant matches, file diversity | 80-95% (measured \~91% on a synthetic 40-file corpus) |
| `LogCompressor` | Build/test logs | Errors, stack traces, summaries | 85-95% (measured \~93% on a synthetic 1,000-line log) |
| `DiffCompressor` | Unified diffs | Changed lines, context | 60-80% (measured \~70% on a multi-file diff) |
| `TextCrusher` | General text | Relevant sentences, anchors | 30-60% (its `target_ratio` defaults to 0.5; measured \~50% on real docs prose) |
| `KompressCompressor` | General text fallback | Learned token scoring via ONNX | Highly variable -- `target_ratio` defaults to `None` ("model decides"), which measured only 6-24% on real prose; an explicit `target_ratio=0.5` measured \~40%. There is no fixed default range; see `benchmarks/text_crusher_quality_eval.py`. |
## SearchCompressor [#searchcompressor]
Compresses search results (grep, ripgrep, ag) while keeping the matches that matter.
```python
from headroom.transforms import SearchCompressor
search_results = """
src/utils.py:42:def process_data(items):
src/utils.py:43: \"\"\"Process items.\"\"\"
src/models.py:15:class DataProcessor:
src/models.py:89: def process(self, items):
... hundreds more matches ...
"""
compressor = SearchCompressor()
result = compressor.compress(search_results, context="find process")
print(f"Compressed {result.original_match_count} matches to {result.compressed_match_count}")
print(result.compressed)
```
**What gets preserved:**
* Exact query matches (lines containing the search term)
* High-relevance matches (scored by BM25 similarity)
* File diversity (results from different files are kept)
* First/last matches (context from start and end)
### Configuration [#configuration]
```python
from headroom.transforms import SearchCompressor, SearchCompressorConfig
config = SearchCompressorConfig(
max_total_matches=30, # Cap total matches kept across all files
max_matches_per_file=5, # Cap matches kept per file (diversity)
max_files=15, # Cap number of distinct files kept
boost_errors=True, # Prioritize lines that look like errors
context_keywords=["auth"], # Extra terms to bias selection toward
)
compressor = SearchCompressor(config)
```
## LogCompressor [#logcompressor]
Compresses build and test output while preserving errors, warnings, and summaries.
```python
from headroom.transforms import LogCompressor
build_output = """
===== test session starts =====
collected 500 items
tests/test_foo.py::test_1 PASSED
... hundreds of passed tests ...
tests/test_bar.py::test_fail FAILED
AssertionError: expected 5, got 3
===== 1 failed, 499 passed =====
"""
compressor = LogCompressor()
result = compressor.compress(build_output)
print(result.compressed)
print(f"Compression ratio: {result.compression_ratio:.1%}")
```
**What gets preserved:**
* Errors and failures (any line with ERROR, FAILED, Exception)
* Warnings
* Full stack traces for debugging
* Test/build summary lines
* Section headers (structural markers like `=====`)
**What gets dropped:**
* Hundreds of `PASSED` lines
* Verbose success output
* Repeated patterns
## DiffCompressor [#diffcompressor]
Compresses unified diffs while keeping the actual changes and enough context to understand them.
```python
from headroom.transforms import DiffCompressor
diff_output = """
diff --git a/src/main.py b/src/main.py
--- a/src/main.py
+++ b/src/main.py
@@ -42,7 +42,7 @@
def process(items):
- return [x for x in items]
+ return [x.strip() for x in items if x]
"""
compressor = DiffCompressor()
result = compressor.compress(diff_output)
```
## TextCrusher [#textcrusher]
Extractive prose compression -- it keeps the most relevant input sentences verbatim (selection, not rewriting). Best for documentation, README files, and prose content. `TextCrusher` lives in its own module rather than the `headroom.transforms` package root:
```python
from headroom.transforms.text_crusher import TextCrusher
long_text = """
... thousands of lines of documentation ...
"""
compressor = TextCrusher()
result = compressor.compress(long_text, context="authentication")
print(result.compressed)
print(f"{result.original_tokens} -> {result.compressed_tokens} tokens")
```
**What gets preserved:**
* Paragraphs relevant to the context query
* Headers and section markers
* Document structure and organization
## Kompress [#kompress]
```python
from headroom.transforms.kompress_compressor import KompressCompressor
compressor = KompressCompressor()
result = compressor.compress(long_output)
print(f"Before: {result.original_tokens} tokens")
print(f"After: {result.compressed_tokens} tokens")
print(f"Saved: {result.savings_percentage:.1f}%")
```
The old LLMLingua transform and helper functions are no longer exported. Use Kompress and ContentRouter for text compression.
## Content Type Detection [#content-type-detection]
If you're building your own routing logic, you can use the content type detector directly:
```python
from headroom.transforms import detect_content_type, ContentType, SearchCompressor, LogCompressor
# A single file:line:content line isn't enough signal -- the detector wants
# several matching lines before it calls something SEARCH_RESULTS.
content = "src/main.py:42:def process():\nsrc/utils.py:10:def helper():\nsrc/models.py:5:class Foo:"
detection = detect_content_type(content)
if detection.content_type == ContentType.SEARCH_RESULTS:
result = SearchCompressor().compress(content, context="process")
elif detection.content_type == ContentType.BUILD_OUTPUT:
result = LogCompressor().compress(content)
elif detection.content_type == ContentType.PLAIN_TEXT:
from headroom.transforms.text_crusher import TextCrusher
result = TextCrusher().compress(content, context="process")
```
## When Each Compressor Is Used [#when-each-compressor-is-used]
The ContentRouter selects the right compressor automatically. Here's when each fires:
| Content Pattern | Compressor | Detection Signal |
| ------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `file:line:content` lines | SearchCompressor | grep/ripgrep output format |
| pytest, npm, cargo markers | LogCompressor | Build tool output patterns |
| `---/+++` and `@@` markers | DiffCompressor | Unified diff format |
| Prose, documentation | KompressCompressor | ContentRouter's default for `PLAIN_TEXT` (`CompressionStrategy.TEXT` and `KOMPRESS` share the same ML compressor dispatch -- see `headroom/transforms/content_router.py:3524-3540`) |
| Long plain text that exceeds Kompress's size gate | TextCrusher | Request-path-safe fallback when the content is too large for synchronous ONNX inference (`_kompress_max_tokens`); TextCrusher is not ContentRouter's default path for ordinary prose |
## Performance [#performance]
Speed depends heavily on input size and shape; the figures below are one
measurement each (warm process, this package version), not a guarantee:
| Compressor | Measured Input | Measured Output | Measured Speed (warm) |
| ------------------ | -------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------- |
| SearchCompressor | 1,000 matches across 60 files | 30 matches | \~2.0ms |
| LogCompressor | 5,000 lines, \~12 errors | 67 lines | \~17ms |
| DiffCompressor | 30-file / 90-hunk synthetic diff | changed hunks only | \~0.4ms |
| TextCrusher | 10,000 chars of prose | \~50% of input tokens (`target_ratio` default) | \~0.7ms |
| KompressCompressor | Plain text | Highly variable (see the ranges above) | model-dependent; first call after process start pays a one-time model-load cost |
Solutions for common Headroom issues.
## Proxy Server Issues [#proxy-server-issues]
### Proxy will not start [#proxy-will-not-start]
**Symptom**: `headroom proxy` fails or hangs.
```bash
# Check if port is already in use
lsof -i :8787
# Try a different port
headroom proxy --port 8788
# Check for missing dependencies
pip install "headroom-ai[proxy]"
# Run with debug logging
headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages
```
### Connection refused when calling proxy [#connection-refused-when-calling-proxy]
**Symptom**: `curl: (7) Failed to connect to localhost port 8787`
```bash
# Verify proxy is running
curl http://localhost:8787/health
# Check if proxy started on a different port
ps aux | grep headroom
```
### Proxy returns errors for some requests [#proxy-returns-errors-for-some-requests]
**Symptom**: Some requests work, others fail with 502/503.
```bash
# Check proxy logs for the actual error
headroom proxy --log-file ~/.headroom/logs/proxy.jsonl --log-messages
# Verify API key is set
echo $OPENAI_API_KEY # or ANTHROPIC_API_KEY
# Test the underlying API directly
curl https://api.openai.com/v1/models \
-H "Authorization: Bearer $OPENAI_API_KEY"
```
### Windows: ML content detection hangs or silently falls back [#windows-ml-content-detection-hangs-or-silently-falls-back]
**Symptom**: On Windows 11 24H2+, every proxied request stalls (historically
`Optimization failed: TimeoutError`), or the first detection in each process
burns \~5 seconds and compression quality drops because detection runs on the
non-ML fallback tiers. The proxy log may show
`magika ONNX session init timed out`.
**Cause**: The Rust core loads ONNX Runtime dynamically. Without
`ORT_DYLIB_PATH`, the bare Windows DLL search resolves `onnxruntime.dll` to
`C:\Windows\System32\onnxruntime.dll` — the Windows ML OS component (1.17.x),
which deadlocks ort session initialization instead of returning an error.
**Fix**: Headroom pins `ORT_DYLIB_PATH` automatically at import time to the
DLL inside the `onnxruntime` pip package (included in `headroom-ai[proxy]`).
Confirm in the startup log:
```
Pinned ORT_DYLIB_PATH to bundled ONNX Runtime: ...\onnxruntime\capi\onnxruntime.dll
```
If the pin is skipped (library install without `onnxruntime`), either install
it or point the variable at any modern ONNX Runtime yourself:
```powershell
pip install onnxruntime
# or
$env:ORT_DYLIB_PATH = "C:\path\to\onnxruntime.dll"
```
`HEADROOM_MAGIKA_INIT_TIMEOUT_SECS` (default `5`) bounds the init as a safety
net; on timeout detection degrades to non-ML tiers for the process lifetime.
## No Token Savings [#no-token-savings]
**Symptom**: `stats['session']['tokens_saved_total']` is 0.
**Diagnosis**:
```python
stats = client.get_stats()
print(f"Mode: {stats['config']['mode']}") # Should be "optimize"
print(f"SmartCrusher: {stats['transforms']['smart_crusher_enabled']}")
```
**Common causes**:
* Mode is `audit` (observation only, no modifications)
* Messages do not contain tool outputs
* Tool outputs are below the 200-token threshold
* Data is not compressible (high uniqueness, code, grep results)
**Solutions**:
```ts twoslash
import { compress } from 'headroom-ai';
// Ensure the proxy is running in optimize mode
// (default, unless --no-optimize was passed)
const result = await compress(messages, { model: 'gpt-4o' });
console.log(`Saved: ${result.tokensSaved} tokens`);
console.log(`Compressed: ${result.compressed}`);
```
```python
# 1. Ensure mode is "optimize"
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
default_mode="optimize", # NOT "audit"
)
# 2. Or override per-request
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
headroom_mode="optimize",
)
# 3. Lower the compression threshold
config = HeadroomConfig()
config.smart_crusher.min_tokens_to_crush = 100 # Default is 200
```
### Dashboard shows 0 compressed/saved tokens after upgrading to 0.31.0 [#dashboard-shows-0-compressedsaved-tokens-after-upgrading-to-0310]
**Symptom**: After upgrading from 0.27.0 to 0.31.0, the dashboard's compression / "Tokens Saved" figures read \~0, even though total token spend is the same or **lower** than before.
**Cause**: This is a default-mode change, not a regression. 0.31.0 ships the `coding` savings profile as the out-of-box default, and `coding` runs the proxy in **cache** mode (see [Savings profiles](/docs/proxy#savings-profiles)). Cache mode freezes the provider prefix and compresses only the newest turn *delta* — this deliberately avoids busting the provider's prompt cache. So the **compression** figure is small, and the savings shift to **cheaper prefix-cache reads** (cached input tokens are billed at a fraction of list price). On a short prompt there is little delta to compress, so the compression tile can read \~0 while your actual cost still drops.
**Where the savings show up**: Look at the **Prefix Cache Impact** panel and the **Compression vs Cache** tile on the dashboard — these reflect cache-read savings rather than per-request compression. The headline "Tokens Saved" tile only counts compression, so in cache mode it understates the real benefit.
**To get 0.27.0-style compression numbers back**: run the proxy in token mode, which prioritizes visible compression. Use this to compare or diagnose, not as a permanent setting — `cache` is the default because prefix-cache stability is usually worth more on a real agent workload than a larger headline compression number:
```bash
# Per run
headroom proxy --mode token
# Or pick a token-mode profile
HEADROOM_SAVINGS_PROFILE=balanced headroom proxy # ~70% target
HEADROOM_SAVINGS_PROFILE=agent-90 headroom proxy # ~90% target
```
Trade-off: token mode maximizes visible compression but rewrites prior turns, which can reduce provider prefix-cache hits — so raw compression goes up while cache-read savings go down. Cache mode is the default because, for long coding sessions, preserving the prefix cache usually wins overall.
## Claude Code context window is larger through the proxy [#claude-code-context-window-is-larger-through-the-proxy]
**Symptom**: After pointing Claude Code at Headroom (`ANTHROPIC_BASE_URL`), `/context all`
shows **more** tokens used than a direct session — the "System tools" and "MCP tools"
lines grow by tens of thousands of tokens, before you send any message.
**Cause**: Claude Code normally defers most tool schemas behind its server-side
**Tool Search Tool** (it sends only tool *names* and loads full schemas on demand).
It enables this only when it believes it is talking directly to `api.anthropic.com`.
The moment `ANTHROPIC_BASE_URL` is a custom host, Claude Code can't assume the endpoint
supports the feature, so it falls back to **eagerly** materializing every tool schema
into the local context window. This is a Claude Code client-side decision made before
the request reaches the proxy — no proxy header can reverse it.
**Solution**: set `ENABLE_TOOL_SEARCH` so Claude Code keeps deferring tools through the
proxy. The proxy forwards the `tool_reference` blocks correctly, so deferral works
end-to-end (both streaming and non-streaming, subscription and API-key auth).
```bash
# Easiest: `headroom wrap claude` sets ENABLE_TOOL_SEARCH=true automatically.
headroom wrap claude
# Choose the mode (true = always defer, the default; auto / auto:N = defer only
# when tool definitions exceed N% of the budget; false = off):
headroom wrap claude --tool-search auto
# Running `claude` manually instead of via wrap? Set it yourself:
ENABLE_TOOL_SEARCH=true ANTHROPIC_BASE_URL=http://localhost:8787 claude
```
**Verify (before / after)** with `/context all` in a fresh session, no messages sent:
| Section | Eager (no `ENABLE_TOOL_SEARCH`) | Deferred (`ENABLE_TOOL_SEARCH=true`) |
| ------------ | ------------------------------- | ------------------------------------ |
| System tools | fully materialized | deferred subset |
| MCP tools | every tool shows a token cost | `(loaded on-demand)`, 0 tokens |
When deferral is off, the proxy log also prints a one-time hint naming the fix.
See [issue #746](https://github.com/headroomlabs-ai/headroom/issues/746) for the full analysis.
Anthropic's VSCode extension webview does not currently render the deferred-tool
content blocks that `ENABLE_TOOL_SEARCH=true` enables through Headroom. Tool
results can show up as `unsupported content type` in the extension even though
the standalone `claude` CLI works correctly. If you use Claude Code inside
VSCode, set `ENABLE_TOOL_SEARCH=false` for that target and restart the Headroom
deployment. See [issue #2028](https://github.com/headroomlabs-ai/headroom/issues/2028).
## Remote Control unavailable through custom ANTHROPIC\_BASE\_URL [#remote-control-unavailable-through-custom-anthropic_base_url]
**Symptom**: When Claude Code runs with `ANTHROPIC_BASE_URL` set to a custom host (for example, Headroom), the Remote Control menu is absent.
**Cause**: This is a Claude-side gate. Headroom only receives normal API traffic and can still compress it, but Claude evaluates Remote Control availability before proxy traffic reaches the server.
**Fix**: Use Headroom for normal proxied API sessions, and launch Claude directly (without `ANTHROPIC_BASE_URL`) when you need Claude Remote Control.
`ENABLE_TOOL_SEARCH` is unaffected and can stay enabled for context-window savings while routing through Headroom.
## Server-managed settings unavailable through custom ANTHROPIC\_BASE\_URL [#server-managed-settings-unavailable-through-custom-anthropic_base_url]
**Symptom**: Settings pushed from **Admin Settings > Claude Code > Managed settings** in the claude.ai console (server-managed settings) don't apply to sessions running through Headroom, even though they apply fine without the proxy.
**Cause**: This is a Claude-side gate, not a Headroom limitation. Per Anthropic's docs, server-managed settings require a direct connection to `api.anthropic.com`; if `ANTHROPIC_BASE_URL` is set to any non-default host — which is exactly what wrapping via Headroom does — Claude Code skips the settings fetch entirely for that session. The request never reaches Headroom, so there is no endpoint for Headroom to implement or proxy.
This is separate from the OS-level `managed-settings.json` file (macOS `/Library/Application Support/ClaudeCode/`, Linux `/etc/claude-code/`, Windows `C:\Program Files\ClaudeCode\`): that file is read straight from local disk at startup and is unaffected by `ANTHROPIC_BASE_URL` or Headroom. If that file isn't taking effect, the cause is unrelated to proxying (path, permissions, or JSON syntax) — check `claude --debug-file ` and search the log for `Remote settings`.
**Fix**: None available on the Headroom side — this is an intentional Anthropic security boundary (a proxy in the path could otherwise forge org policy). If your org relies on server-managed settings, deploy the same policy as [endpoint-managed settings](https://code.claude.com/docs/en/settings#settings-files) (MDM profile, Windows registry, or a local `managed-settings.json`) instead, since those are read locally and unaffected by proxying.
See [Server-managed settings platform availability](https://code.claude.com/docs/en/server-managed-settings#platform-availability) and [issue #3074](https://github.com/headroomlabs-ai/headroom/issues/3074).
## Compression Too Aggressive [#compression-too-aggressive]
**Symptom**: LLM responses are missing information that was in tool outputs.
```python
# 1. Keep more items
config = HeadroomConfig()
config.smart_crusher.max_items_after_crush = 50 # Default: 15
# 2. Disable SmartCrusher entirely
config.smart_crusher.enabled = False
```
`client.chat.completions.create(..., headroom_tool_profiles={...})` is
accepted but not currently wired through `TransformPipeline.apply()` --
the kwarg is documented in the pipeline's own docstring but never read
in its body (`headroom/transforms/pipeline.py`), so passing it changes
nothing. There is also no `skip_compression` key on the per-tool
compression profile (`CompressionProfile` in `headroom/config.py` only
has `bias`, `min_k`, `max_k`) -- passing a plain dict there would raise
`AttributeError` if a code path did consume it. Use options 1 and 2
above instead.
## High Latency [#high-latency]
**Symptom**: Requests take longer than expected.
**Diagnosis**:
```python
import time
import logging
logging.basicConfig(level=logging.DEBUG)
start = time.time()
response = client.chat.completions.create(...)
print(f"Total time: {time.time() - start:.2f}s")
```
**Solutions**:
```python
# 1. Use BM25 instead of embeddings (faster)
config = HeadroomConfig()
config.smart_crusher.relevance.tier = "bm25"
# 2. Increase threshold to skip small payloads
config.smart_crusher.min_tokens_to_crush = 500
# 3. Disable transforms you don't need
config.cache_aligner.enabled = False
# Note: the old rolling-window "drop messages from history" stage was
# retired (Phase B PR-B1); `HeadroomConfig` has no `rolling_window` field
# any more (`headroom/config.py`), so there is nothing left to disable there.
```
## Installation Issues [#installation-issues]
### pipx installs an older Headroom version [#pipx-installs-an-older-headroom-version]
**Symptom**: PyPI shows a newer `headroom-ai` release, but `pipx install` or
`pipx upgrade` keeps an older version. A pinned install can also fail with
`No matching distribution found`.
**Cause**: `pipx` resolves packages inside its app virtual environment. If that
environment uses a Python version that Headroom does not publish wheels for yet,
pip may skip newer releases and choose the newest compatible build it can use.
Check the interpreter:
```bash
pipx list
```
Install with a supported Python explicitly:
```bash
pipx install --python python3.13 "headroom-ai[all]"
```
For a pinned release:
```bash
pipx install --python python3.13 "headroom-ai[all]==0.21.4"
```
If you already have Headroom installed under `pipx`, uninstall it first or
reinstall it with the supported interpreter.
### pip install fails with C++ compilation error [#pip-install-fails-with-c-compilation-error]
**Symptom**: `RuntimeError: Unsupported compiler -- at least C++11 support is needed!`
```bash
# Linux / Debian-based (including Docker)
apt-get install -y build-essential && pip install headroom-ai
# macOS (Xcode command line tools)
xcode-select --install && pip install headroom-ai
```
For Docker, install and remove build tools in one layer:
```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
&& pip install "headroom-ai[proxy]" \
&& apt-get purge -y build-essential && apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
```
### ModuleNotFoundError: No module named 'headroom' [#modulenotfounderror-no-module-named-headroom]
```bash
# Check it is installed in the right environment
pip show headroom-ai
# If using virtual environment, ensure it is activated
source venv/bin/activate
# Reinstall
pip install --upgrade headroom-ai
```
### Missing optional dependency [#missing-optional-dependency]
```bash
# For proxy server
pip install "headroom-ai[proxy]"
# For embedding-based relevance scoring
pip install "headroom-ai[relevance]"
# For code compression (tree-sitter)
pip install "headroom-ai[code]"
# For everything
pip install "headroom-ai[all]"
```
### Windows: Defender blocks ast-grep-cli (sg.exe) during install [#windows-defender-blocks-ast-grep-cli-sgexe-during-install]
**Symptom**: On Windows, `uv tool install "headroom-ai[all]"` (or `pip install`) fails while installing the `ast-grep-cli` wheel, and Windows Defender flags `sg.exe`:
```text
error: Failed to install: ast_grep_cli-0.44.1-py3-none-win_amd64.whl (ast-grep-cli==0.44.1)
Caused by: failed to open file ...\ast_grep_cli-0.44.1.data\scripts\sg.exe:
The operation did not complete successfully because the file contains a virus
or potentially unwanted software. (os error 225)
Threat: Trojan:Win64/Lazy!MTB
```
**Cause**: A known **false positive** in the upstream `ast-grep-cli` wheel's bundled `sg.exe` ([ast-grep/ast-grep#2799](https://github.com/ast-grep/ast-grep/issues/2799)), not a Headroom issue. `ast-grep` is a base dependency, so the block also affects the `[proxy]` extra. Headroom uses `ast-grep` only for optional AST-based Read-output outlining and **runs normally without it** — the only impact is the install-time quarantine.
**Workarounds** (safest first):
1. **Run the proxy in Docker** — no local wheel is installed, so Defender is never triggered. See [Docker install](/docs/docker-install); the image is `ghcr.io/headroomlabs-ai/headroom`.
2. **Restore the file from quarantine and retry** — open **Windows Security → Virus & threat protection → Protection history**, select the `sg.exe` detection, choose **Restore**, then re-run the install command. This changes no persistent settings.
3. **Add a temporary, scoped Defender exclusion during install** (last resort; requires an elevated PowerShell). Only do this if you accept excluding a *known false positive*, and remove the exclusion afterward:
```powershell
# Scope the exclusion to uv's tools directory, install, then remove it
$uvTools = (uv tool dir)
Add-MpPreference -ExclusionPath $uvTools
uv tool install "headroom-ai[all]"
Remove-MpPreference -ExclusionPath $uvTools
```
Do **not** disable Defender wholesale — keep the exclusion narrow and temporary.
4. **Report the false positive to Microsoft** so a corrected signature ships for everyone: submit `sg.exe` at the [Microsoft Security Intelligence sample submission](https://www.microsoft.com/en-us/wdsi/filesubmission) page.
### uv build errors: "src does not appear to be a Python project" [#uv-build-errors-src-does-not-appear-to-be-a-python-project]
**Symptom**: `uv tool install "headroom-ai[all]"` fails to build a dependency (commonly `litellm` or `cryptography`) with:
```text
error: Failed to build: ==
Caused by: `src does not appear to be a Python project, as neither `pyproject.toml`
nor `setup.py` are present`
```
or a wheel install fails with `Unknown wheel data type: .DS_Store`.
**Cause**: Corrupted or stale entries in uv's local build/wheel cache, not a Headroom dependency-pin problem. Headroom does not pin exact versions of `litellm` or `cryptography` that would trigger this.
**Fix**: clear uv's cache and reinstall:
```bash
uv cache clean
uv tool install "headroom-ai[all]"
```
To clear the cache for just one package instead:
```bash
uv cache clean litellm
```
If the failure is specifically for `ast-grep-cli==0.44.1`, that release is already excluded by Headroom's dependency pin (`ast-grep-cli>=0.30.0,!=0.44.1`) due to a compromised supply-chain build ([ast-grep/ast-grep#2799](https://github.com/ast-grep/ast-grep/issues/2799)) — `uv cache clean` and a plain reinstall should pick up a safe version automatically.
## Provider-Specific Issues [#provider-specific-issues]
### OpenAI: Invalid API key [#openai-invalid-api-key]
```python
import os
from openai import OpenAI
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
raise ValueError("OPENAI_API_KEY not set")
client = HeadroomClient(
original_client=OpenAI(api_key=api_key),
provider=OpenAIProvider(),
)
```
### Anthropic: Authentication error [#anthropic-authentication-error]
```python
import os
from anthropic import Anthropic
api_key = os.environ.get("ANTHROPIC_API_KEY")
client = HeadroomClient(
original_client=Anthropic(api_key=api_key),
provider=AnthropicProvider(),
)
```
### Unknown model warnings [#unknown-model-warnings]
```python
# For custom/fine-tuned models, specify context limit
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
model_context_limits={
"ft:gpt-4o-2024-08-06:my-org::abc123": 128000,
"my-custom-model": 32000,
},
)
```
## ValidationError on Setup [#validationerror-on-setup]
```python
result = client.validate_setup()
print(result)
# Common issues:
# {"provider": {"ok": False, "error": "No API key"}}
# -> Set OPENAI_API_KEY or pass api_key to OpenAI()
#
# {"storage": {"ok": False, "error": "unable to open database"}}
# -> Check path permissions, use :memory: for testing
#
# {"config": {"ok": False, "error": "Invalid mode"}}
# -> Use "audit" or "optimize" only
```
For testing, use in-memory storage:
```python
client = HeadroomClient(
original_client=OpenAI(),
provider=OpenAIProvider(),
store_url="sqlite:///:memory:",
)
```
## Debugging Techniques [#debugging-techniques]
### Enable Full Logging [#enable-full-logging]
```python
import logging
# See everything
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s %(name)s %(levelname)s %(message)s",
)
# Or just Headroom logs
logging.getLogger("headroom").setLevel(logging.DEBUG)
```
### Use Simulation to Inspect Transforms [#use-simulation-to-inspect-transforms]
```python
plan = client.chat.completions.simulate(
model="gpt-4o",
messages=messages,
)
print(f"Tokens: {plan.tokens_before} -> {plan.tokens_after}")
print(f"Transforms: {plan.transforms}")
print(f"Waste signals: {plan.waste_signals}")
import json
print(json.dumps(plan.messages_optimized, indent=2))
```
### Test Transforms Directly [#test-transforms-directly]
```python
from headroom import SmartCrusher, Tokenizer, OpenAIProvider
from headroom.config import SmartCrusherConfig
import json
config = SmartCrusherConfig()
crusher = SmartCrusher(config)
# Tokenizer wraps a provider-specific TokenCounter -- it takes no
# zero-argument form (headroom/tokenizer.py).
provider = OpenAIProvider()
tokenizer = Tokenizer(provider.get_token_counter("gpt-4o"), "gpt-4o")
messages = [
{
"role": "tool",
"content": json.dumps({"items": list(range(100))}),
"tool_call_id": "1",
}
]
result = crusher.apply(messages, tokenizer)
print(f"Tokens: {result.tokens_before} -> {result.tokens_after}")
```
## Getting Help [#getting-help]
1. Enable debug logging and check the output
2. Use `simulate()` to see what transforms would apply
3. Run `validate_setup()` for configuration issues
4. File an issue at [github.com/headroomlabs-ai/headroom](https://github.com/headroomlabs-ai/headroom/issues) with your Headroom version, Python version, provider, debug log output, and minimal reproduction code
Headroom integrates with the [Vercel AI SDK](https://sdk.vercel.ai) through three patterns: a one-liner wrapper, composable middleware, and standalone message compression.
## Installation [#installation]
```bash
npm install headroom-ai ai @ai-sdk/openai
```
The TypeScript SDK sends messages to a local Headroom proxy for compression. Start the proxy before using the SDK:
```bash
pip install "headroom-ai[proxy]"
headroom proxy
```
## withHeadroom() one-liner [#withheadroom-one-liner]
The simplest integration. Wraps any Vercel AI SDK language model with automatic compression:
```ts twoslash
import { withHeadroom } from 'headroom-ai/vercel-ai';
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
const model = withHeadroom(openai('gpt-4o'));
const { text } = await generateText({
model,
messages: [
{ role: 'user', content: 'Summarize these results...' },
],
});
```
`withHeadroom()` calls `wrapLanguageModel` + `headroomMiddleware()` under the hood. It works with any provider (`@ai-sdk/openai`, `@ai-sdk/anthropic`, `@ai-sdk/google`, etc.).
## headroomMiddleware() for composition [#headroommiddleware-for-composition]
Use the middleware directly when you need to compose it with other middleware:
```ts twoslash
// @noErrors
import { headroomMiddleware } from 'headroom-ai/vercel-ai';
import { wrapLanguageModel } from 'ai';
import { openai } from '@ai-sdk/openai';
const model = wrapLanguageModel({
model: openai('gpt-4o'),
middleware: headroomMiddleware(),
});
```
Pass options to control compression behavior:
```ts twoslash
import { headroomMiddleware } from 'headroom-ai/vercel-ai';
const middleware = headroomMiddleware({
model: 'gpt-4o',
baseUrl: 'http://localhost:8787',
});
```
## compressVercelMessages() standalone [#compressvercelmessages-standalone]
Compress Vercel-format messages directly without wrapping a model. Useful for custom pipelines:
```ts twoslash
import { compressVercelMessages } from 'headroom-ai/vercel-ai';
const result = await compressVercelMessages(messages, {
model: 'gpt-4o',
});
console.log(`Saved ${result.tokensSaved} tokens`);
// result.messages is in Vercel format, ready for the AI SDK
```
## Streaming with streamText [#streaming-with-streamtext]
Compression happens before the request. Streaming responses are unaffected:
```ts twoslash
import { withHeadroom } from 'headroom-ai/vercel-ai';
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
const model = withHeadroom(openai('gpt-4o'));
const result = streamText({
model,
messages: longConversation,
});
for await (const chunk of result.textStream) {
process.stdout.write(chunk);
}
```
## generateObject with compressed context [#generateobject-with-compressed-context]
Works with structured output:
```ts twoslash
// @noErrors
import { withHeadroom } from 'headroom-ai/vercel-ai';
import { openai } from '@ai-sdk/openai';
import { generateText, Output } from 'ai';
import { z } from 'zod';
const model = withHeadroom(openai('gpt-4o'));
const { output } = await generateText({
model,
output: Output.object({
schema: z.object({
summary: z.string(),
severity: z.enum(['low', 'medium', 'high']),
}),
}),
messages: largeConversationHistory,
});
```
## How it works [#how-it-works]
1. Messages are converted from Vercel format to OpenAI format
2. Headroom compresses them via the proxy's `/v1/compress` endpoint
3. Compressed messages are converted back to Vercel format
4. The original model receives the smaller prompt
All other model behavior (tool calling, structured output, streaming) is unchanged.
The official Claude Code extension for VS Code embeds Claude Code. Headroom can
route its Anthropic API requests through the same local compression proxy used by
`headroom wrap claude`, without changing your Anthropic sign-in or selected model.
Use `headroom wrap claude`. This page is specifically for Anthropic's official
Claude Code extension inside VS Code.
## Requirements [#requirements]
* VS Code 1.98 or newer
* Anthropic's official Claude Code extension, signed in and working
* Headroom with proxy dependencies: `pip install "headroom-ai[proxy]"`
* Loopback access to `127.0.0.1` from the VS Code extension host
Confirm that Claude Code works normally in VS Code before adding Headroom. This
makes authentication or extension problems easier to distinguish from proxy
configuration problems.
## Quick start [#quick-start]
1. Open a terminal in the project you use with Claude Code.
2. Start Headroom:
```bash
headroom wrap vscode-claude
```
Headroom starts its proxy and adds two entries under `env` in the Claude Code
user settings file:
```json
{
"env": {
"ANTHROPIC_BASE_URL": "http://127.0.0.1:8787/p/your-project",
"ENABLE_TOOL_SEARCH": "true"
}
}
```
`ANTHROPIC_BASE_URL` changes the endpoint, not the selected model.
`ENABLE_TOOL_SEARCH` keeps Claude Code's on-demand tool loading enabled when it
uses a custom endpoint. Existing settings and prior values for both variables are
preserved for restoration. Headroom does not store or replace your Anthropic
credentials.
3. After the first configuration, run **Developer: Reload Window** from the VS
Code Command Palette.
4. Keep the wrapper terminal running and use the Claude Code panel normally.
## Verify that it is working [#verify-that-it-is-working]
While the wrapper is running:
1. Open `http://127.0.0.1:8787/health`; it should report a healthy proxy.
2. Send a message in the Claude Code panel.
3. Open the dashboard or proxy log whose locations are printed by the wrapper.
Confirm that the request appears there; savings are recorded with each
completed request.
If the health check succeeds but no request appears in the dashboard or proxy
log, reload the VS Code window and confirm that the extension host can reach the
same `127.0.0.1` as Headroom.
## Settings location [#settings-location]
The default user settings file is `~/.claude/settings.json` on macOS and Linux,
or `%USERPROFILE%\.claude\settings.json` on Windows. `CLAUDE_CONFIG_DIR` is
respected when set. To target another profile explicitly:
```bash
headroom wrap vscode-claude --settings-file /path/to/.claude/settings.json
```
Use `--no-configure` to print the settings without editing a file.
The proxy URL includes the current directory as the project attribution name.
Run the wrapper from the intended project directory. If you select another port,
for example `--port 8788`, Headroom writes that same port to the settings file.
## Stop and undo [#stop-and-undo]
Press `Ctrl+C` to stop the proxy. The endpoint remains configured so requests
fail closed rather than silently bypassing Headroom while it is stopped. Restart
it with `headroom wrap vscode-claude` before using Claude Code again.
Restore the values that existed before Headroom configured the extension:
```bash
headroom unwrap vscode-claude
```
Headroom records only the two values it owns in a sidecar next to the Claude
settings file. It refuses malformed settings or conflicting edits rather than
overwriting them. Unrelated Claude settings are preserved.
If you used `--settings-file` during setup, pass the same option when undoing it:
```bash
headroom unwrap vscode-claude --settings-file /path/to/.claude/settings.json
```
## Remote development [#remote-development]
For Dev Containers, SSH, or WSL, `127.0.0.1` must refer to the environment where
the Claude Code process runs. Run Headroom there or forward the selected port,
and pass that environment's Claude settings file with `--settings-file` when
automatic discovery does not match it.
## Troubleshooting [#troubleshooting]
* Check `http://127.0.0.1:8787/health` while the wrapper is running.
* Run `headroom wrap vscode-claude --port 8788` if port 8787 is occupied.
* Reload the VS Code window after changing Claude Code settings.
* Keep the wrapper process running for the entire Claude Code session. A stopped
proxy intentionally does not fall back to a direct Anthropic connection.
* If configuration reports a conflict, inspect `~/.claude/settings.json`; Headroom
will not replace a managed value that changed after setup.
* If you use `CLAUDE_CONFIG_DIR`, launch Headroom from an environment where it is
set to the same value used by Claude Code.
* This integration is for the Claude Code extension, not the Claude desktop app.
Headroom integrates below VS Code's native Copilot model picker. It overrides the
Copilot API proxy endpoint, not the model: if the user selects GPT-5.5, a GPT-5.6
variant, Claude Sonnet, Claude Opus, or another Copilot model, the same model ID
travels through Headroom to GitHub's Copilot API.
No `Headroom` model appears in the picker. Headroom does not patch the built-in
extension, terminate TLS, or edit Codex configuration.
## Requirements [#requirements]
* Current stable VS Code with GitHub Copilot enabled and signed in
* A GitHub account with Copilot access
* Headroom with proxy dependencies: `pip install "headroom-ai[proxy]"`
* Loopback access to `127.0.0.1` from the VS Code extension host
## One-time authentication [#one-time-authentication]
VS Code keeps its Copilot token in extension secret storage. Headroom deliberately
does not read or modify that encrypted store, so authorize Headroom separately:
```bash
headroom copilot-auth login
```
Open the printed GitHub device URL, enter the code, and approve it. Headroom saves
the reusable OAuth credential in its own auth file with user-only permissions.
At launch it exchanges that credential for a short-lived Copilot API token; that
token remains in the proxy process and is never written to VS Code settings.
## Start [#start]
Run from the project whose savings should receive attribution:
```bash
headroom wrap vscode
```
The command:
1. validates Copilot subscription access and resolves the account API endpoint;
2. starts Headroom on `127.0.0.1:8787` with the short-lived upstream token;
3. adds a marker-owned block to VS Code user settings containing
`github.copilot.advanced.debug.overrideProxyUrl` (inline completions) and
`github.copilot.advanced.debug.overrideCapiUrl` (chat);
4. keeps running until `Ctrl+C` so the local proxy is available to VS Code.
Continue using Copilot's normal model picker. The request body—and therefore the
selected model—is not rewritten by the VS Code integration.
## What is routed [#what-is-routed]
The shipped Copilot extension resolves both its chat/agent endpoint and its
completions-core endpoint through the proxy override. This covers native model
selection without registering duplicate models. Some ancillary Copilot services
(telemetry, GitHub API calls, MCP, embeddings, model discovery, cloud agents) use
separate endpoints and are intentionally not redirected.
```text
Copilot UI: user selects model M
-> native Copilot request with model M
-> http://127.0.0.1:8787/p//
-> Headroom compression, cache alignment, metrics, attribution
-> authenticated GitHub Copilot API, still with model M
-> response from model M
```
## Safe settings lifecycle [#safe-settings-lifecycle]
Headroom edits only a marked block in VS Code's `settings.json`. Existing JSONC
comments, formatting, trailing commas, and unrelated settings remain byte-for-byte
unchanged. Headroom refuses malformed files, incomplete markers, or a pre-existing
unmanaged Copilot endpoint override instead of overwriting them.
| Platform | Stable VS Code user settings |
| -------- | ------------------------------------------------------- |
| macOS | `~/Library/Application Support/Code/User/settings.json` |
| Windows | `%APPDATA%\\Code\\User\\settings.json` |
| Linux | `${XDG_CONFIG_HOME:-~/.config}/Code/User/settings.json` |
For Insiders, VSCodium, portable installations, a custom `--user-data-dir`, or a
remote extension host, provide the exact user settings file:
```bash
headroom wrap vscode --settings-file /path/to/User/settings.json
```
Use `--no-configure` to print the two settings without editing a file.
## Model coverage [#model-coverage]
There is no static Headroom model list. Availability stays controlled by Copilot
and the signed-in account. To validate a model, select it in VS Code and send a
short prompt; Headroom forwards the model identifier unchanged. This naturally
covers newly added Copilot models without a Headroom release.
The newest Copilot models may use the OpenAI Responses API instead of the legacy
Chat Completions API. Headroom proxies both routes. Do not treat a model's
`unsupported_api_for_model` response from `/chat/completions` as a proxy failure;
VS Code uses the endpoint supported by that model.
Live verification on August 27, 2026 confirmed successful responses through
Headroom's `/responses` route for each of these requested model IDs:
* `gpt-5.5`
* `gpt-5.6-luna`
* `gpt-5.6-sol`
* `gpt-5.6-terra`
Headroom retained each requested model alias. GitHub may identify the resolved
snapshot in response metadata (for example, a request for `gpt-5.5` returned
`gpt-5.5-2026-04-23`). Model availability remains subject to the signed-in user's
Copilot plan and organization policy.
The proxy supports the native Copilot OpenAI-compatible request paths used by
GPT and Claude models. Headroom's upstream auth hook replaces local client auth
with the current Copilot API token on every Copilot-bound request.
## Stop and undo [#stop-and-undo]
Press `Ctrl+C` to stop the session proxy. The VS Code endpoint setting remains so
future `headroom wrap vscode` runs need no reconfiguration; while the proxy is
stopped, Copilot requests will fail closed instead of bypassing Headroom.
Remove only Headroom's settings block with:
```bash
headroom unwrap vscode
```
Use the same `--settings-file` override used during setup. Other VS Code and
Copilot settings are preserved.
## Remote development [#remote-development]
Copilot may run in the local or remote extension host depending on the workspace.
For Dev Containers, SSH, or WSL, `127.0.0.1` must refer to the host running
Headroom. Run Headroom in that environment or forward the chosen port. Portable,
remote, and profile-specific settings should use `--settings-file` explicitly.
## Enterprise [#enterprise]
GitHub.com Enterprise Cloud normally requires no override; Headroom uses the API
URL advertised during token exchange. For GitHub Enterprise Server/custom domains,
set `GITHUB_COPILOT_ENTERPRISE_URL` or `GITHUB_COPILOT_ENTERPRISE_DOMAIN` before
both `copilot-auth login` and `wrap vscode`.
## Verification and troubleshooting [#verification-and-troubleshooting]
* `headroom copilot-auth status` should report `logged in`.
* `http://127.0.0.1:8787/health` should be healthy while the wrapper runs.
* The health payload's OpenAI upstream should be the Copilot API endpoint.
* Select several native models and confirm Headroom metrics show each request.
* If a model is unavailable, verify the Copilot account entitlement; Headroom
does not add or rename models.
* If connection is refused, keep the wrapper running and check loopback/remote
port reachability.
* Use `--port 8788` when the default port is occupied; settings update safely.
* If Headroom refuses settings, repair the reported JSONC/marker conflict or use
`--no-configure` and apply the printed settings manually.