Limitations

When Headroom helps, when it does not, and what to watch out for. Honest documentation of compression constraints and safety gates.

Headroom is designed to compress LLM context without losing accuracy. This page documents when it helps, when it does not, and the safety gates that prevent harmful compression.

When Headroom Helps vs. Does Not

Methodology

The compression figures below are not from a published, repeatable benchmark suite -- we could not trace a prior version of this table to one. Instead of an unsourced number, each row is a directional single-sample measurement taken with compress() against a synthetic payload of that shape. For a seeded, committed harness that reproduces figures of this kind exactly, run uv run python benchmarks/index_proof_table.py --seed 20260902. Your own traffic will vary; measure it with client.chat.completions.simulate() (see Troubleshooting) rather than treating these as guarantees.

Content Type	Compression (single-sample)	Latency Impact	Best For
JSON: Arrays of dicts (search results, API responses, DB rows)	~52% (200-item sample)	Net latency win on Sonnet/Opus	Primary use case
JSON: Arrays of strings (file paths, log lines, tags)	~95% (300-item sample)	Net latency win	String dedup + sampling
JSON: Arrays of numbers (metrics, time series)	~97% (500-item sample)	Net latency win	Statistical summary
JSON: Mixed-type arrays	~45% (200-item sample)	Net latency win	Group-by-type compression
Structured logs (as JSON)	Varies widely with duplication -- see methodology note	Net latency win	Log entries in tool outputs
Agentic conversations (multi-turn)	Not independently measured for this page	Break-even to net win	Multi-tool agent sessions
Plain text (documentation, articles)	Requires the optional `kompress`/ML text path; not exercised in this measurement pass	Adds latency (cost savings only)	Cost optimization
Code	Passthrough	Minimal overhead	See below
RAG document contexts	Passthrough	Minimal overhead	Not compressed

Where Headroom Adds the Most Value

Long agent sessions with accumulated tool outputs
JSON-heavy workflows -- API responses, database queries
Build and test output
Multi-tool agents (repeated tool results compound the per-call savings above)

Where Headroom Adds Little Value

Short conversational exchanges -- overhead can exceed savings on small payloads
Code-only sessions (reading/writing files) -- code passes through
Single-turn requests with no accumulated context

What Headroom Does NOT Compress

Short messages (below min_tokens_to_compress, 50 tokens by default -- headroom/transforms/content_router.py:4806) -- overhead exceeds savings
Source code -- passes through unchanged to preserve correctness
grep/search results -- compact structured format, already minimal
Images -- counted at fixed token cost (~1,600 tokens), not compressed
System prompts -- preserved for prefix cache compatibility

Code Compression

Headroom includes an AST-aware CodeCompressor (tree-sitter, 11 languages: Python, JavaScript, TypeScript, Go, Rust, Java, C, C++, Perl, C#, PHP) but it is gated behind safety protections that prevent it from firing in most real-world scenarios. This is intentional.

Why code mostly passes through:

Size gates, not a word count: a message under min_tokens (50 tokens by default; savings profiles set this anywhere from 10-250) is skipped, and a block under min_chars_for_block_compression (500 characters) is skipped -- both measured in tokens/characters, never words (headroom/transforms/content_router.py:4806,1596)
Recent code protection (protect_recent_code=4): Code in the last 4 messages is never compressed
Analysis intent protection (protect_analysis_context=True): If the most recent user message contains keywords like "analyze", "review", "explain", "fix", "debug" -- ALL code in the conversation is protected

Why this is the right default: Code is almost always fetched because the user wants to work with it. Compressing function bodies would remove exactly what they need.

Where code savings come from: Headroom compresses code in the live zone — the newest tool outputs and content blocks — with the AST-aware CodeCompressor, while keeping recent and analysis-context code fully intact. It never drops messages from the conversation history or strips function bodies.

Override: Set protect_analysis_context=False in ContentRouterConfig for aggressive code compression. Requires headroom-ai[code] for tree-sitter.

JSON Compression Constraints

What Gets Compressed

Arrays of dicts: Full statistical analysis with adaptive K (Kneedle algorithm)
Arrays of strings: Dedup + adaptive sampling + error preservation
Arrays of numbers: Statistical summary + outlier/change-point preservation
Mixed-type arrays: Grouped by type, each group compressed independently
Nested objects: Recursed into, arrays within are compressed (up to depth 5)

What Passes Through

Arrays below 5 items (min_items_to_analyze)
Content below 200 tokens (min_tokens_to_crush)
Bool-only arrays
JSON objects without array values
Malformed JSON (silently passes through, no error)

Edge Cases

NaN/Infinity in numeric fields: Filtered out before statistics are computed
Nesting depth > 5: Inner arrays not examined for compression
Mixed-type arrays with small groups: Groups below min_items_to_analyze are kept as-is

Safety Gates

All compressors follow the same principle: fail gracefully, return original content unchanged.

Invalid JSON passes through (no error raised)
AST parse failure falls back to the original content, unchanged
Compression that makes output larger returns the original
A missing optional dependency (tree-sitter) causes a passthrough with a warning log
Errors are logged at WARNING level and never propagated to callers

LLMLingua was removed

LLMLingua is not part of Headroom 0.37.0. The [llmlingua] extra was removed in 0.9.x with no live code path using it (pyproject.toml:115); text compression today goes through the Kompress (ModernBERT ONNX) path instead, which fails open the same as every other compressor.

Adaptive K: How Item Retention Works

SmartCrusher does not use fixed K values. It uses information-theoretic sizing:

Kneedle algorithm on bigram coverage curves finds the point where adding more items stops providing new information
SimHash fingerprinting detects near-duplicate items
zlib validation ensures the subset captures the full set's diversity

The resulting K is split: 30% from array start, 15% from end, 55% for importance-scored items.

Safety guarantees (additive, never dropped):

Error items (containing "error", "exception", "failed", "critical") -- across ALL array types
Numeric anomalies (> 2 standard deviations from mean)
String length anomalies (> 2 standard deviations from mean length)
Change points (sudden shifts in running values)

These are kept even if they exceed the K budget.

Configuration Tuning

Parameter	Default	Effect
`min_items_to_analyze`	5	Arrays below this pass through
`min_tokens_to_crush`	200	Content below this passes through
`max_items_after_crush`	15	Upper bound on retained items
`variance_threshold`	2.0	Std devs for anomaly detection (lower = more preserved)
`protect_analysis_context`	True	Protect code when user asks about it
`protect_recent_code`	4	Messages from end to protect code in
`skip_user_messages`	True	Never compress user messages
`toin_confidence_threshold`	0.3	Minimum TOIN confidence to apply hints

Provider Interactions

CacheAligner maximizes Anthropic/OpenAI prefix cache hit rates
Token counting uses model-specific tokenizers (tiktoken for OpenAI, calibrated estimation for Anthropic)
Compression works with all providers -- no provider-specific limitations
Compressed content is valid JSON -- downstream tools and parsers work unchanged

TOIN Cold Start

The Tool Output Intelligence Network (TOIN) learns compression patterns from usage. For new tool types:

No learned patterns exist -- falls back to statistical heuristics
Confidence below toin_confidence_threshold (default 0.3) -- TOIN hints ignored
Patterns build up over time as tools are used repeatedly
Cross-session learning requires persistence (TelemetryConfig.storage_path)

Telemetry and the Upload Beacon

Headroom ships two independent, separately-defaulted switches (headroom/telemetry/beacon.py:1-92) -- worth knowing about before you treat this page's "nothing leaves the machine" framing as universal:

HEADROOM_TELEMETRY -- off by default, opt-in. Aggregates compression stats locally for the in-process collector and the /stats endpoint. Never leaves the machine.
HEADROOM_BEACON -- on by default, opt-out. Uploads an anonymous summary of how compression behaved, so we can see when a release regresses a ratio or starts skipping a content type across real workloads instead of only our own test corpus. It carries counters and identifiers only: token totals, compression ratios, skip reasons, provider and model ids, failure counts, plus OS and architecture. Never prompts, completions, code, or file paths -- the collector allowlists fields server-side and drops the rest. Disable it with HEADROOM_BEACON=off, the cross-tool DO_NOT_TRACK=1 convention, or by running offline. is_beacon_enabled() is fail-open: an unrecognized value uploads rather than staying silent. See Proxy for the full field list.

These are deliberately separate so that enabling local /stats does not silently start an upload on the next release.

On this page