Guide: Context management

The problem

A session that runs for weeks accumulates hundreds of messages. Every one is re-sent to the provider on every turn. Cost grows linearly with turn count; latency grows too. Rousseau's LLMCompressor (internal/agent/compressor.go) trades a small one-off cost — one summarisation call per compression — for permanent savings on every subsequent turn.

Compression is off by default because the reference deployment uses claudecli on a subscription tier, where token count is not billed. Turn it on when running against Anthropic direct, Bedrock, Vertex, or OpenAI-compatible pay-per-token providers.

The knobs

From CompressionConfig in internal/config/config.go:

agent:
  compression:
    enabled: true
    trigger_messages: 60        # zero uses the default 60
    keep_recent: 8              # zero uses the default 8
    prompt: ""                  # overrides the default summariser prompt

Meanings:

The default summariser prompt

Summarise the following conversation in <=200 words. Preserve every
commitment, TODO, credential, filename, and quoted output. Skip
pleasantries. Return only the summary — no preamble.

Defined as defaultSummaryPrompt in internal/agent/compressor.go. Override with agent.compression.prompt in config.yaml.

Before / after

A session of 68 messages, trigger_messages: 60, keep_recent: 8:

Before compression:                        After compression:

┌──────────────────────────┐              ┌──────────────────────────────┐
│ msg[0]  user             │              │ msg[0]  user (synthetic)     │
│ msg[1]  assistant        │              │   [rousseau-compressed]      │
│ msg[2]  user             │              │   (summary of prior 60       │
│  …  (60 messages)        │      →       │    messages): …              │
│ msg[59] assistant        │              ├──────────────────────────────┤
├──────────────────────────┤              │ msg[1]  user       — verbatim │
│ msg[60] user   verbatim  │              │ msg[2]  assistant  — verbatim │
│ msg[61] assistant        │              │ msg[3]  user       — verbatim │
│  …                       │              │ msg[4]  assistant  — verbatim │
│ msg[67] assistant        │              │ msg[5]  user       — verbatim │
└──────────────────────────┘              │ msg[6]  assistant  — verbatim │
                                          │ msg[7]  user       — verbatim │
                                          │ msg[8]  assistant  — verbatim │
                                          └──────────────────────────────┘
Total messages: 68                        Total messages: 9
Input tokens: ~5000 per turn              Input tokens: ~800 per turn

The marker

The compressor prefixes the synthetic user message with [rousseau-compressed] (constant DefaultCompressorMarker in internal/agent/compressor.go). On subsequent turns, headAlreadyCompressed() uses the marker to detect an already-compressed prefix and skips repeat compression unless the session has grown to 2 * trigger_messages.

This is what keeps compression bounded — you don't pay to re-summarise the summary every 60 messages.

Choosing values

Situation	Recommended
Long-running transport daemon on a paid provider.	trigger_messages: 60, keep_recent: 8. Defaults are tuned for this.
Interactive TUI where you want everything in context.	enabled: false.
Highly technical sessions with lots of quoted code / logs.	trigger_messages: 40, keep_recent: 12. Preserve more recent context; compress sooner.
Cost-critical batch summariser (cron).	Each cron run is a fresh session, so compression rarely triggers. Leave defaults on.

Cost of a compression pass

One summarisation call per firing. The Provider used is whatever Config.Provider selects — the same one the agent uses. That means:

• Sonnet-class compressor call: ~1-2 seconds, roughly the cost of ~2 turns' worth of input tokens.
• Break-even after ~5-10 subsequent turns depending on session shape.

For a cheaper compressor, run rousseau in the two-daemon multi-provider pattern with a Haiku-class model for the compressor daemon. See Guides: Multi-provider.

Emergency: session is too large to load

If a session's payload grows past the model's context window before compression fires — rare but possible with a very small trigger_messages and large tool outputs — the next turn will fail with a provider "context length exceeded" error. Recovery:

rousseau session delete <id> --yes

Then start fresh. Or manually shrink via SQLite:

sqlite3 ~/.local/share/rousseau/sessions.db <<'SQL'
UPDATE sessions SET payload = json_set(payload, '$.messages',
  json_extract(payload, '$.messages[-8:]'))
WHERE id = '<session-id>';
SQL

Note: the exact JSON path syntax depends on SQLite version. Confirm with a SELECT payload first.

Related

• User Guide: Compression + Recall — deeper reference.
• Guides: Rate limits — cost implications.
• Guides: Session management — session lifecycle.
• Reference: Config schema — every field.