Guide: Rate limits

Where rate limiting happens

Rousseau does not implement its own rate-limit handling. Every provider client delegates to the upstream SDK:

• Anthropic direct — anthropic-sdk-go handles HTTP retries, respects Retry-After, applies exponential backoff on 5xx and 429. See internal/llm/anthropic/client.go.
• Bedrock — aws-sdk-go-v2 handles throttling errors with adaptive retries.
• Vertex — Google auth libraries handle their own retries.
• OpenAI / OpenRouter / Ollama — the Go OpenAI-compatible client handles 429s.
• claudecli — Claude Code's own claude binary handles limits. Rousseau just shells out.

Failed requests surface as turn.failed, whatsapp.handler_failed, or cron.run_failed slog events. The message text will include the provider's error string (typically 429 Too Many Requests with a suggested backoff).

When you actually hit a limit

Symptoms in the logs:

{"level":"ERROR","msg":"whatsapp.handler_failed","err":"anthropic: complete: 429 Too Many Requests"}

Because rousseau treats a turn as failed on unrecoverable errors, the operator sees the failure in the transport reply — the daemon does not silently swallow it. This is intentional.

Reducing rate-limit pressure

Three levers, in order of impact:

1. Prompt cache markers (Anthropic direct)

applyCacheMarkers in internal/llm/anthropic/client.go marks a leading window of messages for the Anthropic ephemeral prompt cache. When CacheableMessages > 0, the system prompt is also cache-marked. Cached input tokens are billed at roughly 10% of standard input rates and cache hits do not consume the standard input rate-limit budget.

The agent (internal/agent/agent.go) opts into this on multi-turn sessions. If you build custom loops on top of rousseau's Go API, set Request.CacheableMessages and Request.System — even a shallow cache hit shaves both cost and rate-limit pressure.

Cache markers are Anthropic-direct only today. Bedrock, Vertex, and OpenAI-compat providers ignore them.

2. Compression

For long sessions on a pay-per-token provider (Anthropic direct, Bedrock, Vertex, OpenRouter), enable compression:

agent:
  compression:
    enabled: true
    trigger_messages: 60      # from CompressionConfig default
    keep_recent: 8

The LLMCompressor (internal/agent/compressor.go) summarises the oldest slice of the session into a single synthetic user message when message count crosses trigger_messages, and preserves the last keep_recent messages verbatim. Fewer tokens per turn = less rate-limit pressure.

Compression is off by default because the reference deployment uses claudecli on a subscription tier, where token count is not billed.

3. Slower cron cadence

For pure background daemons, halving the cron cadence halves the requests. rousseau cron cadences are cron expressions — go from every 15 minutes to every hour if the freshness requirement allows it.

Approximate cost by provider

Rate limits and per-token cost move independently, but the two are usually correlated (paid tiers have higher limits). Rough guide as of 2026-07:

Get the current numbers from each provider's pricing page.

When the SDK exhausts retries

If the provider's SDK gives up, rousseau surfaces the final error. The turn is lost — there is no queue and no on-disk retry. Two mitigations:

• Message the operator through the same channel. The turn failure is visible in the transport reply; the operator can rephrase.
• Fall back to a second provider by hand. See Guides: Multi-provider for the two-daemon pattern.

Automatic cross-provider failover is a roadmap item.

Debugging rate-limit trouble

1. Set log.level: debug in config.yaml. The SDK debug output shows the exact Retry-After value.
2. Look for turn.failed, whatsapp.handler_failed, cron.run_failed in the journal.
3. Check the provider dashboard (Anthropic Console, AWS CloudWatch, GCP Cloud Monitoring) for actual quota consumption.
4. If you're on a subscription tier, watch for daily-quota resets — the SDK error usually includes the reset time.

Provider-by-provider quick reference

Provider	Retry behaviour	Rate signal	Cost per 1M input	Cost per 1M output	Cache read cost
anthropic direct	SDK retries 5xx; 429 with Retry-After respected	429 Too Many Requests header carries reset time	~$3 (Sonnet)	~$15 (Sonnet)	~$0.30
bedrock	AWS SDK adaptive retry	ThrottlingException	~$3 (Sonnet)	~$15 (Sonnet)	not yet
vertex	Google SDK exponential retry	429 RESOURCE_EXHAUSTED	~$3 (Sonnet)	~$15 (Sonnet)	not yet
openai	SDK retries 5xx; 429 respected	429 Too Many Requests	model-specific	model-specific	model-specific
openrouter	passthrough to underlying provider	provider-dependent	model-specific	model-specific	provider-dependent
ollama	SDK retries; local so rarely fires	none	$0 (compute cost)	$0 (compute cost)	N/A
claudecli	subprocess errors surface; no rousseau-side retry	opaque	subscription	subscription	opaque

Authoritative sources:

• Anthropic pricing
• AWS Bedrock pricing
• Vertex AI pricing
• OpenAI pricing
• OpenRouter model list

Caller-side retry recipe

Rousseau does not retry inside Complete. If you embed the agent library, wrap Turn in your own retry loop with exponential backoff and jitter:

func retryTurn(ctx context.Context, ag *agent.Agent, sess *agent.Session, maxRetries int) (agent.Message, error) {
    var lastErr error
    for attempt := 0; attempt < maxRetries; attempt++ {
        m, err := ag.Turn(ctx, sess)
        if err == nil {
            return m, nil
        }
        if !isRateLimit(err) {
            return agent.Message{}, err // non-retryable
        }
        lastErr = err
        // Exponential backoff with jitter: 1s, 2s, 4s, 8s, ...
        backoff := time.Duration(1<<attempt) * time.Second
        jitter := time.Duration(rand.Int63n(int64(backoff / 2)))
        select {
        case <-time.After(backoff + jitter):
        case <-ctx.Done():
            return agent.Message{}, ctx.Err()
        }
    }
    return agent.Message{}, fmt.Errorf("giving up after %d retries: %w", maxRetries, lastErr)
}

func isRateLimit(err error) bool {
    s := err.Error()
    return strings.Contains(s, "429") || strings.Contains(s, "rate limit") || strings.Contains(s, "ThrottlingException")
}

Troubleshooting

429 Too Many Requests every request

You are on a low tier or another workload is consuming the quota. Options: (1) request a limit increase, (2) split load across providers, (3) run claudecli for subscription-only workloads.

529 Overloaded intermittently

Anthropic's system is at capacity. Not per-account throttling — the whole region is loaded. Retry with backoff.

Cache markers set but no visible cost saving

Verify that CacheableMessages is actually being set. applyCacheMarkers in internal/llm/anthropic/cache.go is a no-op for zero. Also verify the prefix is stable — a system prompt that regenerates per turn defeats caching.

ThrottlingException on Bedrock with low volume

Bedrock quota is per-account-per-model-per-region. Some models default to very low quotas (2–5 requests per minute). Request an increase in the Service Quotas console.

Slow API responses despite low usage

Some providers de-prioritise low-tier accounts under global load. Anthropic's x-ratelimit-* response headers indicate current bucket state — inspect them if you have SDK access.

Related pages

• Providers: Anthropic — cache-marker details.
• Configuration — every compression knob.
• User Guide: Compression + Recall — deeper compression discussion.
• Guides: Multi-provider — split load across endpoints.
• Guides: Rate/Model Swap — hot-swap providers on failure.

Further reading

• internal/llm/anthropic/client.go — SDK invocation.
• internal/llm/anthropic/cache.go — cache-marker helper.
• internal/agent/agent.go — where turn failures surface.
• Provider pricing pages linked above.