Overview
Rousseau's provider abstraction (agent.Provider and agent.StreamingProvider) lets you switch LLM backends without touching the session store. Every message is stored provider-agnostically. This guide covers the five backends and the practical steps to cut over.
Provider matrix
| Provider | Auth | Best for |
|---|---|---|
claudecli |
Inherits claude CLI OAuth |
Individual operators; zero API keys plumbed through rousseau |
anthropic |
ANTHROPIC_API_KEY |
Direct API, prompt-cache markers |
bedrock |
AWS credential chain | Enterprise with an AWS contract |
vertex |
GCP service-account JSON | Enterprise with a GCP contract |
openai / openrouter / ollama |
Depends | Self-hosted vLLM, LM Studio, OpenRouter aggregation, or local Ollama |
Migration recipes
Motivation: move from OAuth-inherited claude auth to a scoped API key you can rotate.
# before
provider: claudecli
claudecli:
binary: claude
permission_mode: bypassPermissions
# after
provider: anthropic
anthropic:
model: claude-sonnet-4-6
max_tokens: 4096
export ANTHROPIC_API_KEY=sk-ant-…
rousseau doctor # confirms anthropic.api_key present (masked)
rousseau chat # smoke-test
Motivation: unify LLM spend under an AWS contract; centralised billing and data-residency guarantees.
# before
provider: anthropic
anthropic: { model: claude-sonnet-4-6 }
# after
provider: bedrock
bedrock:
region: eu-west-2
model: anthropic.claude-sonnet-4-6-20250101-v1:0
profile: platform
aws sso login --profile platform
rousseau doctor
rousseau chat
Motivation: airgapped deployment or dev-time cost avoidance.
provider: ollama
ollama:
model: llama3.1:70b-instruct
# base_url defaults to http://localhost:11434/v1
ollama pull llama3.1:70b-instruct
ollama serve & # or a systemd unit
rousseau doctor
rousseau chat
Local models may not honour tool-use as reliably — see Providers: OpenAI-compatible.
Motivation: consolidate under a GCP contract.
provider: vertex
vertex:
project: my-gcp-project
region: europe-west4
model: claude-sonnet-4-6@20250101
credentials_file: /etc/rousseau/vertex.json
gcloud auth application-default login # or set GOOGLE_APPLICATION_CREDENTIALS
rousseau doctor
rousseau chat
Data-store compatibility
Every switch preserves the session store as-is. Messages are provider-agnostic: agent.Message records the role, content, and any tool-use blocks — never the model id or the raw provider response.
If you have compression enabled (agent.compression.enabled: true), the summary was written by whichever provider was active at the time. This is fine — the summary is plain text.
Downgrade path
Every provider switch is reversible by editing the config and restarting. There is no data-store lock-in.
Verification checklist
-
rousseau doctor—provider.selectedmatches the new value and any required credentials areok. -
rousseau chat— send "hello", observe a coherent reply. -
rousseau session list --limit 5— pre-existing history remains. - For long-running daemons (WhatsApp, Slack, etc.), restart them so they pick up the new provider factory.
Common failure modes
claudecli: exec: "claude": executable file not found— you setprovider: claudeclibut the CLI is missing. Install it or pick another provider.anthropic: no API key—ANTHROPIC_API_KEYnot exported andanthropic.api_keynot in the file.bedrock: NoCredentialProviders— AWS credentials not resolved. CheckAWS_PROFILE, IAM role, or IRSA.vertex: could not find default credentials— setcredentials_fileorGOOGLE_APPLICATION_CREDENTIALS.- Tool-use quality drops with a local model — small models fail JSON-schema tool inputs. Try a bigger model or fall back to Anthropic for tool-heavy sessions.
Related pages
- Providers — protocol-level detail per provider.
- Reference: Config: Provider
- Recipes: Bedrock multi-account
- Recipes: Airgapped deployment
- Best Practices: Cost control