Voice Mode

What voice mode does

When the WhatsApp transport receives a voice note, rousseau shells out to a locally installed whisper.cpp CLI to transcribe the audio into text, then feeds the text into the agent loop as if the user had typed it. The reply comes back as a normal WhatsApp text message.

The path lives in internal/transport/whatsapp/whisper.go. Every other transport is text-only today.

Opt-in. Voice mode is off by default, and whisper.cpp is not shipped with rousseau's container image — you install and configure the CLI yourself, then flip a single config flag.

Prerequisites

• A working rousseau whatsapp bridge (First transport).
• The whisper.cpp CLI on the daemon's $PATH. Common binary names: whisper, whisper-cli, whisper-cpp.
• A model file. base.en is a good starting point for English-language notes; larger models trade latency for accuracy.

Installing whisper.cpp

Whisper.cpp lives at ggerganov/whisper.cpp. Build recipe (host, not container):

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make -j
bash ./models/download-ggml-model.sh base.en
sudo install -m 0755 main /usr/local/bin/whisper
sudo install -m 0644 models/ggml-base.en.bin /usr/local/share/whisper/ggml-base.en.bin

The binary name after install is whisper; rousseau's default binary lookup expects that name.

Enabling in config

whatsapp:
  reply_header: "💎 *Rousseau Agent*\n\n"
  voice:
    enabled: true
    binary: whisper                                # optional; defaults to "whisper"
    model_path: /usr/local/share/whisper/ggml-base.en.bin
    language: en                                   # optional; empty auto-detects
    extra_args: []                                 # appended before the input filename

Every field in VoiceConfig (internal/config/config.go):

What the daemon does on each voice note

1. WhatsApp delivers an audio message (Opus / OGG / MP3 / M4A / AAC / WAV — the extension is inferred from the mimetype).
2. Rousseau writes the payload to a temp file: /tmp/rousseau-whisper-XXXX/input.<ext> with permission 0o600.
3. Invokes:

   whisper --output-txt --output-file /tmp/rousseau-whisper-XXXX/output [--model <path>] [--language <lang>] <extra_args...> <input.ext>
   

4. Reads /tmp/rousseau-whisper-XXXX/output.txt (falls back to <input>.txt for whisper.cpp variants that write next to the input).
5. Feeds the transcribed text into the agent loop as the user turn.
6. Temp directory is cleaned up with os.RemoveAll (deferred).

Verifying with rousseau doctor

rousseau doctor

Look for:

✔ whatsapp.voice.binary     /usr/local/bin/whisper

or when disabled:

· whatsapp.voice           disabled

A fail on whatsapp.voice.binary means enabled: true but the CLI is not on the daemon's $PATH. Fix the install or turn it off.

Testing end-to-end

1. Enable voice in config, restart rousseau whatsapp.
2. From your phone, record a short voice note ("what does the file main.go do?") and send it.
3. Watch the daemon log:

   whatsapp.voice_enabled binary=whisper model=/usr/local/share/whisper/ggml-base.en.bin
   

4. The daemon replies with a text answer to the transcribed question.

Latency notes

Whisper is CPU-bound by default. Approximate latencies for a 10-second voice note on a modern laptop:

Model	Approx. CPU latency
tiny.en	~1s
base.en	~3s
small.en	~8s
medium.en	~25s

If you build whisper.cpp with WHISPER_COREML=1 (macOS) or WHISPER_CUBLAS=1 (Linux + NVIDIA), transcription can be 2–10x faster. Rousseau does not care — it just shells out.

Container caveats

The rousseau container image (docker/Dockerfile) does not ship whisper.cpp. If you want voice mode inside the container, extend the image:

# Add on top of the reference Dockerfile
RUN apk add --no-cache build-base git && \
    git clone https://github.com/ggerganov/whisper.cpp /tmp/whisper && \
    make -C /tmp/whisper -j && \
    mkdir -p /usr/local/share/whisper && \
    /tmp/whisper/models/download-ggml-model.sh base.en /usr/local/share/whisper && \
    install -m 0755 /tmp/whisper/main /usr/local/bin/whisper && \
    rm -rf /tmp/whisper

Or bind-mount whisper and the model from the host into the Quadlet unit.

Errors surfaced to slog

Event	Meaning
whisper: empty audio payload	The transport delivered a zero-byte audio message. Skipped.
whisper: temp dir: <err>	/tmp is not writable. Check the container's Tmpfs=/tmp:rw mount.
whisper: write audio: <err>	Disk full or permission denied.
whisper: run <binary>: <err>: <stderr excerpt>	The CLI exited non-zero. Excerpt is truncated to 400 chars.
whisper: read transcript: <err>	Whisper ran but did not produce the expected .txt file. Often a whisper.cpp variant that writes to a different path.

Privacy notes

Transcription runs entirely on the host. Audio never leaves the daemon. If you swap the CLI for a hosted transcription service (out of scope for the shipped code), you take on that vendor's data flow — verify against your own privacy posture.

Next

• WhatsApp transport — the transport reference.
• Configuration — every field in internal/config/config.go.
• Deployment — how to bind-mount whisper into the container.