Repository navigation
Voice: mute command only active during active recording — must work at all times (during TTS, agent processing, co-pilot chat) #153
Description
Problem
The mute (and other) voice control words are only checked inside the transcription paths — real-time interim, hands-free finalize, and tap-to-talk final (packages/sdk/src/voice/use-voice.ts handleResult). All three only run when the main STT is actively recording a user turn.
So the mute word does NOT fire when:
- The co-pilot / agent is speaking (TTS playing) — the mic is stopped, nothing is transcribed.
- The agent is working / processing between utterances — the mic is paused (
pausedForThinking). - The mic is muted — obviously nothing is being recorded.
In practice: exactly the moments a user wants to say "mute" (to shut the agent up while it talks or thinks) are the moments the mute word is unreachable. The command is gated on the recording state machine.
Root cause
Control-word detection is coupled to the main recording pipeline. When that pipeline is stopped/paused (TTS, thinking, muted), there is no listener at all, so control words are never checked.
Fix
A continuous, always-on background control-word listener, separate from the main recording pipeline:
- A lightweight browser-dictation listener dedicated to control words (mute / stop), active whenever voice is engaged — regardless of app state (TTS playing, agent processing, mic paused/muted).
- Separate from the main STT so it never interferes with normal recording (it yields to the main recorder while the main path is actively capturing a turn, which already checks control words).
- Checks spoken words against the mute list at any time.
- On a mute word: immediately mute the mic AND cancel any TTS playing.
- Wired through
useVoice, so it works in both the agent Assistant chat and the Coder Co-pilot (both consume the same hook).
Acceptance Criteria
- Saying the mute word while the agent is speaking stops the mic and cancels TTS.
- Saying the mute word while the agent is processing (between utterances) mutes.
- Works in co-pilot chat and agent chat (via
useVoice). - The background listener does not double-fire or fight the main recorder during active capture.
- Pure decision logic (when to run the listener; control-word match) is unit-tested.
Related
- Voice: move hands-free control words to the global user profile #129 (mute control words → global profile — the words exist, but only checked during recording)
- Configurable voice control: hands-off command mode (next / play / record) #8 (hands-off voice command mode — broader wake-word epic)
- Voice: unmute command word — dictation-based wake-to-record with configurable keyword and start-recording signal #152 (unmute wake-word — the mirror; a background listener is a shared prerequisite)
Activity
feat(voice): always-on background control-word listener + multi-word …
serge-ivo commentedon Aug 3, 2026
Implemented in 79f94d6.
Always-on background control-word listener (
packages/sdk/src/voice/use-voice.ts): a lightweight browser Web Speech recognizer (ctrlSttRef), separate from the main recording pipeline, whose only job is to catch a control word at any moment.shouldRunControlListener({ engaged, mainRecording })= engaged (hands-free or ptt) AND NOT the main mic actively capturing. So it's live while the agent is speaking, processing, or the mic is muted. It yields the mic to the main recognizer during active capture (which already checks control words), so the two never fight:startListeningstops it synchronously before opening the main mic, a reconcile effect re-arms it when the main mic idles, and it's disposed on unmount.onResultonly checks control words (never feeds the LLM/STT pipeline): a mute word →muteFromCommand; the stop-speech keyword → cancel TTS while speaking.muteFromCommandnow also cancels in-flight/queued TTS ("mute" = be quiet), so it interrupts the agent mid-sentence.useVoice→ covers both the Assistant chat and the Coder Co-pilot. Degrades gracefully to the existing in-turn detection where Web Speech is absent (iOS Safari).Multi-word phrases (
convo.ts):matchVoiceCommandmatches a single-word command only as the whole utterance (precision), but a multi-word phrase likemute mutealso as a contiguous whole-word run — andmutealone (half the phrase) does NOT trigger. Delimiter parse splits on comma/newline/semicolon only (never space), somute mutestays one phrase.AC status: ✅ mute while speaking (stops mic + cancels TTS) · ✅ mute while processing · ✅ works in co-pilot + agent chat (shared hook) · ✅ yields to the main recorder, no double-fire · ✅ pure logic (
shouldRunControlListener, multi-wordmatchVoiceCommand) unit-tested.Tests: +multi-word phrase matching, +
shouldRunControlListener, +delimiter-preservation. Full voice suite green (170); SDK + console build clean.Caveat: the background recognizer hears the agent's own TTS while it plays, so a mute phrase spoken by the agent could self-trigger — mitigated by using a distinctive phrase (a two-word
mute muteis very unlikely in agent speech). Same tradeoff the existing stop-speech keyword already accepts.serge-ivo commentedon Aug 3, 2026
Fixed in 79f94d6. Added an always-on background control-word listener (
packages/sdk/src/voice/use-voice.ts) — a lightweight browser Web Speech recognizer, separate from the main recording pipeline, that runs whenever voice is engaged but the main recorder is idle (shouldRunControlListener). So the mute word is now interceptable while the agent is speaking, processing, or the mic is muted — it stops the mic AND cancels TTS. Covers both the Assistant chat and the Coder Co-pilot viauseVoice. +tests (multi-word phrase matching, shouldRunControlListener); full voice suite green.fix(voice): mute was unreachable while speaking in Whisper mode; add …
fix(voice): a phrase you bound is yours, the agent cannot command its…
docs(voice): ADR 0001 — mute is available at every moment — and a voi…
test(voice): the mute invariant is now something that objects, not on…
fix(voice): mute silences the agent without deleting what the user sa…
fix(voice): mute command is mic-only — no longer silences the agent (#…