Repository navigation
[bug] The agent's own voice can issue commands — the always-on control listener is the only path with no echo guard, and a partial "stop" exits hands-free #386
Description
The one listener that runs while the agent is talking is the one with no echo guard
handleControlResult (packages/sdk/src/voice/use-voice.ts:834-862) is the always-on control-word
recognizer. Its stated purpose is to catch a command "at ANY moment — while TTS plays, while the
agent is processing, while the mic is muted", and its own comment says it is "the ONLY one
running while the agent speaks".
It checks, in order: commandsEnabled → stop-speech (gated on ttsSpeaking) → matchVoiceCommand
→ dispatch mute / unmute / exit / next.
It never asks whether the audio was the agent's own voice. No isEchoing, no
shouldIgnoreResult, no speakEndedAt tail.
Every other path guards this, and the guard exists because it already happened
machine.ts:43-54—isEchoing()/shouldIgnoreResult(), whose docstring reads: "This is the
guard that, duplicated inline, let the agent transcribe itself and reply to nothing." So
speaker→mic bleed is an observed, fixed failure in this product, not a theoretical one.handleResult(use-voice.ts:897+) — applies it: "ECHO (ALL MODES): while the agent speaks OR
within its ~0.8s echo tail — it's the agent's own voice, not you."shouldScanGateTranscript(convo.ts) — takes an explicitechoingparameter, documented as
"so the agent's own voice can't issue commands to it."
Three places reason carefully about this. The fourth — the only one that runs exclusively inside
the echo window — does not.
The concrete failure, and it needs no user input at all
The control listener judges interim results (onResult is wired to interim + final;
commandStateFor("partial", …)). A single-word command must BE the whole utterance — and for an
interim, "the whole utterance" is whatever has been transcribed so far.
EXIT_BY_LANG.en contains the bare word "stop" (added by #331).
So: the agent says "Stop me if this is wrong" → the recognizer's first interim is "stop" →
whole-utterance match → exit → hands-free is torn down, mid-sentence, with the user having
said nothing.
Multi-word phrases match as a whole-word run inside a longer utterance, which widens it further:
| the agent says | fires |
|---|---|
| "Stop me if…" / any clause whose first interim is "stop" | exit |
| "say mute mute to silence me" (an agent explaining its own controls) | mute |
| "the next agent in the chain is the builder" | next — teleports the user away |
next and scrap are gated on the consumer opting in, and scrap is additionally withheld from
partials (#342) — that gate is right, and it is exactly the reasoning this path needs for the other
three. mute and exit have no equivalent.
While muted it is worse in kind, not degree: the muted branch matches unmute, so the agent's own
voice can re-open a microphone the user just closed.
On acoustic echo cancellation
Browser AEC removes much of the speaker→mic path on one device and is why this is intermittent
rather than constant. It is not a guarantee — different output device, speakerphone, a phone on a
desk, headphones off — and the codebase's own history is the evidence: the main-path echo guard was
added because the agent was transcribing itself through the same microphone.
Suggested fix
Apply the guard this path already has a tested implementation of:
- Drop control results while
isEchoing({ ttsSpeaking, speakEndedAt }, now)— except the
stop-speech check, which must keep running, since interrupting playback is the one command whose
entire purpose is to fire while the agent talks. That asymmetry is the reason this listener
exists, and it is expressible in one condition rather than by omitting the guard wholesale. - Consider withholding
exitfrom interim results the way Delete a single turn — by button and by voice ("scrap that") — because a noise message keeps shaping every later reply #342 withholdsscrap. Exit is not
destructive to data, but it is destructive to the session, and a partial is the only place a bare
"stop"can match a word the agent is still in the middle of saying.
Related: #385 (the same "stop" built-in, reached from the other side — a user phrase the
built-in outranks). Fixing #385 alone leaves this path open; fixing this alone leaves #385 open.
Files: packages/sdk/src/voice/use-voice.ts:834-862, packages/sdk/src/voice/machine.ts:43-54,
packages/sdk/src/voice/convo.ts (EXIT_BY_LANG, shouldScanGateTranscript,
phraseMatchesTranscript).
Activity
serge-ivo commentedon Aug 7, 2026
Self-review: fix (1) as written would regress #153. Do not implement it literally.
I proposed "drop control results while
isEchoing, except the stop-speech check". Checking #153 — the issue that created this listener — its title is:and its problem statement names the exact case: "the mute word does NOT fire when the agent is speaking (TTS playing)… exactly the moments a user wants to say 'mute' (to shut the agent up while it talks)".
So gating this listener on
isEchoingremoves mute-during-TTS — the single capability it was built to add. My carve-out covered stop-speech and missed that mute is in the same position. The finding stands; that fix does not.Why the naive fix cannot work
isEchoinganswers "is the agent speaking right now", which during TTS is true for BOTH the agent's voice and the user's. It cannot separate them, and the whole value of this listener is that it acts in exactly that window. The main path can afford the blunt guard because it has no work to do during TTS; this one does.Alternatives that keep #153, in order of preference
isTranscribeBiasEchoalready compares a transcript against the prompt that was SENT ([bug] Silence is transcribed as the agent's own name — proper nouns in the Whisper bias list come back verbatim #332).VoiceTtsholds the utterance (_queue+ the in-flight item,tts.ts:132,173) but exposes onlyspeaking; publishing the current text is the enabling change, and it is small.stopinside the agent's sentence does not. This is the same shape as Delete a single turn — by button and by voice ("scrap that") — because a noise message keeps shaping every later reply #342's partial-withholding and needs no new plumbing.stop→ exit) while leaving every deliberate command working.(2)+(3) together are probably the shippable version: no new API on the TTS object, no capability lost.
The rest of the ticket — that this is the only unguarded path, that it judges interims, and the three worked examples — I re-verified and they hold.
fix(voice): a phrase you bound is yours, the agent cannot command its…
docs(voice): ADR 0001 — mute is available at every moment — and a voi…
test(voice): the mute invariant is now something that objects, not on…
test(console): the touch half of the mute invariant, and the defect i…
feat(supervision): an agent can hand you to another agent when you as…
serge-ivo commentedon Aug 8, 2026
Delivered in 2532f00, with the invariant test hardened in 99959bb. On
main, deployed.Your self-review was followed — the literal fix (1) was NOT implemented, because it would have regressed #153 (mute-during-TTS is the capability this listener exists to provide, and ADR
docs/adr/0001-mute-is-always-available.mdnow records that as a constraint). What shipped is your alternatives (2)+(3): raise the bar inside the echo window rather than close the door.Verified in the code:
handleControlResult(use-voice.ts:869-899) now passesechoing: isEchoing(readGuard(), Date.now())intocommandStateFor, and the recognizer boundary was changed to forwardisFinal(onResult: (text, isFinal) => …) — previously the partial/final distinction was discarded before the handler could use it.commandStateFor(convo.ts), not inline: while echoing, no partials and whole-utterance matches only. So a deliberate "mute mute" still fires while the agent talks; the agent saying "Stop me if this is wrong" no longer produces an interimstopthat matches the built-in exit word.canScrapis deliberately not passed on this path: it has no scrap dispatch, and passing it would enable a destructive command on a path that cannot act on it.Tests:
convo.test.tsdescribe("the agent's own voice cannot issue commands (#386)")covers the worked examples. More usefully,mute-invariant.test.tsis a guard table that classifies candidate answers to this issue — including the rejected first draft — and fails if someone reintroduces it: it asserts mute stays reachable inside the echo window (ADR 0001 M3) and that the control path still consults the echo guard at all ("#386 is back" if it stops).Suite: 6256 tests / 386 files pass; typecheck, biome, design tokens all clean.
Note the fix is structural rather than a new string, so it is not separately greppable in the deployed bundle; it lives in
convo.ts, the same module and the same commit as #387's bail notice, which I did confirm is live at https://proagentstore.online/console/. I have not been able to exercise speaker→mic bleed on a real device — that needs a physical mic with the agent talking, and it is the one part of this I am not claiming to have verified in production.