Repository navigation
[bug] "Do it" is transcribed "Duet" and "go, send" becomes "context:" — a tmux agent gets no vocabulary bias, and the divergence guard is off below four words #371
Description
The message was recorded. It just isn't the message.
Reported as "it showed as I dictated it, then never stayed on the thread — just disappeared."
Nothing was dropped. Instance cda75e28 (Heartfull tmux Operator), message 0f6c8f6f:
role: user
content: "Duet" ← what was stored, sent to the agent, and shown on the thread
dictation: "Do it" ← what the browser gate heard
audioKey: f220871a-… ← the recording, intact
createdAt: 2026-08-07T06:28:58Z
The user said "Do it". The live bubble showed "Do it". Whisper returned "Duet", the bubble
cleared, and the thread got a word the user never said — which reads exactly like the message
vanishing, because the thing on screen is not theirs.
It broke the conversation, not just the record
The turn before, the agent asked: "…shall I do that now?" — a yes/no. The user answered "Do it".
The agent received "Duet", could not resolve it, and replied by re-listing the same five tmux
sessions it had already listed twice. The task never started.
The pattern across the whole thread — 4 of 5 voice turns are wrong
| when | stored + sent (Whisper) | heard live (browser gate) |
|---|---|---|
| 03:35 | Is tmux ready? |
Yes Timo is ready is the agent replying in the terminal or not |
| 05:35 | Heartfull |
(agreed) |
| 06:26 | Do you see the console output? |
Do you see the console output what's there right now |
| 06:28:19 | context: |
Send |
| 06:28:58 | Duet |
Do it |
The browser gate was more accurate than Whisper on every turn where they disagreed, and the
platform always ships Whisper's reading and files the gate's into a footnote badge.
Root cause: a tmux agent is given no vocabulary at all
buildTranscribePrompt (packages/sdk/src/voice/prompt.ts:27) gates its term lists on surfaces:
if (surfaces.includes("coding") || surfaces.includes("repo")) parts.push(CODING_TERMS);
if (surfaces.includes("apply")) parts.push(APPLY_TERMS);This instance is surfaces: ["tmux"], runtime: "coding". tmux is in neither branch, so
CODING_TERMS is skipped. The call site passes the instance name as extra
(InstanceDetail.tsx:237), so the entire transcription bias for this agent is the literal string
"Heartfull (tmux)" — its own name, and nothing else.
Meanwhile CODING_TERMS already contains the exact words this user says to a terminal agent:
console, terminal, git, npm, pnpm, commit, branch, deploy, build, lint, endpoint. They are sitting
in the file, unused, because the gate asks about the surface rather than the runtime.
That is why Do you see the console output… lost half its sentence, and why two-word utterances had
nothing at all steering them. tmux should be in the coding branch — a one-line change — and the
gate is arguably reading the wrong axis entirely, since runtime: "coding" was true here the whole
time.
Note the second-order risk this creates: the ONE term the agent is biased toward is a proper noun,
which is precisely the #332 failure mode (a proper noun in the bias list coming back verbatim on a
low-information clip). isTranscribeBiasEcho would catch Heartfull (tmux) because it is
multi-word — but it deliberately passes single-word echoes, so a one-word agent name would not
be caught.
The guard that exists is switched off for exactly these turns
dictationDiverged (packages/sdk/src/voice/machine.ts:322) is what flags a transcript as
materially short:
export function dictationDiverged(heard: string, final: string): boolean {
const h = words(heard);
if (h.length < 4) return false; // ← silent below four heard words
return words(final).length < Math.ceil(h.length * 0.6);
}"Do it"→h.length === 2→ false. No warning."Send"→h.length === 1→ false. No warning.
So the two turns that came back as pure nonsense are the two the guard is contractually forbidden to
comment on. The floor is defensible in isolation — four words is "too little signal to accuse
anything" on a volume test — but the result is that short utterances, which are the most
likely to be mistranscribed, are the only ones with no check at all. dictationLoss still renders
a dim "2 words not in the transcript", which is not a signal anyone reads as "this is the wrong
word".
What to change
1. A short utterance needs a different test, not the volume one. When the heard capture is under
four words, compare by overlap instead: if the final transcript shares no word with what the
gate heard, that is divergence regardless of length. Do it/Duet and Send/context: both trip
it; colour/color and ordinary engine disagreement do not, because they overlap.
2. Prefer the gate's reading when the final is a bare unknown token. The gate got it right every
time here. At minimum, a one-tap "use what I heard instead" on the bubble — the data is already
stored (dictation) and already rendered behind the Show what was heard toggle
(SpokenMessage.tsx); today it is archaeology, not a correction path.
3. Re-examine the bias prompt. context: is a prompt fragment surfacing as a transcript. #332
treated the symptom (agent names); the general shape — a low-information clip echoing the prompt —
is still live and produces text the user cannot distinguish from their own words.
Related
#364 (the composer showed nothing while this happened, so the live capture was only visible in the
thread bubble), #332 (prompt-token echo), #319 (the stored live capture that made this diagnosable at
all — without it this report would have been unanswerable).
Activity
serge-ivo commentedon Aug 7, 2026
Self-correction on point 3 of "What to change" — it contradicts the root-cause section above it and should be struck.
I first assumed
context:was a bias-prompt fragment coming back (the #332 shape), and wrote point 3 on that basis. Then I readprompt.tsand disproved it: for this instance the prompt is the single string"Heartfull (tmux)", becausesurfaces: ["tmux"]misses both term-list branches. The wordcontextis not in the bias list, so it cannot be an echo of it.context:is just a bad transcription of a ~1-second clip with no vocabulary steering it — the same failure asDo it→Duet, not a separate mechanism. The trailing colon is suggestive but is not evidence of anything I can substantiate.So the fixes are points 1 and 2, plus the root cause: put
tmuxin the coding-terms branch (or gate onruntimerather thansurfaces). Point 3 as written asks someone to hunt a leak that this investigation ruled out.Worth keeping from it, restated honestly: #332's guard passes single-word echoes deliberately, so if an agent's name is one word it would still surface verbatim on a silent clip. That is a real gap, but it is not what happened here.
serge-ivo commentedon Aug 7, 2026
Follow-ups filed for the vocabulary side of this: #372 (derive the proper nouns the platform already holds — tmux targets, repo names, subordinate names — into the prompt's existing
extraslot, which today receives only the instance name) and #373 (a user-editable vocabulary list at profile + agent scope).Worth recording here because it constrains both: the browser Web Speech recogniser cannot be given a vocabulary at all. There is no
SpeechGrammarListinpackages/sdk/src, and adding one would not help — the API is specced but Chrome's default recogniser ignores JSGF grammars. Since the browser gate is what returnedTimofor tmux in this very thread, a bias-only solution cannot fix the live-dictation half. #373 handles that with a post-transcript fuzzy correction over the same list, which is the only lever that reaches both engines.fix(voice): a terminal agent gets a vocabulary, and a two-word turn g…
feat(voice): a vocabulary the platform derives, and one the user owns (…
serge-ivo commentedon Aug 7, 2026
Fixed in bb11a69.
Which layer failed: transcription, both times — not the command matcher.
Send→context:looked like it might be a command-matching problem. It is not: there is nosendinconvo.ts's closed vocabulary (repeat/mute/unmute/exit/next/scrap), so "send" was never a candidate for matching and was always going to be sent as speech.Do it→Duetis the same layer. Both are the decoder returning a word for a short clip it had no vocabulary to constrain.Root cause 1 — the gate read the wrong axis.
buildTranscribePromptgated onsurfaces; a tmux operator issurfaces: ["tmux"], in neither branch, soCODING_TERMSwas skipped for an agent whoseruntimewas"coding"the whole time. Both axes are read now, plus a newTMUX_TERMSlist carryingtmux,pane,detach,stdout— the shell nouns with a strong wrong English prior. Deliberately excluded:run,open,list,show— said constantly, transcribed correctly, and single words, which is the shape that comes back as a phantom turn under #332 for no accuracy gain.Root cause 2 — the guard was off for exactly these turns.
dictationDivergedis a volume test, silent below four heard words, so the two turns that came back as nonsense were the two it was forbidden to comment on. Below four words it now judges by OVERLAP: no word in common means the two engines did not hear the same utterance, at any length.colour/colordoes not trip it —nearWord(newvoice/vocabulary.ts) counts a spelling variant as the same word, with 0 edits allowed below four letters sogo/nostay different. The volume test is untouched above four words.Both reported turns now flag on the bubble (
SpokenMessagerenders the warning icon off this predicate).Item 3 of the ticket ("prefer the gate's reading" / a one-tap "use what I heard instead") is NOT done — it needs an edit-and-resend path on a sent message, which is a bigger change than either fix here and belongs in its own ticket. What ships is that the divergence is now stated rather than silently rendered as a dim word count.
The bias prompt was re-examined as asked (#372/#373, bfbfd29): the term-entry rule and the #332 interaction are written down in
packages/sdk/src/voice/prompt.ts.fix(voice): the dictation gate accumulates the utterance, so a pause …