Repository navigation
[bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512
Description
temperature=0 removed Whisper's repetition/truncation mitigation — clauses the user spoke are missing from the transcript
The owner reports voice sometimes not transcribing. Two recorded messages show a different failure from a dropped clip: the clip transcribed, and a whole trailing clause he spoke inside the recorded window is absent.
The evidence
dictation is persisted only when the live capture and the final transcript diverge (packages/sdk/src/voice/machine.ts:599-604), and heard is stamped at end-of-turn (machine.ts:410) — so anything in heard was spoken inside the clip. Two messages on Chess coder 2 (bd43f4de-ef35-4051-bdec-43f8571414a1), 2026-08-11:
| UTC | sent | heard (live capture) |
|---|---|---|
| 23:04:28 | …and file issues that come out of it. |
…file issues that come out of it **don't wait for the night rebuild we have to run** |
| 23:00:49 | …Headless, headless browser, but they should run. |
…**also run Browser** but they should run **are they running an HP** |
And a repetition loop that was sent as a real user turn at 21:44:05: "apps chess academy, chess academy, chess academy, …" ×6. A second, longer loop was caught and ignored at 01:33:37 ("chess-academy" ×14, client:voice "result ignored").
Mechanism — INFERRED, and the experiment that settles it
1449ee1 (#490) added form.append("temperature", "0") to all requests, including the streaming default model — packages/sdk/src/voice/stt.ts:401.
OpenAI's reference Whisper decoder uses a temperature fallback ladder (0 → 0.2 → 0.4 → … → 1.0), triggered on compression-ratio and average-logprob thresholds, and it exists specifically because pure greedy decoding produces repetition loops and premature truncation. Pinning temperature to 0 with no fallback removes that mitigation.
I cannot claim it introduced the repetitions. A drifting (not looping) repetition appears pre-#490 at Aug 10 02:11:24. What is established is that the documented mitigation is now gone, and pure loops appear after.
Settled by: unset temperature for a day and count repetition/truncation rows against the same window. That is a one-line change and a query, not a design argument.
Why it matters beyond tidiness
A dropped clip is visible — the user says it again. A silently truncated clip is not: the agent receives a grammatical, plausible instruction that is missing its operative clause. In the 23:04:28 case the lost words were "don't wait for the night rebuild we have to run" — an instruction with real consequences, spoken and never delivered.
Acceptance criteria
- Decide, with reasons written down, whether
temperature=0should apply to the streaming model at all — it was added for the phantom-turn problem, whichno_speech_probwas also meant to address (and does not, on this model — see the gate-calibration issue). - If greedy decoding is kept, add the repetition/compression-ratio check the ladder provided, or detect a repetition loop client-side before it is sent as a user turn (one was, at 21:44:05).
- Where the live capture and the final transcript diverge by more than a threshold, that divergence is surfaced rather than silently resolved in favour of the transcript — the data is already persisted (
machine.ts:599-604). - A test with a known repetition-loop transcript asserting it is not sent as a user turn.
Activity
fix(voice): a decoder that stuck can no longer be sent to an agent as…
serge-ivo commentedon Aug 12, 2026
Fixed in
de02397(onmain; live — verified below). Part of the stated mechanism is inverted,so it is not implemented as written. Detail first, because it changes what the fix is.
Criterion 1 — the decision, with reasons
For
whisper-1the issue's mechanism is backwards. OpenAI documentstemperatureon thisendpoint as: "If set to 0, the model will use log probability to automatically increase the
temperature until certain thresholds are hit." On the Whisper decoder, 0 is the value that
selects the fallback ladder, not the one that removes it. #490's use of it there stands, and it
is kept.
It is dropped for the streaming
gpt-4o-transcribefamily, which is a different architecturethat wording does not describe. There, 0 is plain greedy decoding — the regime that produces
repetition loops — and it was added for the phantom-turn problem, which on that model is handled by
the energy gate and the noise filter. It was never shown to help there, so the model's own default
is now used.
Honest limit: if that default is itself 0, this change is inert and the loops have another
cause. The client cannot observe it. Which is exactly why the temperature line is not the
deliverable:
Criterion 2 — the deliverable, which holds whichever way that lands
isRepetitionLoop(audio.ts) judges the shape, not the words: a phrase of up to 5 wordsrepeated ≥4× consecutively and covering ≥50% of the transcript.
isNoiseTranscriptcould nevercatch these — it matches a fixed list of silence hallucinations, and a loop is built from your own
vocabulary, so every phrase in it is plausible. Wired into
planSendas a third drop reason,judged before the language gate so a decoder glitch cannot earn a "say that again in your
language" nudge, and the words stay on the failed bubble like the other two drops.
Both bars are set high on purpose: a false positive here drops a real turn, which is the harm
being prevented. "no, no, no, that is not what I meant", "very very good", and a real instruction
that merely ends in a stutter all still send. The recorded
"chess academy"×6 and"chess-academy"×14 do not.Criterion 3 — divergence surfaced
Two things were wrong, both verified in code:
pags:voice-debuglocalStorage flag, so in production itnever fired at all;
dictationDivergedasks about volume — for a long turn it needs the transcript to be >40%shorter, so a clause lost off the end of a paragraph does not move it.
lostTail(heard, final)asks the other question — how many words at the end of the live capturethe transcript does not account for — and ≥4 now reports unconditionally as
voice-transcript.It is a floor, not an exact count: the walk stops at the first word the transcript does have, so
your 23:04:28 case scores 6 for 10 lost words. Undercounting is the right direction for something
that only decides whether a row is worth writing.
Criterion 4 — tests
The exact transcript that reached the agent is asserted to be refused; the ×14 loop likewise;
punctuation/casing variants; six negative cases so a real turn is never eaten; ordering vs the
language gate; and
lostTailagainst the recorded truncation.Verified live
Deployed console bundle at
https://proagentstore.online/console/:i.e.
temperatureis sent to whisper-1 only. The detector anddropNotestrings are in the bundle too.Not verified — and the query that closes it
The mechanism remains inferred; I did not reproduce it and I am not claiming it is fixed. The
detector prevents the symptom regardless of cause; the temperature change is the experiment you
named, now shipped as the default.
The measurement: in a week, count
list_errorsrows over a comparable window —source:voicecarrying the new repetition drop, andvoice-transcript"dropped a clause heardlive". Two readings:
temperature line is inert, and the detector is the whole fix.
Either way nothing reaches an agent as your words again, which is the part that had consequences.
The
voice-transcriptrows are the more valuable half: they are the first evidence this platformhas ever collected about truncation, and if they show a pattern (one language, one mode, long turns)
that is a sharper issue than this one.
feat(feedback): a message can be reported from the bubble it is in, a…
fix(voice): the phantom that reached an agent as ad copy is caught by…
feat(voice): mark suspect transcripts rather than drop them (#626)