Repository navigation
[bug] Hands-free start has no feedback until the mic is live — and a failed start reverts silently #284
Description
The report
Pressing Hands-free feels like nothing happened — there is a visible delay before anything changes,
with no indication the tap registered.
Cause: every state flag flips only AFTER the whole startup completes
packages/sdk/src/voice/use-voice.ts:1080 — toggleConvo:
try { getAudioCtx().resume(); } catch {}
unlockSpeechSynthesis(); // prime TTS for iOS/Safari
void ensureTts().then((t) => t.unlock()); // warm the TTS audio context
try {
setPaused(false);
sttRef.current = await makeStt(); // ← may fetch voice config
await sttRef.current.start(); // ← getUserMedia: device open + permission
startAudioMonitor(); // ← AudioContext + analyser
setConvoOn(true); // ← FIRST visible change
setSpeakOn(true);
setMicOn(true);
void acquireWakeLock();
playListeningChime();
} catch { setConvoOn(false); }setConvoOn / setMicOn / setSpeakOn are the last statements. Everything the UI keys off happens
after two awaits — makeStt() and start() (which is getUserMedia, opening a hardware device
and possibly checking permission). Until those resolve the button is in exactly the state it was
before the tap.
Confirmed: there is no starting / pending / busy state anywhere in the hook. The UI has no
value it could bind a spinner to even if it wanted one.
And the failure path is silent, which is worse than slow
} catch { setConvoOn(false); }The catch swallows the error and resets to off. If the mic fails to open — device busy, permission
denied, another tab holding it — the user taps, waits, and the control simply stays off. No message,
no error, nothing distinguishing "still starting" from "failed" from "you didn't press it".
There is already classifyVoiceError / micUnavailableMessage for exactly this, used elsewhere in
the hook. This path does not reach them.
What it should do
- Expose a
startingstate, set synchronously at the top oftoggleConvoand cleared in a
finally. The mode control binds a spinner to it, so the press registers on the same frame. - Keep the chime where it is — it confirms ready, not pressed, and those are different events.
- On failure, surface it through
micUnavailableMessagerather than resetting silently. A user who
denied permission months ago currently has no way to learn that is the problem. - Worth measuring which await dominates. If
makeStt()fetches voice config on every entry, that is
cacheable and would cut the delay rather than just papering it with a spinner — the better fix
where it applies.
Verification
- Tapping Hands-free changes the control's appearance within one frame, before the mic is live.
- The spinner clears exactly when listening actually begins (the chime and the spinner agree).
- A denied or busy mic produces a visible message, not a silent revert.
- Tapping again while starting does not open a second
getUserMedia(there is an existing
leaked-stream note at:1260about ptt→handsfree doing precisely that).
Activity
serge-ivo commentedon Aug 6, 2026
Fixed in
56ca1ae(onmain; the Host Worker deploy for that commit succeeded, so it is live).Your read of
toggleConvowas exactly right, including that there was nostarting/pending/busystate anywhere in the hook — the UI had nothing to bind a spinner to even if it wanted one.The press is acknowledged on the same frame
startingis now real state, set synchronously at the top oftoggleConvo(before the config read and beforegetUserMedia) and cleared in afinally, so a thrown start can never strand the spinner. The mode control binds to it: the Hands-free button shows a spinner and reads "Starting…" the moment you tap, in both the Assistant tab and the Coder Co-pilot. It is also a first-classVoicePhaseand aresolveVoiceStatusstate, so the pill agrees.One subtlety worth recording, because it is easy to get wrong:
startinghad to be checked above themode === "text"early return in bothderivePhaseandresolveVoiceStatus.convoOnhas not flipped yet while the mic is opening, so the mode is still"text"— checked below that line, the state would have been unreachable for its entire lifetime and the press would still have looked unacknowledged. There is a test naming exactly that.The chime stays where it is. As you said, it confirms ready, not pressed — the spinner covers the gap before it rather than replacing it, and clears exactly when listening begins, so the two agree.
The silent revert is gone
resolveToggleAction(pure, tested) makes a second tap while starting anignore, not a stop — so it cannot run a secondgetUserMedia(the leaked-stream note at:1260), and an impatient tap cannot cancel the session it just asked for.On failure it now reaches
micUnavailableMessage. The reason that path "did not reach them" turned out to be a vocabulary mismatch:classifyVoiceErroronly ever knew the Web Speech codes (not-allowed,audio-capture), while this catch receives agetUserMediaDOMException namedNotAllowedError/NotFoundError/NotReadableError— so it fell through to the generic"error"branch. AddednormalizeMediaErrorto translate one dialect into the other, keeping a single classifier, plus a newaudio-busycode forNotReadableError: the device-held-by-another-tab-or-app case has no Web Speech equivalent and is the one a user is least able to guess. So a mic denied months ago now says so.The await you asked about
You were right to ask which one dominates, and right that it was cacheable.
makeSttcalledinvalidateVoiceConfig()and thenawait getVoiceConfig(...)on every entry — a full round trip to/v1/instances/:id/voice-settings, plus a second to/v1/keys/statuswhenever an OpenAI provider is configured, standing between the tap andgetUserMedia.That is now served from cache and revalidated behind the start (
refresh: "background", with in-flight de-duping so a revalidate racing a cold read doesn't fan out). The obvious cost — a settings change landing one start late — is paid off rather than accepted: the settings writer callsinvalidateVoiceConfig()on save and on "use my defaults", which is the one moment we know the cache is wrong. So the change still applies on the very next mic start, and the round trip is off the hot path entirely. The spinner covers what is left (getUserMediaitself, which is genuinely slow and genuinely can prompt).iOS/Safari priming is intact
Deliberately verified, since this change inserts code ahead of it:
getAudioCtx().resume(),unlockSpeechSynthesis(), andensureTts().then(t => t.unlock())are still intoggleConvo, still before the firstawait. The only things now preceding them are two synchronous statements (startingRef.current = trueand aflushSyncsetState), so the call stack is still inside the user-gesture activation window."suspended"or"interrupted"resume logic lives intts.ts/cues.ts, and neither file is in this diff — untouched.setVoiceMode("ptt")'s own priming is unchanged.Verification against your list
getUserMediapnpm typecheck, the full unit suite (3660 tests), andbiome checkat its pre-existing baseline.test(console): the touch half of the mute invariant, and the defect i…
fix(voice): a hands-free send hands the bubble to the message in one …