Skip to content

[design] Command-vs-content is one decision serving two actions with asymmetric costs — split the verdict and park the uncertain case instead of adding a fifth matcher rule #457

Closed

Description

@serge-ivo

Design assessment: is repetition-collapse a hack? Partly yes — and here is what the proper shape looks like

Asked directly, so answered directly. The fix proposed in #456 is a good tactical patch and a poor architecture. It is worth shipping, and it is worth knowing what it is: the fourth rule stacked on a classifier that cannot be made reliable by adding rules.

Why the current approach keeps producing bugs

Voice carries two multiplexed streams over one channel — content for the agent, control for the app. Demultiplexing them is the hard problem. Today it is done by string matching in convo.ts, and the accumulated rules are:

  1. a single-word command must be the whole utterance (:834-838)
  2. a trailing command is stripped only if the phrase is multi-word (:846-853)
  3. destructive commands (scrap, back) are whole-utterance only, judged on a final (:810-817)
  4. proposed in [bug] Repeating "mute" breaks it — one "mute" matches, two or more match nothing and get sent to the agent as a message #456: repeated bare words count as a command

Each rule is individually defensible and each was added after a real defect. That pattern — a rule per bug, none wrong, the set still leaky — is the signature of a classifier working without enough evidence. Rule 5 will be needed too.

Two structural faults produce this.

Fault 1 — one confidence threshold governs two actions with wildly asymmetric costs

splitTrailingCommand returns { command, text }: one decision that simultaneously decides whether to mute and whether to remove words from the user's message. Those costs are nothing alike:

Decision Wrong in one direction Wrong in the other
Fire mute mic stays open (annoying) muted unintentionally — one tap to undo
Strip words command word ships to the agent (#456) the user's sentence is silently truncated

Muting is idempotent and trivially reversible; truncation is lossy. They should not share a threshold — but the return type forces it. That is why rule 2 exists: to protect "don't forget to mute" from truncation, the design gave up firing mute at all in that shape. A precision requirement on the expensive action suppressed the cheap, safety-critical one — and ADR 0001 M1 says the cheap one must always be reachable.

Fault 2 — the decision is made after the evidence has been discarded

const finalize = (msg: string) => {          // use-voice.ts:1136

A bare string. Yet every dictate() upstream carries timing — dictate({ type: "speech", text, at: now }) (:387), { type: "endOfTurn", at: now } (:448) — the VAD knows where the silences were, and the gate knows whether it already fired a command mid-capture.

All of it is thrown away before the only place that needs it. "push everything, mute" and "don't forget to mute" are near-identical strings and acoustically different events: one has a pause and a falling contour before the final word, the other does not. The pipeline had that signal and dropped it, so the matcher is asked to recover intent from the one representation that cannot carry it.

The proper design

1. A three-way verdict, not two

Replace {command, text} with { fire, strip, park }:

  • fire — liberal threshold. Any plausible mute signal mutes. Reversible, and M1 demands it.
  • strip — conservative threshold. Remove words only when certain.
  • park — the two disagree: fire the command, and instead of sending a possibly-truncated message, hand the text to the composer.

Park is the honest outcome for "probably a command, not sure", and it needs no new machinery: onRecoveredTextRef already exists and is wired in both consumers (InstanceDetail.tsx:287, CodingTab.tsx:195). It is the #175 recover contract — visible, editable, one tap to send — applied to a second kind of uncertainty.

This alone dissolves the dilemma behind rules 2 and 4. "don't forget to mute" → fire mute (cheap, reversible), park the sentence (nothing truncated, nothing lost, user sends it if they meant it). No guess has to be right.

2. Decide on evidence, not on a string

Pass a structured utterance to the decision — tokens with timings, plus two flags the system already has: did the gate fire mid-capture, and was there a pause before the final token. Then the rules become signals rather than special cases:

Signal Available today Meaning
gate already fired this command mid-capture yes (use-voice.ts:336-351) certain — it was a command
pause > ~400ms before the trailing token yes (VAD state) likely a command
token repeated yes (from the string) likely a command
token mid-sentence, no pause yes content

The gate-fired flag is the strongest and the cheapest: if the app already muted because of a word, that word is a command by construction, and it must not also be sent as text. That single fact resolves the live case in #456 with no heuristics at all.

3. Keep the rules pure and testable

None of this changes the property that makes this code good: the decisions stay pure functions in convo.ts / a new command-intent.ts, unit-tested. The change is the shape of the input (an utterance with evidence, not a string) and the shape of the output (three verdicts, not two).

Is this over-engineering?

No, on one condition: it is staged. The three-way verdict is a change to one return type and one call site, and it removes the forced trade-off that generated rules 2-4. The evidence-threading is bigger and should wait for a second forcing case.

Recommended order:

  1. Ship [bug] Repeating "mute" breaks it — one "mute" matches, two or more match nothing and get sent to the agent as a message #456's repetition-collapse now. It fixes a live defect, it is ~5 lines, it is safe. Ship it knowing it is tactical.
  2. Three-way verdict + park. The real fix. Removes the class rather than the instance.
  3. Thread the gate-fired flag. One boolean, converts the strongest signal from "inferred from text" to "known".
  4. Full timing evidence — only if a case appears that (1)-(3) cannot handle. It may never.

Steps 2 and 3 are where "proper design" lives. Step 1 is a patch, and calling it one is the point: shipping a patch knowingly, with the structural ticket already written, is different from shipping a patch as if it were a fix.

What I would not do

  • A wake word ("hey pags, mute"). Removes the ambiguity at the source and is what assistants do — but it is a product-level UX change with a learning cost, and it does not fit an app whose whole hands-free pitch is talking normally. Worth considering only if (1)-(3) still leak.
  • Ask the model to classify. A command that reaches the model has already failed to mute the microphone. Control must not depend on a round trip.
  • Keep adding matcher rules. Rule 5 is already foreseeable; the set will keep leaking because the input cannot support the decision.
  • Make truncation more aggressive to "solve" spillage. That trades a visible failure (a stray word in a message) for an invisible one (a silently cut sentence). Strictly worse.

Acceptance criteria — step 3 SHIPPED, step 2 is what remains (audit 2026-08-23)

Step 1 (#456's repetition-collapse) and step 3 (the gate-fired flag) are on main. Step 2 — the three-way verdict and park — is untouched, and it is the whole of this ticket.

  • A command that fired during capture never appears in the sent message. — 37b65248, #457 step 3. packages/sdk/src/voice/convo.ts:963-966: if (state?.fired) { const stripped = stripStopWord(text, commandPhrases(state.fired, words, lang)); if (stripped.ended) return { command: null, text: stripped.text }; }, with the reasoning at :950-962 — "if the app muted because of a word, that word WAS a command — known, not inferred … Here the expensive decision has EVIDENCE rather than a threshold." Tests: send-handoff.test.ts, turn.test.ts.

What remains — step 2, the three-way verdict

  • Firing a command and stripping text are separate decisions with separate thresholds. Today they are still one return value. matchVoiceCommand (packages/sdk/src/voice/convo.ts:~900-968) returns { command, text }, and the state?.fired branch at :963 is explicitly "the one place the two decisions are allowed to disagree" — a single carve-out, not the split.
  • An utterance that fires a command but cannot be confidently stripped is PARKED in the composer, never sent truncated and never silently dropped. Verified absent: grep -n "park" packages/sdk/src/voice/*.ts returns only two prose mentions (convo.ts:939, :975) and no verdict, no field, no call site. There is no command-intent.ts.
  • "don't forget to mute" mutes and parks the sentence. Today it does the OPPOSITE — convo.ts:925-935 deliberately keeps it a whole message and fires NOTHING, because with two verdicts the expensive rule has to be forced onto the cheap action: "stripping a trailing bare word would reintroduce that … silently truncating it AND muting is far worse than making the user pause before saying a one-word command." That comment IS the forced trade-off this ticket exists to remove.
  • mute-invariant.test.ts's guard table gains the new verdicts, so a future refactor cannot collapse them back into one. (packages/sdk/src/voice/mute-invariant.test.ts — grepped, no park today.)

Files: packages/sdk/src/voice/convo.ts (the verdict type + matchVoiceCommand), a new packages/sdk/src/voice/command-intent.ts if the decision is lifted out, packages/sdk/src/voice/use-voice.ts (the one call site + the composer hand-off, which already exists for #175's recover-to-composer contract), packages/sdk/src/voice/mute-invariant.test.ts.

Collides with #744 (merging the three recognizers) — both change convo.ts and use-voice.ts. Do this one first: it is a pure-function change with a decided shape, whereas #744 moves device ownership underneath it.

Step 4 (full timing evidence) stays out of scope, per the body above: "only if a case appears that (1)-(3) cannot handle. It may never."

Related: #456 (the tactical fix and the live evidence), #420 (the same finalize path; mute destroying the pending turn), #175 (the recover-to-composer contract this reuses), #228 (why the words must survive the command), ADR 0001 M1/M3 (mute always reachable; guards may raise the bar, never close the door), #438 (a rule-per-bug pattern is the signature this identifies).

Activity

  1. serge-ivo commented on Aug 8, 2026

    @serge-ivo
    ContributorAuthor

    Step 1 shipped (c95479d, #456). Step 3 shipped (37b6524). Step 2 is NOT implemented — it needs a product decision, stated below. Leaving this open on step 2. Both commits are live; CI and both deploys green, and firedDuringCapture is confirmed present in the deployed console bundle.

    Step 3, delivered — and it was better than "one boolean" suggested

    You called the gate-fired flag "the strongest signal and the cheapest". It is also the fix for something the code already claimed to do. At the gate's own dispatch:

    // … the audio already captured is still transcribed and sent, WITH THE COMMAND WORD STRIPPED BY FINALIZE (splitTrailingCommand). Latching here discarded the turn, so "run the tests, mute" muted and threw the request away …

    splitTrailingCommand does not strip it — its trailing branch is multi-word only, precisely so "don't forget to mute" stays a message. So the exact utterance the comment names muted the microphone and delivered the word "mute" to the model. Measured on the shipped planFinalizedTurn: "run the tests, mute" → { action: "send", text: "run the tests, mute" }.

    The new pass returns command: null on purpose — firing already happened, and re-reporting it would run it twice. That is your fault-1 asymmetry in miniature, and a reader tidying it into "return the command like every other branch" would fire mute twice a turn, so it says so.

    Bounded exactly as your table implies: relaxed for the command that fired and nothing else; trailing-only, so a mid-sentence false positive cannot cut a word out of the middle; exit deliberately does not set the flag (it tears the session down, so there is no turn left for a strip to matter to); and with no fired command every ordinary turn is byte-identical, asserted.

    Three statements in three callbacks, none load-bearing alone, so the wiring is asserted structurally in send-handoff.test.ts. The reset is the sharp one: per turn, not per session, or the flag outlives the turn that set it and the next utterance ending in "mute" is silently truncated — a stray word traded for a lost clause, the strictly worse direction by your own argument. Verified red on origin/main: 4 failures, the pure case plus all three wiring assertions.

    Step 2 — the decision I am not making

    The three-way verdict is a clean refactor. What needs signing off is what park does to the user, and I checked two things rather than assuming them:

    1. The M1 argument does not carry it. The issue says "a precision requirement on the expensive action suppressed the cheap, safety-critical one — and ADR 0001 M1 says the cheap one must always be reachable." M1 is not violated today. Mute is reachable in every phase by "mute" alone, by "mute mute" (since #456), and by touch — mute-invariant.test.ts asserts exactly that, per phase and now per phrasing at one, two and five repetitions. What step 2 adds is mute firing on an utterance that merely mentions the word. That is an improvement to argue on its own merits, not an invariant being restored.

    2. Parking costs the hands-free promise. A parked turn is "visible, editable and one tap from being sent" (ADR 0001 M5) — a tap, in the mode whose entire value is that the screen is down. Under step 2, any sentence containing a command word stops being delivered until the user picks the phone up. And ADR 0001 M5 already states the opposite default for the neighbouring shape:

    "The one exception is a mute arriving on the END of a request. 'Run the tests, mute' is a request plus a request for quiet, and #228 exists because latching there threw the request away."

    So the question is: for an utterance we cannot confidently classify, does the product prefer to deliver it and not mute (today), or to mute and hold it for a tap (step 2)?

    Both are defensible. Step 2 trades a rare unwanted mute for a routine screen touch, and #228 and ADR 0001 M5 both lean the other way — which is why I am not choosing it inside a fix for something else. If the answer is "park", the refactor is exactly as scoped here (one return type, one call site) and I will build it; it may also want an ADR 0001 amendment, since M5's exception is the paragraph it changes.

    Worth noting the pressure for step 2 is lower than when this was filed: step 3 resolves the live #456 case with no heuristics, and rule 4 (repetition) turned out to need a precision guard of its own anyway — folding reopened #385's collision, caught by #385's own tests. Your prediction that "rule 5 is already foreseeable" is holding; what is less clear is that park is rule 5's replacement rather than a different trade.

    Step 4 (full timing evidence) remains where you put it: only if a case appears that steps 1–3 cannot handle.

  2. added
    P3: laterDesign, strategy, or deferred pending a decision or demand
    enhancementNew feature or request
    voiceVoice / STT / TTS
    on Aug 18, 2026
  3. serge-ivo commented on Aug 23, 2026

    @serge-ivo
    ContributorAuthor

    PARTIAL — step 3 shipped in 37b65248; step 2 (the three-way verdict + park) is untouched. Body updated so the remaining work is pickable without re-deriving it.

    Delivered-work audit, 2026-08-23, against main.

    Verified: the state?.fired branch at packages/sdk/src/voice/convo.ts:963-966 and its rationale block at :950-962; that the repetition-collapse rule (stripRepeatedTail, :970-990) is present from #456; that "don't forget to mute" is still handled by the multi-word rule at :925-935, which fires nothing and sends the whole sentence — i.e. AC3 is not merely unbuilt, current behaviour is its opposite; that grep -n "park" packages/sdk/src/voice/*.ts returns two prose mentions and no verdict; that no command-intent.ts exists.

    Inferred: nothing.

    One thing worth naming for whoever takes it: 37b65248's own comment argues the shape this ticket asks for — "firing and stripping have wildly asymmetric costs (an unwanted mute is one tap to undo; a truncated sentence is lost), so one verdict for both forces the expensive rule onto the cheap action" — and then makes a single evidence-backed exception rather than splitting the verdict. That is the right order (evidence before threshold), and it means step 2 now has one worked example to generalise from rather than none.

  4. serge-ivo commented on Sep 26, 2026

    @serge-ivo
    ContributorAuthor

    Step 2 shipped: c19cc47a. With step 1 (c95479d, #456) and step 3 (37b65248), everything in scope is done; step 4 stays out of scope as the body says.

    The product decision, and who made it

    The open question on this thread was: for an uncertain utterance, deliver it and don't mute, or mute and hold it for a tap? It was never answered by the owner. It was decided by the Pilot, on the strength of this issue's own acceptance criterion ("'don't forget to mute' mutes AND parks the sentence"): park-and-mute. ADR 0001 M5 now says so in place, including that the Pilot made the call and what to change together to reverse it. If the owner prefers deliver-and-don't-mute, it's a small reversal: the park verdict in splitTrailingCommand, the park branch in finalize, and that M5 paragraph.

    What changed

    splitTrailingCommand returns a verdict (type and pure rule in the new packages/sdk/src/voice/command-intent.ts):

    Two bounds, because firing on weaker evidence needs them:

    • mute only. It's the one command a wrong firing costs a tap to undo. The others aren't cheap:
      • unmute would open the mic on "remind me to unmute later";
      • exit and next end or move the session;
      • repeat replaces the turn ("don't repeat yourself").
    • Negation guard. "don't mute", "do not mute" and "never mute" fire nothing.

    A user's binding of the word to something else still wins (#385), and nothing parks while already muted.

    Plumbing:

    • turn.ts: FinalizedTurn gains { action: "park", text, command: "mute" }.
    • use-voice.ts finalize: mutes with pendingTurn: "send" (the turn is in hand), then hands plan.text to the composer, before and instead of the send path.

    Tests

    mute-invariant.test.ts, 6 new:

    Watched failing: the park tests fail on the old code, and removing the negation guard fails the adversarial test.

    Updated:

    • convo.test.ts and turn.test.ts assert the verdict field.
    • Their "don't forget to mute" → send assertions now expect park.
    • turn.test.ts's legacy-chain oracle gained the park branch.

    Results

    • Voice suite: 23 files, 740 tests passed.
    • Full suite: 12,786 of 12,787 passed. The one failure was a browser-runner Playwright test that timed out at 120s under load; it passed 7/7 in isolation.
    • Checks: SDK, SDK tests, console and coder-web typechecks, biome, docs:drift and the file-size ratchet are clean. Pins: convo.ts +7, use-voice.ts +8, with reasons recorded.

    Not verified

    Nobody has spoken "don't forget to mute" into a live hands-free session since this change. The behaviour is proven on the decision functions and the finalize wiring, not observed in a browser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3: laterDesign, strategy, or deferred pending a decision or demandenhancementNew feature or requestvoiceVoice / STT / TTS

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions