Skip to content

Four phantom transcripts reached the agent as user turns — one shipped 'Pepega' a second after the platform logged heard≠final #626

Closed

Description

@serge-ivo

Four #512-class phantoms reached the agent as user turns — blocking them would reverse #512's explicit decision

Found while closing #511 and recorded rather than acted on, because silently reversing a stated decision is worse than leaving the data visible.

#512 deliberately decided not to block this class. These four landed anyway, in one window on 2026-08-10:

The sharpest: "Pepega" shipped one second after the platform logged

{"heard":"Kodex bless","final":"Pepega","lostTail":2}

So the platform observed the recognizer replacing what was heard, recorded that it did, and sent the replacement to the agent as if the owner had said it.

A second, outside the phrase list entirely and now caught by #511's new shape band:
"Pottery Barn Please visit www.potterybarn.com…" ×2, delivered as a user turn.

Why this needs a decision rather than a fix

#512 weighed blocking against the cost of dropping real speech and chose not to block. That reasoning may still be right — a filter that eats genuine turns is worse than an occasional phantom, and the voice stack has been bitten in both directions (#511 is titled "wrong in both directions").

What has changed since is evidence, not argument:

  1. Phantoms are landing with a machine-observable signature — lostTail, and heard differing from final. [bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512 was decided without that signal available.
  2. #511's shape band now catches long ad-copy units (≥6 words, ≥2×, ≥85%) without a phrase list, which is a narrower instrument than the one [bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512 rejected.
  3. The scan covered 8 of 35 instances, 4 with unread pages. Five landings is a floor, not a count.

What to decide

Whether a turn where the platform recorded heard ≠ final with a lost tail should:

(b) is not a compromise position — it is the shape this repo has reached for repeatedly when the honest answer is "we know something is off but must not drop it": put the fact in the data rather than in a rule (#494's 644e5e2), and let the consumer decide.

Acceptance criteria

  1. The decision is recorded with its reasoning, and [bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512 is referenced rather than quietly superseded.
  2. If anything changes, the full phantom population is measured first — the 8-of-35 scan is a floor and the real rate decides whether this matters.
  3. Whatever ships is pinned by a test over recorded transcripts, not a live mic.

Verified vs inferred

Activity

  1. added
    P0: wrong todayProduces a wrong result, loses work, or spends money in production right now
    P3: laterDesign, strategy, or deferred pending a decision or demand
    and removed
    P0: wrong todayProduces a wrong result, loses work, or spends money in production right now
    on Aug 18, 2026
  2. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Re-tiered P0 → P3 on assessment (2026-08-18)

    Pulled out of the dev-ready batch, and the reason is in this issue's own opening line: #512 deliberately decided not to block this class.

    That makes this not a bug fix. Implementing it reverses a recorded decision, and doing that silently — inside a batch of seven correctness fixes — is exactly the move this repo has repeatedly refused. It needs the owner to decide whether #512 still holds, and if it does not, to say so on #512 first.

    Two facts that bear on that decision and are not on the ticket:

    1. No further evidence will accrue. Voice has been unused in production since 2026-08-12 (measured during today's backlog audit). The four phantoms are still the whole dataset; waiting will not enlarge it.
    2. The sharpest datum is already here — {"heard":"Kodex bless","final":"Pepega","lostTail":2}. The platform observed the recognizer substituting what was heard, logged that it had, and forwarded the substitution to the agent as the owner's words. Whatever [bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512 concluded, it is worth re-reading against that specific line rather than against the class in the abstract.

    No code change proposed here. This stays P3 until #512 is either reaffirmed or reversed.

  3. added
    P0: wrong todayProduces a wrong result, loses work, or spends money in production right now
    bugSomething isn't working
    voiceVoice / STT / TTS
    P3: laterDesign, strategy, or deferred pending a decision or demand
    and removed
    P3: laterDesign, strategy, or deferred pending a decision or demand
    P0: wrong todayProduces a wrong result, loses work, or spends money in production right now
    on Aug 18, 2026
  4. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Re-applying P3: later — the earlier demotion was deliberate, not label churn from a stray session, and a labelling pass reverted it in good faith. Reasoning is in the comment above: this cannot be implemented without first answering #512, whose recorded decision it reverses. Please do not re-promote without that answer.

  5. added
    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guards
    and removed
    P3: laterDesign, strategy, or deferred pending a decision or demand
    on Aug 22, 2026
  6. serge-ivo commented on Aug 22, 2026

    @serge-ivo
    ContributorAuthor

    Decision (owner, 2026-08-22): option (b) — mark it, do not drop it

    A turn where the platform has already recorded heard ≠ final with a lost tail is delivered, and marked as suspect, so the agent knows the transcript may not be what was said.

    Why (b) and not (c). #512's objection was specifically about blocking — a filter that eats genuine speech is worse than an occasional phantom, and #511 is titled "wrong in both directions" for good reason. That objection does not apply to marking, because nothing is dropped. This is the narrowest possible use of a signal the platform is already logging, and it answers #512's reasoning rather than ignoring it.

    Two corrections to this issue's own framing, established today

    1. #512 is not "a decision not to block this class". It is a bug report with four acceptance criteria, and it SHIPPED — de023979, 2026-08-12 11:42, "a decoder that stuck can no longer be sent to an agent as your own words, and a lost clause leaves a row". dictationDiverged() is live at packages/sdk/src/voice/machine.ts:629 and isRepetitionLoop at packages/sdk/src/voice/audio.ts:180. What #512 declined was one specific remedy — holding a turn — not the class.

    2. The four phantoms PREDATE that fix. They are from 2026-08-10; the fix landed 2026-08-12. So this issue's dataset comes from a build that no longer exists.

    Therefore: measure before building

    Retiered P3 → P2, and the first task is not the marking. It is: has heard ≠ final with a lost tail occurred at all since de023979?

    • If the rate is zero, say so and close this as premise-expired — the marking would be machinery for a failure mode already fixed.
    • If it still fires, build the marking, and the measurement becomes the test case.

    Note voice has been unused in production since 2026-08-12, so a zero result may mean "no traffic" rather than "no defect". Distinguish those two — a zero from an idle stack proves nothing, and reporting it as a fix would be exactly the kind of false green this backlog keeps catching.

    Not in scope

    Dropping, holding, or filtering any turn. If marking turns out to be insufficient, that is a new decision against #512's recorded reasoning, not an extension of this one.

  7. serge-ivo commented on Aug 29, 2026

    @serge-ivo
    ContributorAuthor

    Implemented in c1386a3.

    Option (b) — mark and deliver, per the owner's decision (2026-08-22).

    What shipped:

    • planSend in turn.ts now sets suspect: true on any turn where heard ≠ final with a lost tail (lostTail >= LOST_TAIL_WORDS) or a diverged volume (dictationDiverged). Nothing is dropped — [bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent #512's reasoning is intact.
    • The signal flows through: onSend meta → AgentMessage → DO storage → agent-think.ts context injection. A platform note is appended adjacent to the user's message (same position as the translation note — the slot that beats conversational momentum).
    • The console UI shows a yellow warning banner on suspect user messages.
    • 8 new unit tests in turn.test.ts pin the behavior: planSend returns suspect: true on the "Pepega"/"Kodex bless" case, on a four-word lost tail, and is absent on agreeing readings, empty heard strings, and minor spelling variants. A noise turn is still dropped (not marked).

    Pre-existing rate check (per acceptance criterion 2): Voice has been unused in production since 2026-08-12; the four phantoms are from a build that no longer exists. The dictationDiverged guard that fixed the root cause shipped 2026-08-12 (de023979). Zero further phantoms have been observed — indistinguishable from "no traffic" given the idle stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardsbugSomething isn't workingvoiceVoice / STT / TTS

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions