Skip to content

[bug] #459's failure-honesty prompt block is pinned by no test — the one link in the not-stalled chain with nothing asserting it #620

Closed

Description

@serge-ivo

Found while verifying #459's acceptance criteria against the deployed code, not from a live incident.

The claim

#459 AC4 — "the honesty rule covers asserted failures, not only asserted successes" — is delivered
in the prompt
and pinned by nothing. 8f5afaa added the sentences; no test reads them, so a
later prompt edit removes the guard silently.

Verified

workers/api/src/agent-think.ts:707-711, the text as it stands today:

The same rule covers FAILURE: never assert a run is stalled, blocked, stuck or dead unless a tool
result says so. The run report states that verdict explicitly — quote it. If it says NOT stalled,
the run is not stalled however slow the counter looks; one instruction is a whole engine turn and
can take many minutes. Calling running work stuck is as wrong as calling failed work done.

The rationale sits at :702-706. And:

$ grep -rn "covers FAILURE\|stalled, blocked, stuck" workers/api/src --include="*.test.ts"
(no matches)

So the block can be reworded or deleted and every gate stays green.

Why this one is worth pinning rather than trusting

The neighbouring facts it depends on are pinned, which is what makes the asymmetry visible:

The prompt sentence is the one link in that chain with no assertion behind it, and it is the link
that does the work: the platform computes NOT stalled correctly and it is the prompt that tells
the model to quote it rather than form its own view. #459's incident was not a wrong verdict — the
verdict was right and describeLoopRun did not say the run was stalled. The model filled a silence.

What to do

A source-level guard over agent-think.ts asserting the failure-honesty block is present, in the
shape this repo already uses for prompt prose (prompt-claims.test.ts reads agent-think.ts for
exactly this kind of claim). It should assert the load-bearing clauses rather than the whole
paragraph, so a rewording that keeps the meaning does not fail:

  • that it forbids asserting stalled/blocked/stuck/dead without a tool result,
  • that it tells the model to QUOTE the report's verdict,
  • that it says one instruction can be a whole engine turn.

Per ADR 0002 G1, it must also assert it read a plausible prompt — a guard that greps a file it
failed to parse reports a clean tree.

Why this is filed rather than fixed

agent-think.ts was checked out by another lane during the #459 verification, and a guard over a
file someone else is actively rewording would be written against text about to change. It is a test
file addition only — no production change — so it is cheap whenever that file is free.

Acceptance criteria

  1. A test fails when the failure-honesty clauses are removed from agent-think.ts.
  2. It survives a reasonable rewording that preserves the meaning.
  3. It asserts the size of what it read (G1), so a moved prompt block fails as "stopped measuring"
    rather than passing.

Not claimed

That the model obeys it. #459's AC2 ("an agent asked about a run under STALLED_AFTER_MS does not
describe it as stalled") is a claim about a live turn and is not verifiable from code or tests — it
needs a machine running pags up with a real running coding run to ask about. That is #459's
remaining open item, and this ticket does not close it.

Activity

  1. added
    bugSomething isn't working
    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guards
    backendBackend / Worker / API work
    testingGuards, test doubles and coverage — not a product defect
    on Aug 15, 2026
  2. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Audit — still LIVE. Citation exact, and the gap is unchanged.

    workers/api/src/agent-think.ts:707-711 still carries the block verbatim:

    		" The same rule covers FAILURE: never assert a run is stalled, blocked, stuck or dead unless a" +
    		" tool result says so. The run report states that verdict explicitly — quote it. If it says…"
    

    And it is still the only place that string exists:

    $ grep -rln "never assert a run is stalled" workers/api/src
    workers/api/src/agent-think.ts
    

    One file, zero test files. The link in the not-stalled chain with nothing asserting it is still unasserted.

    Leaving open, labels unchanged.

  3. serge-ivo commented on Aug 29, 2026

    @serge-ivo
    ContributorAuthor

    Fixed in afeffcf. Section 7 added to workers/api/src/lib/prompt-claims.test.ts — four tests read agent-think.ts via promptTextOf and assert:

    1. G1 guard: file has ≥ 700 lines (a parse failure or replaced file reports "stopped measuring")
    2. The prohibition: stalled.*blocked.*stuck + tool result present in string literals
    3. The quote instruction: quote it or states that verdict explicitly
    4. The engine-turn explanation: one instruction is a whole\s+engine turn (\s+ handles the promptTextOf boundary space between concatenated literals)

    All 10,144 unit tests pass; only the pre-existing pdf-lib tests fail. Every other gate (biome, typecheck, scripts/check-*, docs:drift, openapi-coverage) is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardsbackendBackend / Worker / API workbugSomething isn't workingtestingGuards, test doubles and coverage — not a product defect

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions