Skip to content

coding_session_capture drops the fields that disambiguate 'idle', advertises two values it cannot emit, and coding_diagnostics scores a refusing engine 0 issues #593

Closed

Description

@serge-ivo

The engine plane cannot be read truthfully: capture strips the disambiguating evidence, advertises impossible values, and diagnostics calls a refusing engine healthy

Three defects on one question — is the engine actually working? — which is the half of #580 that was never closed.

1. MCP strips exactly the fields that make idle falsifiable

workers/mcp/src/index.ts:854 projects the response to {sessionId, runState, pane}, dropping runnerConnected, alive, ready, authPrompt.

The API returns runState: "idle" for three different worlds and disambiguates with precisely those siblings: routes/coding.ts:234 (no connection) and :240 (runner never answered) both collapse to idle. So over MCP, "the engine is idle", "the runner is gone" and "the probe failed" are one indistinguishable answer.

2. The tool advertises two values it cannot emit

index.ts:840 — "Also returns run state (idle/working/offline)."

The runner's vocabulary is "idle" | "thinking" | "responding" (packages/browser-runner/src/coding/runtime.ts:82). working is not a member, and offline is manufactured only by the timeline route (routes/coding-feed.ts:82), never by /capture.

coding-feed.ts:74-85 has the corrected vocabulary (ended/offline/unknown) with an explicit comment that unknown must not collapse into idle — so two surfaces sit over one probe with two vocabularies, and the older one is the one MCP exposes.

3. coding_diagnostics reports 0 issues for a stalled engine

Measured live on bd43f4de: healthySessions: 1, issueCount: 0, issues: [], live.alive: true, runState: "idle" — while the captured pane's last two lines are You've hit your weekly limit · resets Aug 17 at 4pm and [error].

The same instance's board says "Needs you" and its run says engine_limit. Both are right. The issue rule (lib/coding-diagnostics.ts:245-251) fires only on runner/relay faults, so an engine that is up but refuses to work is invisible to the tool whose stated purpose is "debug why sessions are offline or stuck".

Why this misleads into a wrong action

An owner debugging a stuck agent is sent to the tool named for it, told everything is healthy, and given a runState that cannot distinguish "thinking hard" from "the machine is gone". This is the exact question that cost 4.35 hours on 2026-08-15.

Acceptance criteria

  1. coding_session_capture carries enough to distinguish engine-idle, runner-absent and probe-failed. If payload size is the reason for the projection, say so and choose deliberately — index.ts:854 currently drops them silently.
  2. The description names only values the code can emit. A guard asserts the advertised vocabulary equals the emitted one, denominator stated per ADR 0002 over every tool that publishes a state enum — this is the third such mismatch found today.
  3. One vocabulary over one probe: coding-feed.ts:74-85 and routes/coding.ts:234,240 agree, and unknown never collapses into idle.
  4. coding_diagnostics reports an engine that is up but refusing as an issue, with the reason. issueCount: 0 beside a board card reading "Needs you" is the defect.
  5. Red demonstrated on the live case: a weekly-limit pane must not yield issueCount: 0.

Verified vs inferred

  • Verified: every file:line; the live bd43f4de values; the runner's three-member union; the coding-feed.ts comment.
  • Inferred: that the projection at index.ts:854 was for payload size. Not stated in the code — AC1 should establish the intent before widening it.

Activity

  1. serge-ivo commented on Aug 15, 2026

    @serge-ivo
    ContributorAuthor

    Shipped in 927f963 (on main, CI + API/MCP/Host deploys all green).

    AC1 — the projection's intent, established before widening it

    The issue inferred the projection at index.ts:854 was for payload size and asked that the intent
    be established first. It was not size.

    git log -L 854,856:workers/mcp/src/index.ts puts the three-field shape in the tool's original
    commit — d63333d, 2026-06-28, "read live terminal output + run state". So:

    • alive / ready / runnerConnected were already on the API route from f0785b1 (2026-06-26),
      two days before the projection was written. They were available and simply not carried.
    • authPrompt arrived on 2026-08-04 (260dec1, 78383be) — five weeks after. It could not
      have been dropped; nobody revisited the tool.
    • The decisive evidence against a size motive: the projection kept pane — up to 64 KB — while
      dropping four booleans. Nothing in the commit or the code mentions size.

    coding_session_capture now returns runnerConnected, alive, ready and authPrompt beside
    runState and pane.

    AC3 — one vocabulary over one probe

    lib/coding-run-state.ts (new) is coding-feed.ts's comment made executable and shared:
    idle|thinking|responding is the engine's own union (derived and guarded against
    runtime.ts:82), and ended|offline|unknown is what the platform reports when no engine was
    reached. routes/coding.ts:234 and :240 now answer offline and unknown instead of idle,
    and coding-feed.ts resolves through the same function. An unrecognised word from a runner becomes
    unknown rather than passing through, so a value no engine can emit cannot reach a client by
    riding the snapshot.

    Worth noting: this removes a console workaround. agents/coder/web/src/repo-status.ts exists
    partly because /capture reported a vanished machine as an idle engine, so a repo row read "Ready",
    in green, under the tab's own "run pags up" banner. Its reconciliation is unchanged and still
    needed (runnerOnline is on its own 10s timer), but it already handled state === "offline", so
    the honest value makes it more correct, not less. Its comments no longer document a contract that
    has since changed.

    AC2 — the guard, and its denominator

    Two mechanisms, because a guard alone would not have prevented this:

    1. Generation. The description is now RENDERED from the constant (runStateSentence()) — the
      shape list_instance_tools adopted for its tiers after list_instance_tools documents 2 of the 4 tier values it returns, and its default response is 117 KB — enough to exceed a host's limit on one instance #569. A sentence built from the
      vocabulary cannot name a value the code does not have.
    2. Inventory. state-vocabulary.test.ts sweeps every registered tool (136, driven through
      a real registration) for enum-shaped claims and finds 12 value-set claims. Each must be
      either backed by a vocabulary derived from the emitting source, or recorded in UNBACKED_CLAIMS
      with the reason. That inventory is a ratchet, not an allowlist: a stale entry fails too, so
      it cannot rot.

    Denominator stated per ADR 0002: 136 tools swept, 12 value-set claims, 1 backed vocabulary, 9
    inventoried.
    G4 evidence: the scanner is proven on the literal defective string
    (idle/working/offline), which it reduces to working as unemittable.

    Honest about scope: 9 of the 12 claims are emitted by code the MCP worker cannot import
    (feedback status, trace source, write-consent, board view, …). They are inventoried, not verified,
    and the file says so — pretending otherwise would be the "certifies ground it never walked" failure
    ADR 0002 exists for. Backing them is a follow-up, and each entry names where its source lives.

    AC4/AC5 — a refusing engine is an issue

    refusingEngineIssue reports an engine that is up but not working, and healthySessions now means
    able to work rather than merely running — the two numbers a reader checks first were both wrong
    in the same direction.

    One deliberate deviation from AC5's wording. The verdict comes from the run, not the pane.
    engine_limit is already a first-class waitingReason (#541), so reading it is an observation;
    grepping a terminal buffer for "limit" would be a guess about free text written by whichever CLI
    the user chose — and the diagnostics route only receives paneLines (a count), not the pane text,
    so a pane rule was not even available without changing the runner protocol. The live bd43f4de
    case had both signals, and it is pinned as a test: a run parked on engine_limit can no longer
    yield issueCount: 0. A hand-driven session with no run still reports nothing, because inventing
    a verdict for it would be a guess.

    No MCP_SERVER_VERSION bump: the surface fingerprint excludes descriptions and covers no response
    body, and this tool declares no outputSchema. conformance.test.ts stays green on the 0.1.4 lock.

    Verified / not verified

    Verified: full suite green (9014); 14 new API-side tests + 8 guard tests; typecheck, biome,
    file-size (pin raised deliberately, +4 for the projection, with the reason recorded), bare-catch,
    surface-lock, migrations, test-isolation, design-tokens, agents-allowlist, qa-config, docs-drift;
    Playwright 405 passed (chromium + webkit); the four script gates re-run against the exact commit in
    a throwaway worktree; CI and all three deploys green; both /health endpoints live (MCP still
    reports 136 tools — no tool added).

    NOT verified in production: the MCP endpoint is OAuth-gated and I hold no session token, so I
    could not call coding_session_capture live to see the four new fields, nor coding_diagnostics
    on bd43f4de to see it stop reporting issueCount: 0. Leaving this open for that check — it
    needs an authenticated MCP call, which the pags-live agent can make.

  2. serge-ivo commented on Aug 15, 2026

    @serge-ivo
    ContributorAuthor

    Verified live — coding_diagnostics now reports the refusing engine

    Read from production 2026-08-15 on bd43f4de (Chess coder 2), the instance this issue measured:

    before after
    issueCount 0 1
    healthySessions 1 0
    issues [] one warn

    The issue text:

    "Session 5f88790b (apps/chess-academy): the engine is up but not working — the run is parked waiting for a person — outcome: waiting — The coding CLI is still reporting its own usage limit and the run has used its 6 hours of waiting. You've hit your weekly limit · resets Aug 17 at 4pm (Australia/Melbourne)"

    fix: "Answer the handoff on the board, then the run continues"

    Cause and the action, on the tool whose stated purpose is debugging stuck sessions. The AC5 deviation recorded in 927f963 — deriving the verdict from the run's waitingReason rather than from the pane, because the diagnostics route receives only paneLines (a count) — produces the right answer here and is visible in the wording ("the run is parked", an observation, not an inference about the terminal).

    The session's live block reads alive: true, runState: "idle" alongside it, so the two facts now coexist without contradiction: the runner is up, the engine is idle, and the run is parked. That is the distinction #580 was filed over, readable from one call.

    Still unverified on this issue

    coding_session_capture answering offline / unknown instead of idle is not exercised here — this runner is connected, so the path that used to collapse runner absent and probe failed into idle was not reached. It needs a disconnected runner to confirm, which is a deliberate act on the owner's machine rather than something to manufacture.

    Verified vs inferred

    • Verified: every value above read from the deployed tool.
    • Inferred: nothing.
  3. added
    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guards
    bugSomething isn't working
    backendBackend / Worker / API work
    mcpMCP server surface — the operator's instrument panel, not the product
    coderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, Overseer
    observabilityA displayed value the code cannot produce, or that means something other than its label
    on Aug 18, 2026
  4. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Audit verdict — DELIVERED, all five criteria, re-verified against main and production today

    Re-read the code, not the comments. Per criterion:

    AC1 — the siblings that make idle falsifiable are carried. workers/mcp/src/coding-tools.ts (the tool moved out of index.ts) now declares and returns runnerConnected, alive, ready, authPrompt beside runState and pane, with the projection's history recorded in the comment above the handler. The "was it payload size?" question AC1 asked to settle first was answered in 927f963f's comment with git log -L: it was not, and the decisive evidence is that the projection kept a 64 KB pane while dropping four booleans.

    AC2 — the description names only emittable values. runStateSentence() (workers/mcp/src/state-vocabulary.ts:135-137) RENDERS the sentence from CODING_RUN_STATES, so the string cannot name a value the constant does not hold. workers/mcp/src/state-vocabulary.test.ts sweeps every registered tool for enum-shaped claims and requires each to be backed or recorded in UNBACKED_CLAIMS — a ratchet, so a stale entry fails too.

    AC3 — one vocabulary over one probe. workers/api/src/lib/coding-run-state.ts:49 is the single union (ENGINE_RUN_STATES + PROBE_RUN_STATES), and all three readers resolve through resolveRunState:

    • routes/coding.ts:242 → offline (no connection), :259 → unknown (probe failed), :332-338 normalises the runner's own word after the spread, so an unrecognised value becomes unknown rather than riding through;
    • routes/coding-feed.ts:146,148,150 → the same function.

    Measured live today, GET /v1/instances/{bd43f4de,3cbb1d1d}/coding/timeline?limit=1 → "runState": "ended". That is a PROBE_RUN_STATES member reaching a client; on the code this issue was filed against, both would have read idle.

    AC4 — a refusing engine is an issue. refusingEngineIssue (lib/coding-run-state.ts:135-153) is called from routes/coding-diagnostics.ts:274-281 for every active session, and derives the verdict from the run's waitingReason rather than from pane text.

    AC5 — red demonstrated on the live case. Recorded on this issue 2026-08-15: bd43f4de went issueCount 0 → 1, healthySessions 1 → 0, with the reason and the fix in the message.

    Two things to carry forward rather than lose in the close

    1. AC4's fix has a follow-up defect, already filed: coding_diagnostics calls a healthy session "up but not working" — refusingEngineIssue reads park columns off a CLOSED run, the guard its sibling work-report.ts has #654. refusingEngineIssue lacks the run.status !== "running" gate its sibling lib/work-report.ts:288 has, so a CLOSED run that ended needs_human still reads as a park. That is the inverse over-shoot, and it is tracked separately — it is not a reason to reopen this.
    2. The offline/unknown capture paths remain unexercised in production. The 2026-08-15 comment said so and it is still true: measured today, every coding session across all 20 coding instances is ended and the runner is connected, so neither branch is reached. Unit-covered, not production-observed. Recording that rather than implying otherwise.

    Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardsbackendBackend / Worker / API workbugSomething isn't workingcoderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, OverseermcpMCP server surface — the operator's instrument panel, not the productobservabilityA displayed value the code cannot produce, or that means something other than its label

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions