Skip to content

Application Runner: diagnose a CLI that exits without using the browser bridge #975

Closed

Description

@serge-ivo

Parents: #947, #953. Partly shipped: #971 exposed the browser-event trace, the policy
gate and the supervisor checkpoints over MCP (application_run, application_run_supervision,
tailoring_run, application_runs), so the third bullet below is now only about the new
diagnostic
and the Kanban surface — the trace/gate/checkpoint half is done.

What remains is the part #971 could not do: the CLI's own stdout never crosses the runner
contract (LocalApplyStatusResponse carries state/pause/events/lastSeq/result; only
result.summary is derived from that output), so a bounded, privacy-safe diagnostic requires a
runner-contract change and a CLI release, plus a decision about what may be retained —
CLI prose can contain profile values, which is why the contract excludes it today.

Parents: #947, #953

Observed on production

Application 435d31c8-01a4-48d7-87f2-08584fa6d02e reached the new Runner. Tailor produced validated materials. The local Claude CLI authenticated and exited 0 after ~51 seconds, but emitted no browser navigation, supervisor checkpoint, field fill, upload, or review event. Runner correctly ended blocked: incomplete; the reason is not actionable from MCP because raw CLI stdout/stderr never crosses the runner contract.

Required behaviour

  • Treat zero bridge use by an application CLI as a distinct, actionable failure (bridge_unused or equivalent), not generic incomplete.
  • Persist a bounded, privacy-safe diagnostic: redacted structured cause/engine final message or owner-only redacted output tail. It must never retain cookies, credentials, secrets, typed form values, or source document text.
  • Expose the diagnostic, browser-event trace, policy gate, and supervisor checkpoints through API, Kanban, and MCP.
  • Add real connected-machine acceptance coverage for Claude and Codex application runs, including bridge launch, checkpoint, field fill, pause, unavailable listing, and final submit authorization.

Acceptance

A CLI that exits before its first bridge call reports a specific cause and safe diagnostic. A healthy Claude and Codex run emits browser and checkpoint events. The owner can diagnose either result from MCP without inspecting a machine terminal.

Activity

  1. added
    observabilityA displayed value the code cannot produce, or that means something other than its label
    runnerpackages/browser-runner + packages/cli — ships only via an npm publish
    on Oct 8, 2026
  2. serge-ivo commented on Oct 8, 2026

    @serge-ivo
    ContributorAuthor

    Implemented in 1c0fddf.

    1. Zero bridge use is now its own reason

    bridge_unused (both vendored contract copies). Zero bridge calls is a different failure from stopping partway — different cause, different fix — and reporting incomplete for both is why that 51-second run couldn't be explained.

    2. The diagnostic, and the privacy decision behind its shape

    It rides the result the platform already stores: cause (closed enum: bridge_unused · engine_exited_nonzero · timed_out · no_engine_output), bridgeCalls, engineExit, activeMs, pages, filled, and signals — ids this platform defines, matched by the runner against its own output, never the line that matched. The most valuable is approval_policy_blocked: that's #952's live failure, invisible from the cloud and fixable by configuration. The questions the owner reads are built from those ids, so the advice follows the observation instead of guessing.

    I did not ship an output tail, redacted or otherwise — the issue offered that as an option, and it isn't safe. redactText removes credential shapes; it cannot remove the owner's own résumé prose, their typed answers, or a quoted source document, all of which a CLI legitimately prints while it works. So no new free-form channel was opened. The engine's closing sentence continues to cross as summary, where it already did — redacted and capped — which gives the operator the CLI's own words without adding a second path for content.

    The cloud parser accepts the closed vocabulary only: a runner that sends outputTail, a note, or its own signal names gets them dropped. Asserted from both ends — the runner never builds prose, and the cloud never stores it.

    3. Surfaces

    The diagnostic sits on the run result, so #971's application_run returns it over MCP unchanged, and the board card reads that same record rather than a second copy. No new MCP tool, no version bump.

    4. Acceptance coverage, both engines

    describe.each(["claude", "codex"]) on the connected runner:

    Phase What's asserted
    Bridge launch Claude via its strict MCP config file, Codex via -c mcp_servers.… flags — they differ, and both must really get it
    Supervisor checkpoint a fill before the cloud releases the page is refused
    Field fill grounded in the owner's approved source
    Pause a value it doesn't have stops the run and asks, instead of inventing one
    Authorized submit reaches the page under #973's gate, and is recorded attempted-not-confirmed
    Unavailable listing its own run per engine — reporting it ends the application, so it cannot coexist with a submit

    Plus the #975 cases: zero calls on exit 0, the approval-policy signal, no output at all, a non-zero exit, and a diagnostic carrying none of the CLI's prose (résumé text, a typed answer, a cookie and an API key all asserted absent, with the key set pinned to exactly seven fields).

    Two things deliberately left alone

    • A run that did real work carries no diagnostic. None of the four causes describes it, and inventing one would make "there is something to diagnose" meaningless. What it did is already on the result and the trace.
    • The vendored contract copies stay byte-identical — that invariant is what caught my cloud-only edit before it shipped.

    CLI 0.4.84 — the runner changed, so it has to publish for the new reason and diagnostic to appear on a machine.

    Verification: full suite 857 files / 14,461 tests green; biome (exit 0), all three tsconfig.test.json typechecks, and all eleven custom checks clean. One ratchet pin was raised deliberately with its reason (the card's diagnostic field in the console type mirror); a second apparent cut turned out to be unreachable dead code and was removed instead of pinned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    observabilityA displayed value the code cannot produce, or that means something other than its labelrunnerpackages/browser-runner + packages/cli — ships only via an npm publish

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions