Skip to content

The runner cuts every engine tool result to 240 characters, so no instruction can return file contents to the Pilot — and the prompt tells it to keep trying #700

Closed

Description

@serge-ivo

The runner cuts every engine tool result to 240 characters, so no instruction can return file contents to the Pilot — and the prompt tells it to keep trying

What the owner saw

Live run on bd43f4de-ef35-4051-bdec-43f8571414a1, session
csess_6ce3627a-3c3f-439a-90f3-ff4a6de3167a, 2026-08-17. Twelve of the run's sixteen
instructions
(seq 9286–9297) were the Pilot trying to get the raw text of one file,
web/e2e/helpers.ts, out of the engine. Each is a full BYOK Claude decision round. Verbatim,
from coding_timeline:

seq what the Pilot sent
9286 cat web/e2e/helpers.ts && echo "---SPLIT---" && cat …full-game.spec.ts
9287 "Show me the full contents … one at a time. First: cat web/e2e/helpers.ts"
9288 "Show me the raw contents of helpers.ts lines 1-100: sed -n '1,100p'"
9289 "Just show the raw terminal output without summarizing or interpreting it."
9290 "…no interpretation or summarizing: cat -n web/e2e/helpers.ts | head -60"
9291 "…print every character … If the output is being truncated by your tool, instead write … to a temp file"
9292 "The CLI is summarizing file contents rather than showing raw output. Let's work differently."
9293–9297 five more, same shape, on the spec files and again on helpers.ts

The owner's reading of step 9292 — "the CLI is summarizing" — is what the evidence in front of
the Pilot supports. It is not what happened.

The mechanism: the runner truncates, not the CLI

Claude Code returns the whole file over stream-json. packages/browser-runner/src/coding/headless.ts:680
renders each tool_result into the transcript through toolResult(), and :910-919 is that
function:

function toolResult(content: unknown): string {
	let text = "";
	…
	text = text.replace(/\s+/g, " ").trim();
	return text.length > 240 ? `${text.slice(0, 240)}…` : text;
}

Two losses, both before the pane exists:

  1. 240 characters. ~3 lines of code, whatever the command asked for.
  2. \s+ → " ". Every newline and indent is collapsed, so even those 3 lines arrive as one
    unbroken line. sed -n '1,50p' and cat -n | head -60 produce byte-identical shapes at that
    resolution.

The sibling shortInput() at :899-903 does the same to the argument at 160 chars.

So no phrasing can work. Every one of those twelve instructions routes the file through a
Bash or Read tool_result, and the cap fires on all of them identically. The Pilot was not
being obtuse; it was running a search whose space is empty.

Nothing in the loop prompt asks the engine to summarise —
grep -n "summar" workers/api/src/lib/coding-loop.ts returns nothing. The engine summarises
because a model asked "show me this file" answers conversationally; its text block is what
carries the answer, and that block is pushed untruncated (headless.ts:671).

How often it fires — population, not one run

Swept every stored terminal snapshot on this instance (20 sessions, 43 snapshots, 251 tool-result
lines), counting lines ending in the truncation ellipsis:

count
tool-result lines 251
cut at the 240 cap 150 (59.8%)
…of which ⚙ Bash 102
…of which ⚙ Read 43

Max observed result-line length 246 = " ↳✓ " (6) + 240. The cap is not a corner case; it is
the common case, and it fires on Read — the tool whose entire job is returning file contents.

The second half: the prompt names the wrong constraint

workers/api/src/lib/coding-loop.ts:542:

- You see only the LAST ~${PILOT_PANE_CHARS} characters of the terminal, never all of it. An instruction whose output is longer than that can never be verified by you: **ask for a bounded slice (a line range, a grep, head/tail)** rather than a full dump…

PILOT_PANE_CHARS = 6000 (lib/coding-repetition.ts:37). That line was added for #522 cause B,
whose recorded diagnosis is at coding-repetition.ts:17-21:

"The Pilot reads {@link PILOT_PANE_CHARS} characters of terminal while instructing a dump of
roughly twice that, so its own instruction is unverifiable by construction"

That attribution is wrong, and this run is the disproof. The dump is not in the pane in
truncated form — it was never in the pane at all. The binding limit is 240, not 6,000, and it does
not depend on how much was asked for. The prompt therefore hands the Pilot a remedy that cannot
work and a reason that is not the reason: sed -n '1,100p' → sed -n '1,50p' → head -60 →
sed -n '1,80p' is a model correctly following instructions that are false. #522's renderPaneForPilot
fix stands (stating a measurement beats leaving it implicit); the number it states is the wrong one
for this failure.

This is not #534. That is repo_read_file's 8 KB cloud-side cap on the repo connector path,
where an offset argument is the fix. This is the engine-drive path, where the Pilot has no
tool at all — it types a shell command and reads a pane. Different file, different remedy. #534
correctly predicted the Pilot would "page through a file by asking the engine to print it"; it
attributed the failure of that fallback to the 6,000-char pane, same as #522, and this is the
measurement that says otherwise.

What to do — cheapest first, and they are separable

1. Fix the prompt (one string, ships today). Replace the "ask for a bounded slice" advice at
coding-loop.ts:542. It must state the real bound and the one channel that has full fidelity:

A tool's OUTPUT reaches you truncated to ~240 characters on a single line, whatever command you
ran — cat, sed, head and grep all arrive the same size, so re-asking with a smaller range
will not help and a shell command can never show you a file. The engine's own REPLY text is not
truncated: to learn a file's contents, ask the engine to answer you with them ("quote lines
1-40 of X verbatim in your reply"), or ask it a question about the file and take its answer.

This alone converts twelve wasted rounds into one, and it is the fix that reaches every run
immediately — a runner change needs a CLI publish plus each machine upgrading.

2. Raise the result cap, deliberately — and this is a design decision, not a constant bump.
The pane is a fixed window: the API stores TERMINAL_SNAPSHOT_CHARS = 8000
(lib/terminal-snapshot.ts:57) and the Pilot reads 6,000 of it. At 240 chars a result, ~59% of
251 results still fit alongside the narrative. At 2,000, three large results fill the entire
stored snapshot and evict everything else — including the ⚙/↳ framing that
engine-tool-calls.ts parses into toolCalls, and the engine-error lines #580's
acceptance criterion exists to keep visible. So a flat raise buys file text by spending history.

The shape that does not: cap by tool. Read/Bash are the ones the Pilot needs verbatim;
Edit/Write/TodoWrite results are status strings that were never over 240 anyway (see the
sample above — "has been updated successfully", 176-214 chars). A per-tool cap raises exactly the
results that carry content. Owner's call whether that is worth the runner change at all once (1)
lands — my recommendation is to ship (1), measure whether runs still stall, and only then decide.

3. Stop collapsing whitespace regardless of the cap. \s+ → " " destroys line structure for
no size benefit worth naming, and it is the reason cat -n and sed are indistinguishable to the
Pilot. Cheap, independent of (2).

4. Correct the recorded cause. coding-repetition.ts:17-27 is the place a future reader learns
why this happens. Its "CAUSE B" paragraph should name the 240-char result cap as the binding limit
and keep PILOT_PANE_CHARS as the secondary one, or the next person re-derives the wrong fix.

Alternatives considered and rejected

  • Give the Pilot a read_file tool of its own. Rejected here as out of scope and as a
    collision: repo_read_file already exists on the connector path and is [bug] repo_read_file caps at 8KB of the 64KB the runner already read, with no offset to fetch the rest — the root cause under #522's repeated "read the full file" #534's subject. Two file
    readers with different caps is worse than one working prompt. Revisit only if (1) is measured
    insufficient.
  • Detect the pattern and stop the run. repetitionVerdict (coding-repetition.ts:116)
    deliberately does not catch this — the twelve instructions are paraphrases, not repeats, and its
    own docblock says no honest normalisation unifies them. Correct; a stall detector is not a
    substitute for removing the impossibility.
  • Raise PILOT_PANE_CHARS. Rejected: it is not the binding limit. It would raise cost per
    decision and change nothing about this failure.

Acceptance criteria

  1. A test replays the twelve instructions' shape — a Bash/Read result of 4,000 characters — and
    asserts what the Pilot's snapshot actually contains, so the 240-char loss is pinned rather than
    described.
  2. The prompt no longer advises a bounded slice as the remedy for unverifiable output, and states
    the ~240-character tool-result bound and the untruncated reply-text channel.
  3. coding-repetition.ts's CAUSE B names the binding limit correctly.
  4. If (2) is taken: Read/Bash results survive to a stated length; the guard states the length
    and shows the stored snapshot still carries its ⚙/↳ framing afterwards.
  5. Whitespace is no longer collapsed within a result.

Regression risk

(1) and (4) touch a prompt string — the risk is that a Pilot now asks the engine to quote long
files into its reply, which lands in the pane untruncated and can push the 8,000-char stored
snapshot's older content out. The #432/#466 snapshot-dedup and prune behaviour is where that
would show; lib/terminal-snapshot.ts's tests are the place to assert it.

(2) is a runner change: it ships only via a CLI version bump and per-machine upgrade
(platform/CLAUDE.md § Distribution), and old runners keep the 240 cap — so any cloud-side reader
must keep tolerating short results. engine-tool-calls.ts already models this (resultTruncated,
:116) and must keep doing so.

Verified vs inferred

  • Verified: all twelve instructions, read verbatim from GET …/timeline?full=1 on the named
    session; the 150/251 truncation sweep across 20 sessions; the 246-character maximum; the four
    cited headless.ts lines; that block.text is pushed untruncated at :671; that
    grep -n "summar" workers/api/src/lib/coding-loop.ts returns nothing; PILOT_PANE_CHARS = 6000;
    TERMINAL_SNAPSHOT_CHARS = 8000.
  • Inferred, and NOT reproduced end-to-end: that the corrected prompt would have got the file in
    one round. I did not drive a live run — the session belongs to the owner and was still live. The
    claim rests on block.text being untruncated, which is verified in code but not observed
    carrying a deliberately-quoted file.
  • Not measured: whether the twelve rounds' BYOK cost is recoverable from ai_usage. The run
    was cancelled (outcome seq 9303: "cancelled — Stopped by you."), so a cost figure would need
    the ledger, not the timeline.

Activity

  1. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Partially delivered — leaving this open. ff09ed3 is on main and the commit itself says "#700 stays open on them", which a re-verification against the code confirms.

    What landed (the cloud half — the Pilot is told the truth):

    • workers/api/src/lib/coding-loop.ts:553-554 — the prompt now states the ~240-character, single-line bound on a tool result and names the untruncated reply channel. The old "bounded slice" wording is gone, and there is a negative assertion on it.
    • workers/api/src/lib/coding-repetition.ts:20-44 — CAUSE B corrected to name the 240-char cap as the binding constraint, with PILOT_PANE_CHARS demoted to secondary.
    • Pinned through a real spawned engine on a 4,000-char / 92-line fixture: packages/browser-runner/src/coding/headless.test.ts:147 (tool_result line ≤246 chars, marker001 present, marker010/marker092 gone, no file line surviving as its own line), plus prompt arms at coding-loop.test.ts:103,116 and the banner arm at coding-repetition.test.ts:135.

    So the twelve-instruction search loop this ticket opens with should stop: the Pilot now knows the space it was searching is empty, and where to look instead.

    What did NOT land — the runner half, which is the actual truncation:

    • AC5 (unconditional) is undelivered. packages/browser-runner/src/coding/headless.ts:918 still does text.replace(/\s+/g, " ") and :919 still caps at 240. headless.ts is untouched by the commit. Every newline and indent is still collapsed before the pane exists, so sed -n '1,50p' and cat -n | head -60 still produce byte-identical shapes.
    • Item 2 / AC4 (raise the per-tool result cap) is deferred pending measurement. That one was conditional on item 2 being taken, so it is a legitimate defer; AC5 was not conditional.

    Both remaining items are packages/browser-runner changes, which reach users only through a packages/cli/package.json version bump and a CI publish — worth noting for whoever picks this up, since the fix is not live for anyone until that happens.

  2. added
    P1: blocks external usersMust be true before someone who is not the owner can run an agent (#68)
    bugSomething isn't working
    P0: wrong todayProduces a wrong result, loses work, or spends money in production right now
    runnerpackages/browser-runner + packages/cli — ships only via an npm publish
    coderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, Overseer
    and removed
    P1: blocks external usersMust be true before someone who is not the owner can run an agent (#68)
    on Aug 18, 2026
  3. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    The runner half is done and PUBLISHED. f4efe71b on main, CLI 0.4.55 on npm, CI + API deploy green.

    toolResult() is gone. The rendering moved to a new pure module, packages/browser-runner/src/coding/transcript-lines.ts, and the file-size ratchet is why it is a module rather than more of headless.ts (pinned at 959, sitting at 958).

    AC5 — whitespace is no longer collapsed

    Newlines survive. Every line of a result after the first carries │, which does two jobs: the pane reads as a transcript, and a result can no longer forge the framing — with newlines preserved, a file containing a line beginning ⚙ would otherwise parse as a tool call the engine never made. That risk did not exist while every result was one collapsed line, and it is the reason the continuation prefix is not optional.

    AC4 — the cap, raised per tool rather than raised

    1,500 characters over 60 lines for Read · Bash · BashOutput · Grep · Glob · NotebookRead. The historical 240 for everything else, and for an unknown tool (a pane that began mid-turn, a runner restarted between call and result). Edit/Write/TodoWrite results were 176-214 characters in the sample — widening them would spend pane and buy nothing.

    What it costs the pane, since that was the trade you asked to see stated. 1,500 is derived from the window, not picked: a quarter of PILOT_PANE_CHARS in content, and RESULT_RENDERED_MAX ≈ 1,840 once per-line framing and the disclosure are paid for — under a third of the 6,000 the Pilot reads, so three consecutive content results coexist with the ⚙/↳ framing and the narrative instead of replacing them. Both figures are pinned as arithmetic in transcript-lines.test.ts; raising the cap means arguing with them.

    A cut now also states its size — …[cut: 1,500 of 18,432 chars] — which is what turns "ask for a slice that fits" into a strategy rather than a guess.

    Verified end to end, then verified again against the published tarball

    headless.test.ts drives a real spawned stream-json engine returning the same ~4,000-character file two ways (the harness this issue's first round used to disprove the pane-window attribution). Measured off npm pack @proagentstore/cli@0.4.55, executing the shipped bundle:

    before after
    result line(s) 1 35
    characters delivered 246 1,703
    marker030 present no yes
    disclosure … …[cut: 1,500 of 4,047 chars]

    ⚙ Read and ↳✓ framing intact; marker092 still absent, correctly.

    Two consequences elsewhere, fixed here because leaving either made this a net loss

    • parseEngineToolCalls read the arrow line only. A 34-line result would have been reported as its first line, and outputCut (keyed on a trailing …) would have read a cut result as a complete one — the [bug] The Coder Lead can only ever see 3 of 6 agents — subordinate_status returns 60,239 chars into a 24,000 cap, so half the roster is silently dropped #503 shape. It now reads │ continuations, bounded by an inResult flag so a stranded tail whose arrow line fell outside the 8,000-char window is dropped rather than attributed to the wrong call.
    • FEED_TOOLCALL_BYTES 12,000 → 24,000. That knee was measured against 240-char results; at the new cap the same mean production row (12.05 calls) serialises to ~25,000 B, and a call dropped by that cap is gone for good — no cursor reaches it. 24,000 is one of the two numbers the original measurement showed behaviour-identical on the existing corpus.

    The prompt now states a MECHANISM, not a number

    ff09ed36 had it say "cut to ~240 characters on a single line" and "NO shell command can show you the contents of a file". True of every runner then published; false of an upgraded one — the same defect pointing the other way, and the cloud cannot tell which machine it is talking to. It now names the signal readable on either: a cut result says it was cut and by how much; ask for a slice that fits once; if the next result is cut at the same size and shape, the cap is on the result and re-asking is the same request. The untruncated reply channel is still named, because it works on every runner ever published.

    coding-repetition.ts's CAUSE B (AC3) is updated to say this half is now fixed at source, and that it is fixed only from RESULT_LINES_MIN_CLI = 0.4.55 on — an older machine still cuts at 240 on one line, so every cloud reader keeps tolerating that.

    Not verified

    I have not driven a live Pilot run to watch twelve rounds become one — that needs your machine and a session, and it is the one inch left. pags up on 0.4.55 is the prerequisite; a machine still on 0.4.54 behaves exactly as before. pnpm test:e2e was not run (no store/, console or coder/web code in the diff).

    Closing on the acceptance criteria: 1 ✅ (a real engine, not a described one) · 2 ✅ · 3 ✅ · 4 ✅ · 5 ✅.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0: wrong todayProduces a wrong result, loses work, or spends money in production right nowbugSomething isn't workingcoderThe Coder wedge agent (#68) — Engine, Pilot, Co-pilot, Loop, Overseerrunnerpackages/browser-runner + packages/cli — ships only via an npm publish

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions