Repository navigation
[bug] A pinned-to-an-offline-machine agent is told to run pags up --force — describeFacts drops the two fields the pinned-machine diagnosis needs, so the branch is dead on every surface but one #461
Description
The one diagnosis that names a pin is unreachable from the surface that needs it — so a pinned agent is told to run pags up --force
diagnoseAttachment has a pinned-machine-offline branch that says the right thing. describeFacts,
which is how three call sites reach it, does not pass the two fields that branch depends on. So every
one of those call sites falls through to machine-online-agent-detached and prescribes
pags up --force — the remedy the pinned case's own code comment says must not be offered.
Measured, production, 2026-08-08 13:2x UTC
One instance, 12ebf1f0-… ("Coder Home"), pinned to a machine that is switched off:
GET /v1/instances/12ebf1f0-…/runner-node
runnerNode: "Sergeys-Mac-mini.local"
nodesDetail: Mac connected=true nodeOnline=true
Sergeys-Mac-mini.local connected=false nodeOnline=false
resolvedNode: null
Two surfaces, same instance, same second:
GET /v1/instances/12ebf1f0-…/runtime/status ← pin-aware
runtime.status "online" runnerNode "Mac" runnerVersion 0.4.44 health.ok true
GET /v1/instances/12ebf1f0-…/coding/repos ← pin-blind
recheck.reason "The machine is online but this agent isn't attached — another runner on it may
already hold this agent. Try: `pags up --force`"
Both sentences are produced by the same function, from the same facts, about the same instance. The
second is wrong in both halves: nothing else holds this agent, and --force cannot fix it. pags up
is already running on Mac with a live socket for this very instance. What the user needs to do is
change one dropdown.
Where it is
lib/runtime-attachment.ts:38-52 — the diagnosis takes the pin:
pinnedNode?: string | null;
liveNodeExcludedByPin?: string | null;and :63-73 uses it, before the heartbeat check, with the reasoning already written down:
// BEFORE the heartbeat check, deliberately (#380). … a pin onto a dead machine reads as "fresh"
// and diagnoses `machine-online-agent-detached`, prescribing `pags up --force` for a machine that
// is switched off.That comment describes, exactly, the output measured above.
lib/runner-availability.ts:110-116 — the adapter that drops them:
export function describeFacts(facts: RuntimeFacts): AttachmentDiagnosis {
return diagnoseAttachment({
hasRuntimeRow: facts.hasRuntimeRow,
relayConnected: facts.relayConnected,
lastSeenAt: facts.lastSeenAt,
});
}RuntimeFacts (lib/instance-connectivity.ts:12-20) carries no pin at all, so the adapter has nothing
to forward even if it wanted to. Grepping the whole tree, pinnedNode is supplied by exactly one
caller — routes/instances.ts:793-806, the /runtime/status route — which is why that surface gets it
right and no other does.
The three call sites that inherit the wrong answer
routes/coding-repos.ts:176(noRunnerReason) → therecheck.reasonabove. This is the one that
matters most, because it is the sentence [bug] A repo has been marked broken for five days by a dropped WebSocket — a transport failure is stored as the repo's state, and no verdict is ever re-taken #440 added specifically so a stuck repo would explain
itself, on the very instance [bug] A repo has been marked broken for five days by a dropped WebSocket — a transport failure is stored as the repo's state, and no verdict is ever re-taken #440 was filed about.lib/runner-availability.ts:161(pauseMessage) → what a Workflow announces into the owner's
thread when it pauses for an unreachable runner. A pinned instance is told to--forcea machine
that is off, for up to 30 minutes of waiting.lib/runner-availability.ts:175→ the same, on the terminal path.
Two decisions composing into the bug
Neither half is wrong on its own, which is why it shipped. diagnoseAttachment was given the pin
because #380 proved it was needed. describeFacts was written as a thin adapter over RuntimeFacts,
which is a connectivity record and has no business knowing about instance config. The defect is that
the second is the only path the first is reached by, outside one route — so the branch exists, is
correct, is tested, and is dead everywhere it is needed.
What to do — cheapest first
- Carry the pin on the facts. Add
pinnedNodeandliveNodeExcludedByPintoRuntimeFactsand
populate them inruntimeConnectivity(which already reads the instance row to resolve routing, so
the pin is in hand), then forward both indescribeFacts. One field each, and all three call sites
are fixed at once because they all go through it. - Make the reason assertable.
routes/coding-repos.test.ts:377currently asserts only
expect(String(body.recheck.reason)).toMatch(/\S/)— that the reason is non-empty. That is why a
wrong reason shipped green. Assert the pinned case names the pinned machine and the live one, and
does not contain--force. - Pin the invariant where it can be stated once: no diagnosis whose state is
pinned-machine-offlinemay carry a remedy, and no diagnosis produced for an instance with a live
socket on a pin-excluded node may bemachine-online-agent-detached.
Alternatives considered and rejected
- Have
coding-repos.tscalldiagnoseAttachmentdirectly with the pin. Rejected: it fixes one of
three call sites and leaves the workflow pause messages wrong, which is the harder failure to notice
because it appears mid-run in a chat thread rather than in an API response. - Drop the
pinned-machine-offlinebranch and let the pin surface only in/runtime/status.
Rejected — the branch is correct and [bug] /runtime/status answers "attached · Connected." for an instance whose pin points at a dead machine — the pin-blind fallback overrides the pin-aware resolution (#238) #380 built it for a reported symptom. The problem is reach, not
design. - Make
getBoundRunnerConnfall back to a live node when the pin is offline. Firmly rejected, and
out of scope: the pin is authoritative by design ([bug] A machine's identity is os.hostname(), so one laptop is three machines — and a pin to a name it no longer uses answers every call with "run pags up" while pags up is running #379/[bug] /runtime/status answers "attached · Connected." for an instance whose pin points at a dead machine — the pin-blind fallback overrides the pin-aware resolution (#238) #380), and a silent fallback would run an
agent's coding session on a machine the owner did not choose. The fix is to explain the pin, not to
ignore it. - Widen
RuntimeFactsinto a general "everything about this instance" record. Rejected as scope
creep; two named fields with a comment saying why connectivity has to know about the pin.
Acceptance criteria
- With an instance pinned to an offline machine while another machine holds a live socket for it,
GET /coding/reposreturns arecheck.reasonthat names both machines and tells the user to
change "Runs on" — and does not containpags up --force. - The same holds for a Workflow pause announcement on that instance.
- A test asserts the content of the pinned-case reason, not merely that it is non-empty.
- The unpinned
machine-online-agent-detachedcase still sayspags up --force, unchanged.
Regression risk
describeFactsfeeds user-facing copy on three surfaces; changing what it returns changes three
messages. The existingruntime-attachment.test.tscovers the branch logic and should catch a
mis-ordering, but the adapter is what is untested — add a case there specifically.- Populating the pin in
runtimeConnectivityadds an instance-config read to a function used in a
batched path (MAX_RELAY_PROBES,subordinate_status). Resolve it from the row already being read
rather than adding a query per instance.
Files: workers/api/src/lib/runner-availability.ts:110-116,161,175,
workers/api/src/lib/instance-connectivity.ts:12-20, workers/api/src/lib/runtime-attachment.ts:38-73,
workers/api/src/routes/coding-repos.ts:176-195, workers/api/src/routes/instances.ts:793-806,
workers/api/src/routes/coding-repos.test.ts:377.
Found while auditing #440, whose fourth item this defeats. Related: #379, #380, #237.
Activity
fix(runner): carry the "Runs on" pin on RuntimeFacts, so a pinned age…
serge-ivo commentedon Aug 8, 2026
Shipped in
07ec7af, deployed — item 1, which fixes all three call sites at onceRuntimeFactsnow carriespinnedNodeandliveNodeExcludedByPin,runtimeConnectivitypopulatesthem, and
describeFactsforwards both. Because all three call sites reachdiagnoseAttachmentthrough that adapter,
routes/coding-repos.ts'srecheck.reasonand both Workflow pause messagesare fixed by the same two lines.
The batched-path regression risk was real, and is handled
The issue warned that populating the pin adds an instance-config read to a function used under
MAX_RELAY_PROBES/subordinate_status.runtimeConnectivityManyreads every instance'sconfig.runnerNodein one batched statement, and callsliveNodeIgnoringPinonly where apin exists AND nothing resolved. The unprobed tail gets its pin but never an invented live node, so
it degrades to its previous answer rather than to half a diagnosis.
The assertion has teeth now
coding-repos.test.ts:377assertedtoMatch(/\S/)— that the reason was non-empty. That is why awrong reason shipped green. It now asserts the pinned case names both machines and does not
contain
--force, and the unpinnedmachine-online-agent-detachedcase still sayspags up --forceunchanged. Sensitivity was verified the honest way: reverting the two forwarded lines in
describeFactsmakes the newcoding-repos.test.tscase and both newrunner-availability.test.tscases fail; restoring them makes the suite green again.
The two invariants from item 3 are recorded on the
pinned-machine-offlinestate.What I could NOT verify live, and why — stated plainly
The live reproduction needs one machine connected while the instance is pinned to a different,
offline one. Right now that state does not exist on the reported instance:
With nothing connected,
no-runneris the correct diagnosis and the correct remedy — so thisoutput neither confirms nor refutes the fix. Reproducing #461 requires starting
pags uponMacwhile the pin stays on the Mac mini, and starting a runner on the owner's machine is not mine to do.
So: the pinned branch is proven by test, not in production. If you bring a runner up on
Mac, thissame
GET …/coding/reposshould name both machines and point at the "Runs on" dropdown instead ofoffering
--force. That is the one thing left to eyeball.Left as a follow-up, not done
routes/instances.ts:793-806still assemblespinnedNode/liveNodeExcludedByPinby hand and couldnow go through
describeFacts. Untouched here — a concurrent lane owned that file, and it isduplication rather than a defect (that route was always correct). Worth a ticket; not filing one
unprompted.
Full bar green (416 files / 6944 tests at the time of the lane's run, 7052 in the integrated tree);
pnpm test:e2eskipped deliberately — the change is three API-worker source files and touches nostore/, console, oragents/coder/web. CI and both deploy workflows green.fix(supervision): the second adapter forwards the pin too, so a pinne…
fix(agents): the chat prompt states the attachment diagnosis, so a ru…
fix(agents): a runner-too-old refusal names the machine, its version,…
fix(coding): engine sign-in on a runner without tmux is a clear 409, …