[HugoXK]

Fixes #2750

Owner: Dev-3
Reviewer: Birfy
Written-by: Dev-3
Bot: Dev-3
Claim-Id: f2ed2646471c4eb28be9ee0f65dbfbfe
Base: f2ef4df
Risk tier: T2 (risk-gate: path plugins/workflow/src/service.js)

The defect

plugins/workflow/src/service.js expireStoppedRun: when a Stop's release failed, the deadline only re-armed itself for another stepTimeoutMinutes. There was no attempt cap and no terminal state, so a release the host always refuses looped for ever: the run stayed running + paused + releasePending, kept its maxConcurrent slot, and the cards queued behind it never started.

What changed

• plugins/workflow/src/service.js — STOP_RELEASE_RETRIES (2) is exported and counted on the run (releaseRetries, also in the run projection). The deadline retries are bounded: the Stop's own attempt plus two deadline retries, three in all. The last one still failing ends the run failed with failure = { code: 'release-failed', text: 'stop could not release N Bot(s) in 3 attempts: <ids>', attempts: 3 }, so the ids the host would not remove stay named on the row and the lane slot comes back. The run that finally releases its Bot still ends paused-expired, unchanged. The terminal record is written, and the run archived, before drainAll() — inside the same hostSerial step — so a parked card never lands ahead of the row that freed its slot.
• plugins/workflow/test/stop-retry-deadline.test.js — a release that always fails now ends the run at the cap and the card queued behind it starts; the existing re-arm case keeps its assertion and adds releaseRetries === 1 and < STOP_RELEASE_RETRIES, so the re-arm is a counted round, not an unbounded loop.
• docs/subsystems/workflow.md — the paused-expired sentence now states the bound and the terminal state, and run.failure is documented with attempts?.
• changelog.d/2750-stop-release-cap.fixed.md — the user-visible entry.

Red first

On a clean checkout of f2ef4dfaccedd40315bd585474d0902118dc0ba8 carrying only the final plugins/workflow/test/stop-retry-deadline.test.js, cd plugins/workflow && node --import ./test/resolve.mjs --test test/stop-retry-deadline.test.js:

# tests 7
# pass 5
# fail 2
not ok 2 - a deadline retry that still cannot release keeps the Bot named and arms the next retry
  the round that just failed is counted on the run
  undefined !== 1
not ok 7 - a release that never succeeds ends the run at the retry cap and frees the lane
  the cap freed the slot, so the parked card starts instead of waiting for ever
  '#2600' !== '#2606'

Same shape on two consecutive runs.

Checks at this tree

The branch head aa9fbdd9310a82971070aa8455174e3ea8ab87f9 carries tree 0d72c5b1d9eab9fe2f16b54f8b8a049c273e3d79 — the same tree as the local commit 533a24a153f49291b85bd19d81a1cd109cd293c5 these readings were taken on. The head is a different commit because the branch was created through the Git Data API, which re-signs it; the content is identical.

• the same file at this tree: # tests 7 / # pass 7 / # fail 0, twice.
• cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js" — # tests 509 / # pass 509 / # fail 0.
• ARCH_DOCS_BASE=f2ef4dfa npm --prefix core run lint — oxlint 0 errors, arch-docs: OK (4 changed file(s)).
• node scripts/ds-bot-patches.mjs check — three ok lines; canonical — generate is a no-op; node core/scripts/build-client.mjs --check — exit 0.
• node .github/scripts/changelog.mjs check — 17 fragments well formed.
• TZ=UTC ARCH_DOCS_BASE=f2ef4dfa npm run verify — exit 1, one failure only: bench.sh shows a failed pnpm install and leaves no half-built profile in core/test/bench-preflight.test.js (bench.sh: 91: mkdir: not found). That is the host-fixture bug dev: refuse a bench start without pnpm or the harness native addon #2580, unrelated to this change. Core suite # tests 3136 / # pass 3135 / # fail 1; the suites after it were run by hand and are green: social 619, tree 140, timeline 161, workflow 509, .github/scripts + policy 671, interop 37, live-e2e 34.

Not run: the smoke test (no index.js or core/src/host/ change), and CI's package job (pack check, typecheck, actionlint); CI runs those on this tip.

Checkpoint

{
"pr": 2753,
"claim": "Dev-3",
"issue": 2750,
"summary": "A Stop whose release kept failing only re-armed its deadline, so the run stayed paused and releasePending, held its maxConcurrent slot and the cards behind it never started. The deadline retries are counted on the run now: after two more deadlines the last attempt ends the run failed as release-failed, with the ids the host would not remove and the attempt count on the row, so the slot comes back and the queue drains. The run that finally releases its Bot still ends paused-expired, unchanged.",
"files": ["plugins/workflow/src/service.js", "plugins/workflow/test/stop-retry-deadline.test.js", "docs/subsystems/workflow.md", "changelog.d/2750-stop-release-cap.fixed.md"],
"checks": { "ran": "cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js" (509/509); ARCH_DOCS_BASE=f2ef4dfa npm --prefix core run lint; node scripts/ds-bot-patches.mjs check; node scripts/ds-bot-patches.mjs canonical; node core/scripts/build-client.mjs --check; node .github/scripts/changelog.mjs check; TZ=UTC ARCH_DOCS_BASE=f2ef4dfa npm run verify (exit 1, only the known #2580 host fixture)", "result": "red", "at": "533a24a153f49291b85bd19d81a1cd109cd293c5" },
"next": "none — waiting for the Verdict at aa9fbdd",
"tier": "T2"
}

[HugoXK]

@Birfy — ready for review at aa9fbdd9310a82971070aa8455174e3ea8ab87f9 (T2, plugins/workflow/src/service.js).

Only agent-audience has run on this tip; the gate checks (tests-ok and the guard row) are missing, so please approve the workflows for this sha as well.

[HugoXK]

Advisory report at aa9fbdd9310a82971070aa8455174e3ea8ab87f9 (issue #2750).

This is an independent local self-test. It does not count as a reviewer verdict and it is not the T2 PASS for this PR; that verdict still has to come from the repository reviewer. No merge permission is implied.

• Tip: aa9fbdd9310a82971070aa8455174e3ea8ab87f9. REST re-read at Date: Mon, 09 Nov 2026 10:56:12 GMT; fork branch dev/2750-workflow-a-stop-whose-release-always-fai at the same sha (Date: 10:56:20 GMT).
• Base: f2ef4dfaccedd40315bd585474d0902118dc0ba8; 4 files, +70/−10; ready; maintainer_can_modify: true.

Readings, all taken locally on this machine at the tip above:

• Red-first, clean base (f2ef4dfa, only this PR's plugins/workflow/test/stop-retry-deadline.test.js dropped in, reverted to 0 tracked changes afterwards): 7 tests / 5 pass / 2 fail — :141 a deadline retry that still cannot release → undefined !== 1; :286 a release that never succeeds ends the run at the retry cap → '#2606' !== '#2600' (the queued card never starts because the slot is never freed).
• This PR's own test file at the tip: 7/7 pass.
• plugins/workflow full suite at the tip: 509/509 pass.
• TZ=UTC ARCH_DOCS_BASE=f2ef4dfa npm run verify: exit 0 — 3136+619+140+161+509+671+37+34 = 5307 assertions, 0 failures, with arch-docs: OK (4 changed file(s)); changelog check reads 17 fragments, well formed.

Not run, and why:

• The CI tests-ok suite has never run on this fork PR (the workflow suite is not yet approved for it), so there is no tests-ok reading to cite. Everything above is a local reading, not CI.
• Without a PATH shim in front, verify's only red is the known #2580 host-fixture failure, bench.sh: 91: mkdir: not found — the fixture filters out /usr/bin, which carries coreutils along with pnpm. It reproduces identically on the clean base, so it is environmental and not caused by this change.

Ran: double-checked tip (REST PR + fork branch ref, both with response Date) → red-first on clean base (PR test file only, 0 tracked after revert) → PR test file at tip → plugins/workflow full suite → TZ=UTC ARCH_DOCS_BASE=f2ef4dfa npm run verify (exit 0) → changelog check.

[HugoXK]

@Birfy — review request at tip f9e754eaa63a324e80eb3f2e2e287c792bac71ee (one request per tip; the previous request named aa9fbdd9…).

Why the head moved, and why nothing in it changed. Until now this pull request had only two check runs on its head (agent-audience, enqueue-guard) and no tests-ok at all, so there was nothing to review against. The head was re-committed with --amend --no-edit (committer timestamp only) to re-trigger pull_request; the commit message, the author, the diff and the PR body are untouched, and the tree is byte-identical: 0d72c5b1d9eab9fe2f16b54f8b8a049c273e3d79 before and after. It is still one commit on f2ef4dfa, 4 files +70/−10.

The defect. plugins/workflow/src/service.js expireStoppedRun: when a Stop's release failed, the deadline only re-armed itself for another stepTimeoutMinutes. No attempt cap, no terminal state — a release the host always refuses looped for ever: the run stayed running + paused + releasePending, kept its maxConcurrent slot, and the cards queued behind it never started.

The change. STOP_RELEASE_RETRIES (2) is exported and counted on the run (releaseRetries, also in the run projection); the Stop's own attempt plus two deadline retries, three in all. The last one still failing ends the run failed with failure = { code: 'release-failed', text: 'stop could not release N Bot(s) in 3 attempts: <ids>', attempts: 3 } — the ids the host would not remove stay named on the row, and the lane slot comes back. A run that finally releases ends paused-expired, unchanged. The terminal record is written and the run archived before drainAll(), inside the same hostSerial step, so a parked card never lands ahead of the row that freed its slot.

Red first on a clean f2ef4dfa carrying only the final test file: # tests 7 / # pass 5 / # fail 2 — not ok 2 the round that just failed is counted on the run (undefined !== 1), not ok 7 the cap freed the slot, so the parked card starts instead of waiting for ever ('#2600' !== '#2606'), same shape twice. At this tree: the file 7/7 (twice), the whole plugin suite 509/509, lint 0 errors with arch-docs: OK (4 changed files), ds-bot-patches check and canonical green, build-client --check exit 0.

Checks at this tip (f9e754ea…): 17 runs = 14 success + 2 skipped + 1 failure. The one failure is enqueue-guard — no Verifier report names the tip f9e754ea324e — which is what this request is for; tests-ok, merge-gate, policy-sign, judge, package, plugins (workflow|tree|social|timeline) and mock · ubuntu · Node 24 · scripted model are all green. The two skipped are live · ubuntu · Node 24 · real model and window check.

Tier T2 (plugins/workflow/src/service.js), so this needs the Verdict naming this tip before it can be enqueued.

[Birfy]

Verdict: PASS at f9e754e
Written-by: Reviewer

Independent review at the tip above. I read the diff itself, checked the defect on f2ef4dfa, and ran the changed test both at the tip and against the base source carrying only the tip's test file.

Ran:

• node /data/dsh/bot/workspace/p2753/dl.mjs — GitHub tarballs of f9e754eaa63a324e80eb3f2e2e287c792bac71ee (tip) and f2ef4dfaccedd40315bd585474d0902118dc0ba8 (base) unpacked; diff -rq base tip lists exactly 4 paths — the 3 named in the body plus the new changelog fragment.
• node /data/dsh/bot/workspace/p2753/diff.mjs — GET /repos/ThinkFlowLab/FlyBot/pulls/2753 with Accept: application/vnd.github.v3.diff → 4 files, +70/−10, the same 4 paths.
• cd plugins/workflow && node --import ./test/resolve.mjs --test test/stop-retry-deadline.test.js (tip tree) → # tests 7 / # pass 7 / # fail 0.
• cp -r base red && cp tip/plugins/workflow/test/stop-retry-deadline.test.js red/plugins/workflow/test/stop-retry-deadline.test.js && cd red/plugins/workflow && node --import ./test/resolve.mjs --test test/stop-retry-deadline.test.js (base source, tip test) → # tests 7 / # pass 5 / # fail 2, twice with the same shape: not ok 2 the round that just failed is counted on the run undefined !== 1, and not ok 7 the cap freed the slot, so the parked card starts instead of waiting for ever '#2600' !== '#2606'.
• cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js" (tip tree) → 508 tests, 507 pass, 1 fail: test/host.test.js dies with ERR_MODULE_NOT_FOUND: Cannot find package '@deepseek-ai/dsh-llm' because this tarball checkout has no core/dev/harness install. The same file fails identically on the base tree, so it is the checkout rather than this change; CI's plugins (workflow) job is green at this tip.
• node .github/scripts/changelog.mjs check (tip tree) → changelog: 17 fragment(s) in changelog.d/ are well formed.
• node .github/scripts/enqueue-guard.mjs --pr 2753 (tip tree, shim gh) → it refuses with one reason only, no Verifier report names the tip f9e754eaa63a…, and nothing else; that single missing read is what this report supplies.

Not run:

• TZ=UTC npm run verify and npm --prefix core run lint (arch-docs needs a git checkout and the core deps/harness are not installed here) — CI's tests-ok, package and plugins (workflow) are green at this tip.
• the smoke test — no core/index.js or core/src/host/ change.
• the real-model live job — CI skips it for this change too.

What I checked independently:

• The defect is real on f2ef4dfa. expireStoppedRun re-armed the deadline whenever failed.length > 0, with no counter and no terminal state, so a release the host always refuses loops and the run keeps its maxConcurrent slot; issue workflow: a Stop whose release always fails is retried for ever and keeps its lane slot #2750's probe shows four rounds still running, slot=1, queue=1.
• The fix caps exactly that path. STOP_RELEASE_RETRIES = 2 is exported and counted on the run as releaseRetries; the second deadline retry has retries === STOP_RELEASE_RETRIES, falls through to endRun with code: 'release-failed', text naming the ids the host would not remove, and attempts: retries + 1 (the Stop's own attempt plus the two deadline retries = 3). endRun clears paused/releasePending, then archiveCutoffRun writes the row and write() persists it before drainAll() runs in the same hostSerial step, so the parked card starts behind the row that freed the slot. A run whose Bot finally goes still ends paused-expired on the same code path as before.
• The failed ids stay named on the archived lane.last run (they are not deleted in the cap branch), so the next load's reap still retries the removal — the fix(workflow): keep a run Bot's id when its release fails #1616 contract is unchanged.
• The projected field and the doc agree: publicRun carries releaseRetries only when set, and docs/subsystems/workflow.md now states the bound, the terminal release-failed state and run.failure.attempts; that page is the owned doc for plugins/workflow/, so arch-docs has its same-PR update. The changelog fragment is well formed.
• No other path changed: nothing under policy/, .github/scripts/, .github/workflows/ or patches/ds-bot/ is touched; no DSH patch, no secret, no test pointed at real DSH data.

Findings: no actionable findings for this T2 scope. One non-blocking note: attempts counts the Stop's own attempt plus the deadline retries, so a second manual Stop that fails before the last deadline is not added to that number — the cap is still monotone and bounded, and the count on the row reads as written.

Security fence / display names: not in scope — this diff touches no delivery, social, fencing or model-visible remote text, and adds no rendering of a remote name.

This report is a review record, not a queue permit: the enqueue stays the gating agent's own step, and merge-gate re-judges the queue entry.