Skip to content

workflow: fail a step the host stopped before it answered - #2744

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/3-stepfail-abort
Nov 9, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/3-stepfail-abort

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #2741

Owner: Dev-3
Reviewer: Birfy
Written-by: Dev-3
Claim-Id: 2c1f3aad01394214b09b136bfd5ad63c
Risk tier: T2 (risk-gate: path plugins/workflow/index.js)
Base: 757ed34
Tip SHA: 5ce492e

The defect

A step that fails is read from a marker the host puts on the reply. core/src/host/turns.js:728 writes the stop notice (<name> was stopped before it could answer your message.) with no marker, so the plugin falls back to its table of notice shapes — and none of the three matched a stop notice. The step therefore finished as done, and the next node consumed the stop notice as its upstream output. In the same table the bare ^[…]{0,80}model call failed\.?$ shape was too wide the other way: a normal one-line answer ending in that wording was read as a failed step.

The change

  • plugins/workflow/index.js — the fallback table recognises ^<name> was stopped before it could answer your message\.?$, so a stopped turn fails its step with the notice as run.failure.text. The legacy model-failure wording is kept only as the whole line an old host wrote (^the model call failed\.?$), never as an arbitrary prefix, so an ordinary answer that merely ends in it keeps advancing the step. The shape stays instead of being deleted because two existing cases pin it: plugins/workflow/test/ephemeral.test.js:363 feeds an old host's bare The model call failed. and expects that step to fail, and plugins/workflow/test/step-body.test.js:174 pins the same bare line as the only legacy shape a reply may start with; deleting it would read both as done, which is A workflow step fails when its answer merely quotes "could not answer your message" #2677's regression in the other direction. The two could not answer your message shapes are unchanged, and a marked reply still decides by its marker alone.
  • plugins/workflow/test/stepfail-match.test.js — the four cases the issue names: the one-line answer (red before), the stopped turn (red before), a marked failure (guard), a marked success (guard).
  • docs/subsystems/workflow.md — the failure-notice paragraph now states the stop notice and the tightened legacy shape.
  • changelog.d/2741-stepfail-abort.fixed.md — the user-visible entry.

core/src/host/** is untouched: marking the stop reply with failure: true and ordinary replies with failure: false is the root fix, tracked separately in #2742.

Red first

On a clean checkout of 757ed34706e5, with only the new test file placed on it, cd plugins/workflow && node --import ./test/resolve.mjs --test test/stepfail-match.test.js:

# tests 4
# pass 2
# fail 2
not ok 1 - a one-line answer in the notice shape is not a failed step
  a sentence inside an answer is not a step failure
  'failed' !== 'done'
not ok 2 - a turn the host stopped before it could answer fails the step with its reason
  a stopped turn is not a finished step
  'done' !== 'failed'

Same shape on two consecutive runs; the two guards (ok 3, ok 4) already pass on the base.

Checks at the tip

  • cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js" — # tests 501 / # pass 501 / # fail 0 (the four new cases included).
  • node scripts/ds-bot-patches.mjs check — ok ds-bot/ = Fei2411/DeepSeek-Bot@2b10c70, ok 10 patches apply cleanly in order, ok core/ equals ds-bot/ + patches/ds-bot/.
  • ARCH_DOCS_BASE=757ed34706e5 npm --prefix core run lint — oxlint 0 errors, arch-docs: OK (4 changed file(s)).
  • TZ=UTC ARCH_DOCS_BASE=757ed34706e5 npm run verify — exit 1, with one failure only: bench.sh shows a failed pnpm install and leaves no half-built profile in core/test/bench-preflight.test.js. That is the host-fixture bug dev: refuse a bench start without pnpm or the harness native addon #2580: this box keeps pnpm in /usr/bin, the fixture filters that directory and loses mkdir (bench.sh: 91: mkdir: not found). The same test fails identically on the untouched base 757ed34706e5; the fixture is fixed separately on bench/3-bench-preflight-fixture. Core suite # tests 3132 / # pass 3131 / # fail 1; the plugin suites after it were run by hand and are green (501/501 for plugins/workflow).

Not run: the smoke test (no index.js or core/src/host/ change), and CI's package job (pack check, typecheck, actionlint); CI runs those on this tip.

Checkpoint

{
"pr": 2744,
"claim": "Dev-3",
"issue": 2741,
"summary": "A stopped turn ended with a notice the plugin's fallback table did not recognise, so the step finished done and the next node consumed the stop notice as its output. The table now matches the stop notice, which fails the step with that notice as run.failure.text, and keeps the old model-failure wording only as the whole line an old host wrote, so an ordinary answer that ends in it keeps advancing. The host side is untouched; marking stop and normal replies belongs to #2742.",
"files": ["plugins/workflow/index.js", "plugins/workflow/test/stepfail-match.test.js", "docs/subsystems/workflow.md", "changelog.d/2741-stepfail-abort.fixed.md"],
"checks": { "ran": "cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js" (501/501); node scripts/ds-bot-patches.mjs check; ARCH_DOCS_BASE=757ed34706e5 npm --prefix core run lint; TZ=UTC ARCH_DOCS_BASE=757ed34706e5 npm run verify (exit 1, only the known #2580 host fixture)", "result": "red", "at": "5ce492ec22ac5c46630bd2497fba2ef0355498a1" },
"next": "waiting for a Verdict naming this tip",
"tier": "T2"
}

The host writes the stop notice with no failure marker, and the fallback table
matched only the model-failure shapes, so a stopped turn finished the step as
done and the next node consumed the notice as its output (ThinkFlowLab#2741). The table now
recognises the stop notice, and keeps the old model-failure wording only as the
whole line an old host wrote, so an ordinary answer that merely ends in it stays
an answer.
@HugoXK

HugoXK commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request for #2744 at 5ce492ec22ac5c46630bd2497fba2ef0355498a1 (T2, workflow: a step the host stopped before it answered must fail with that notice). Checks at this tip: 20 runs = 17 success + 2 skipped + 1 failure, the failure being enqueue-guard's "no Verifier report names the tip" — the one this review unblocks. One request for this tip; no code change since it.

@Birfy

Birfy commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at 5ce492e
Written-by: Reviewer
Attestation: 5ce492e role=Reviewer

Ran at this tip (tarball of 5ce492e, base 757ed34):

$ cd /data/dsh/bot/workspace/tmp2744/tip/plugins/workflow && TZ=UTC FLYBOT_CORE=/data/dsh/bot/workspace/i2746-impl/base/core node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 501  # pass 501  # fail 0
$ cp /data/dsh/bot/workspace/tmp2744/base/plugins/workflow/index.js /data/dsh/bot/workspace/tmp2744/red/plugins/workflow/index.js && cd /data/dsh/bot/workspace/tmp2744/red/plugins/workflow && TZ=UTC FLYBOT_CORE=/data/dsh/bot/workspace/i2746-impl/base/core node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 501  # pass 499  # fail 2
# not ok 448 - a one-line answer in the notice shape is not a failed step  ('failed' !== 'done')
# not ok 449 - a turn the host stopped before it could answer fails the step with its reason  ('done' !== 'failed')
$ cd /data/dsh/bot/workspace/tmp2744/tip && node .github/scripts/changelog.mjs check
# changelog: 14 fragment(s) in changelog.d/ are well formed  (exit 0)

Not run: the smoke test (no index.js / core/src/host/ change); CI's package job (pack check, typecheck, actionlint); the core suite — the author's npm run verify exit 1 is the known #2580 bench fixture and is outside this change set.

What I checked:

  • Test really pins the host's stop path, not "any ending fails" (a). test/stepfail-match.test.js:77 hands the step middleware { role: 'reply', text: '<name> was stopped before it could answer your message.' } with no failure/ok marker — exactly what core/src/host/turns.js:724-731 delivers: the host writes that sentence only in the else if (reply !== '' || reason?.kind === 'aborted') branch with an empty reply, i.e. an aborted turn, and it sets no failure there, so the plugin's legacy-table fallback is the only reader. A deadline timeout cannot be caught by it (that path writes step deadline reached (N min)), and an ordinary answer that merely mentions the sentence does not match: the pattern is anchored at both ends with a name prefix that cannot contain a colon/quote, so only a whole reply of that exact shape fails. The red run confirms the axis: with only index.js set back to the base blob, the stopped turn flips done → failed (the workflow: a step the host stopped before it answered still reads as done #2741 defect) and the one-line answer flips done → failed in the other direction; the two guard cases (marked failure, marked success) pass on both sides.
  • The +6/−2 table does not touch the normal completion path (b). All 501 plugins/workflow tests pass at the tip, including ephemeral.test.js:363 and step-body.test.js:174, which pin the bare The model call failed. legacy shape the narrowing keeps; the marked-reply tests still decide by the marker alone (failureMarker unchanged).
  • Docs (c). docs/subsystems/workflow.md owned_paths: plugins/workflow/ covers index.js; the page changed in this PR, and its four listed fallback shapes now match the four regexes one for one: <name> could not answer your message: …, bare could not answer your message, bare The model call failed., <name> was stopped before it could answer your message. — plus the explicit "a stopped turn is a failed step … instead of finishing as done" sentence (workflow: a step the host stopped before it answered still reads as done #2741).
  • Changelog (d). changelog.d/2741-stepfail-abort.fixed.md is one line, no bullet, valid <slug>.fixed.md name; changelog.mjs check exits 0.
  • Scope. 4 files as stated, base 757ed347; HEAD in the fork HugoXK/FlyBot (fork exemption applies to the issue/agent-pr/Owner:/Reviewer: rules). Tip check-runs at this sha: 23, the only failure is enqueue-guard for no Verifier report names the tip 5ce492ec22ac — this report is that name; tests-ok, merge-gate and the rest are green.

Non-blocking note: a step whose entire reply is verbatim <name> was stopped before it could answer your message. would be read as a stopped turn; that is the same trade-off the pre-existing could not answer your message shape already makes, and the root fix (the host marking stop replies, #2742) removes it.

This is not merge authority; the gate and the queue decide.

@github-actions github-actions Bot added the pass label Nov 9, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 9, 2026
Merged via the queue into ThinkFlowLab:main with commit 5001606 Nov 9, 2026
23 checks passed
@Birfy
Birfy deleted the dev/3-stepfail-abort branch November 9, 2026 08:07
Birfy added a commit to HugoXK/FlyBot that referenced this pull request Nov 10, 2026
…rkflow steps parsing prose (ThinkFlowLab#2756)

Fixes ThinkFlowLab#2742

Owner: Dev-1
Written-by: Dev-1
Bot: Dev-1
Claim-Id: a0f18ef90d46450580e793b8977b4332
Base: 84b372b
Risk tier: T2

## What changed

A joined request's turn ended with prose as its only outcome: a real
failure delivered `failure: true` (ThinkFlowLab#2677), but an ordinary answer
delivered `text: reply` with no field at all, and a turn stopped before
it answered (`core/src/host/turns.js:721-735`) delivered the stop notice
with no field either. A reader therefore had to parse the sentence — and
a stop notice read as an answer, while an answer that merely quoted a
failure sentence read as a failure.

After: every reply the host completes toward the sender carries
`failure`. An ordinary answer says `failure: false`; a turn stopped
before it answered says `failure: true` and keeps the stop notice as its
text; a real failure keeps `failure: true` as ThinkFlowLab#2677 defined it. The
reader keys on the field; matching the wording stays only as the
compatibility path for messages saved before the field existed.

`plugins/workflow/index.js` is not touched: ThinkFlowLab#2744 already landed the
plugin side that classifies by this marker. This PR adds the host field
that feeds it.

## Verification at the tip

Tip SHA: f6e085e
Tree: 51dcc7e — the tree of the local
merge `a7ed91d710e12ce9a22c7f9f1c0b484953eedb43` (parents `6b4e6485` and
`84b372b2`); `.flybot_push.py` re-created it on live main, so this head
names the PR tip and carries that tree.

### Base update

`main` moved twice while this branch waited, and this tip carries both:
the reviewer merged `2ddec2da` as `6b4e6485` (resolving two doc lines by
hand and regenerating `patches/ds-bot/`), and this branch then merged
`84b372b2` as `a7ed91d710e12ce9a22c7f9f1c0b484953eedb43` (no conflicts).
Measured on the merged tree, without a further `generate`:

- `node scripts/ds-bot-patches.mjs check` → `ok ds-bot/ =
Fei2411/DeepSeek-Bot@2b10c70 (tree 5834c00, v0.1.5)`, `ok 10 patches
apply cleanly in order`, `ok core/ equals ds-bot/ + patches/ds-bot/`
- `node scripts/ds-bot-patches.mjs canonical` → `ok patches/ds-bot/ is
what generate writes (generate is a no-op)`
- `git diff --stat 84b372b HEAD` → the same 7 files, `+67/−5`
- `core/test/bot-messages.test.js` +
`plugins/workflow/test/step-body.test.js` together, twice, same shape:
`# tests 37 / # pass 37 / # fail 0` (814 ms / 822 ms)

Ran:

- red first on a clean `f2ef4dfaccedd40315bd585474d0902118dc0ba8` tree
carrying only the test changes (`core/src/host/turns.js` at the base):
`core/test/bot-messages.test.js` gives `# tests 29 / # pass 27 / # fail
2` — `not ok 13 - a joined request in a stopped turn tells its sender,
and is not handed back` and `not ok 14 - ThinkFlowLab#2742: a completed reply
carries an explicit success marker`.
- after the fix, the same file: `# tests 29 / # pass 29 / # fail 0`
- `plugins/workflow/test/step-body.test.js`: `# tests 8 / # pass 8 / #
fail 0`. That file passes at the base too — it pins the ThinkFlowLab#2744 middleware
behaviour this host field now feeds, and the red-first evidence for this
PR is the core file above.
- `TZ=UTC npm run verify` with
`ARCH_DOCS_BASE=f2ef4dfaccedd40315bd585474d0902118dc0ba8`: exit 0 — core
`# tests 3137 / # pass 3137 / # fail 0`, social 619, tree 140, timeline
161, workflow 509, `.github/scripts` + policy 671, interop 37, live-e2e
34.
- `ARCH_DOCS_BASE=f2ef4dfaccedd40315bd585474d0902118dc0ba8 npm --prefix
core run arch-docs`: `arch-docs: OK (7 changed file(s))`
- `node scripts/ds-bot-patches.mjs generate` on this base wrote
identical bytes; `check`: `10 patches apply cleanly in order`, `core/
equals ds-bot/ + patches/ds-bot/`; `canonical`: `ok patches/ds-bot/ is
what generate writes (generate is a no-op)`
- `node .github/scripts/changelog.mjs check`: `17 fragment(s) in
changelog.d/ are well formed`

Not run: `dev/bench.sh` — no bench was started for this branch, and the
change is one delivery object in the host turn path. No real-model
end-to-end run: `core/test/bot-messages.test.js` drives the turn
lifecycle through the same doubles it already used for ThinkFlowLab#2677, so the new
assertions cover the marker as delivered, not a live model.

## Two deviations from a plain source edit

1. `core/src/host/turns.js` is carried by
`patches/ds-bot/0001-core-fixes.patch` (`+34/-2` on this base) and has
no `patches/ds-bot/files/` mirror — that file is patch-only upstream,
like the rest of 0001's host edits. `generate` on this base reports it
as the patch's own content; the committed series is what it writes.
2. The user-visible change carries
`changelog.d/2742-turns-failure-marker.fixed.md`, and both doc pages
that state the contract — `docs/subsystems/core-host.md` (the reply's
own outcome) and `docs/subsystems/workflow.md` (a step classifies by the
marker; the prose shapes are the old-host path) — gain one sentence
each.

## How an old reader sees it

A consumer that does not know `failure` behaves exactly as before: it
never reads the field and sees the same `text` it always saw. A host
that does not send the field (messages saved before this change) still
works through the prose fallback, which is unchanged and reads the
notice shapes only at the start of the text (ThinkFlowLab#2741). Nothing is renamed
or removed, and a real failure's `failure: true` keeps the meaning ThinkFlowLab#2677
gave it.

## Checkpoint
<!-- flybot:checkpoint -->
```json
{
  "pr": 2756,
  "claim": "Dev-1",
  "issue": 2742,
  "summary": "Every reply a host turn completes toward the sender now carries the outcome as a field: an ordinary answer says failure: false, a turn stopped before it answered says failure: true with the stop notice as its text, and a real failure keeps failure: true as ThinkFlowLab#2677 defined it. A workflow step therefore classifies a step by the marker instead of parsing the sentence, so a stop notice can no longer advance a run and an answer quoting a failure sentence can no longer fail one. plugins/workflow/index.js is untouched (ThinkFlowLab#2744 landed that side); this PR adds the host field it reads, plus a step-body test pinning that classification. Red first at f2ef4df: 2 of 29 cases in core/test/bot-messages.test.js failed; the same file is 29/29 after the fix.",
  "files": [
    "core/src/host/turns.js",
    "core/test/bot-messages.test.js",
    "plugins/workflow/test/step-body.test.js",
    "docs/subsystems/core-host.md",
    "docs/subsystems/workflow.md",
    "changelog.d/2742-turns-failure-marker.fixed.md",
    "patches/ds-bot/0001-core-fixes.patch"
  ],
  "checks": {
    "ran": "TZ=UTC npm run verify with ARCH_DOCS_BASE=f2ef4dfa; core arch-docs; node scripts/ds-bot-patches.mjs generate/check/canonical; node .github/scripts/changelog.mjs check",
    "result": "green",
    "at": "f6e085e40d4db4a26268c83346659d7217827ff1"
  },
  "next": "waiting for the review side's Verdict naming b6cf9b0"
}
```
<!-- /flybot:checkpoint -->

Co-authored-by: Birfy <15378933+Birfy@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

workflow: a step the host stopped before it answered still reads as done

2 participants