From d5d97b9c8ad40d171078b3e2d3f2da7a96ad7d9f Mon Sep 17 00:00:00 2001 From: Danyang Chen Date: Wed, 28 Oct 2026 14:44:25 +0800 Subject: [PATCH 1/2] fix(workflow): show a failed sibling run on the stage with its reason (#1493) The snapshot only carried running runs, so the failed-run handling added in #1494 was dead while another run was live: the header and the timeline never showed the sibling that failed. A run that ends failed now records what ended it (step failure, stop, hand-off failure, host reap), stays in runs while no running run carries its task, and its history row keeps the run id and that reason, so the Runs tab can show the time and the reason. --- .../1493-workflow-failed-runs.fixed.md | 2 +- docs/subsystems/workflow.md | 4 +- plugins/workflow/client.js | 62 +++++++++- plugins/workflow/src/service.js | 69 +++++++++-- plugins/workflow/test/client.test.js | 74 +++++++++++ plugins/workflow/test/failed-sibling.test.js | 115 ++++++++++++++++++ 6 files changed, 308 insertions(+), 18 deletions(-) create mode 100644 plugins/workflow/test/failed-sibling.test.js diff --git a/changelog.d/1493-workflow-failed-runs.fixed.md b/changelog.d/1493-workflow-failed-runs.fixed.md index 68f74e4f7..e6b3a758e 100644 --- a/changelog.d/1493-workflow-failed-runs.fixed.md +++ b/changelog.d/1493-workflow-failed-runs.fixed.md @@ -1 +1 @@ -Workflow stage now shows failed sibling runs in the header and timeline, not only in the Runs tab. ([#1493](https://github.com/ThinkFlowLab/FlyBot/issues/1493)) +Workflow stage now shows failed sibling runs in the header and timeline, not only in the Runs tab, and a Runs row shows when the run ended and why it failed. ([#1493](https://github.com/ThinkFlowLab/FlyBot/issues/1493)) diff --git a/docs/subsystems/workflow.md b/docs/subsystems/workflow.md index 9d32ad9d7..0897f3889 100644 --- a/docs/subsystems/workflow.md +++ b/docs/subsystems/workflow.md @@ -46,8 +46,8 @@ Everything in `plugins/workflow/`: the graph rules, the run store, the matcher, 1. Friends' Bots are not nodes. A `remote` Bot is refused by create, define and the matcher. 2. The canvas and the run share one graph: nodes, edges and positions. A drag commits `workflow/graph`, but only while **Edit** is on — a factory preset is copied first (#1378). Status highlighting reads that same graph. Replacing edges does not recompute who is already running. The Settings canvas paints that graph on the same 960px design plane, scaled to the width the settings page gives it, so the canvas is not a second scroll container inside the page's own scroll; its pointer deltas are divided by the scale, and its wires keep a stronger stroke than a plain divider so an edge can be followed at that size (#1483). -3. **A live run advances only when its ephemeral Bot replies, following the graph's edges in order.** That reply advances the run and may appear on the workflow stage log; it is not delivered into a persistent sidebar Bot conversation (for example Chief). The `workflow-step` send middleware denies those replies after `stepResult` runs. Nodes start only when their forward predecessors finish and edges reach them. **Early termination (#1537, #1538):** When a node completes with `needs-human` or `already-claimed` detected in its outputs or reply text (see `checkEarlyEnd` in Key files), the run ends immediately with status `done`, sets `run.earlyEnd` to the condition name, and does not advance to or spawn successor nodes. This allows Triage, Claim, or any step to terminate the pipeline cleanly when human intervention is required or the issue is already owned by another agent. There is no scripted or seeded run path: `workflow/state` exposes only `live === true` runs. A run does not create a schedule. `scheduleLabel` is display text. When the host can create Bots, starting a live run (`assign`, `run_workflow`, `ingest`) spawns one ephemeral Bot per node from that node's prompt, role, model and tools, sends it the step from the Main Bot, and removes every instance when the run finishes, fails for good, or is reset. A `failOnce` retry keeps them. They are hidden and left out of the sidebar. Loading the plugin does not spawn Bots for the default workflows. Plugin unload (`dispose`) and the load-time `reap` both end a live run still marked `running`: in-flight nodes are failed, the run is archived as `failed`, and any ephemeral Bot ids are released. A run already written as `running` with no ephemeral map (an older unload) is healed the same way on the next load (#1414). Loading is also the one moment a run the host left behind is noticed when its ephemeral map is still present: `reap` releases those instances and records the run as `failed`. Nothing resumes it. A run that already ended keeps its own status (#1388). `workflow/reset` stopping one live run ends it the same way before its Bots are released: nodes still `running` become `failed`, so the strip cannot keep painting a run that is over (#1424). A dispatch that does not land is not marked handed. The chat names the code, and the next hand-off tries that node again. It is not sent twice in the same hand-off. A reply that says `could not answer your message` or `model call failed` fails that node and does not start its successor. A 429 or rate-limit mention in any other reply does not. -4. Each workflow keeps a FIFO queue and up to `maxConcurrent` live runs (default 2, an integer from 1 to 8 on the record, via `workflow/concurrency`, or in Settings → Workflow graphs). A task that does not fit is enqueued. When a run finishes, every workflow with a free slot is drained, oldest enqueue first. A new assign on a workflow that already has a queue waits behind it. A task does not replace a running run or release that run's Bots. Migrating an old store keeps only runs that were still `running` with ephemeral Bot ids; other legacy scripted rows are dropped from active state (finished tasks may remain in lane history). History records a finished or failed live run. Lowering the cap does not stop a run that has already started. Assigning `{ botId }` claims the card for that local Bot and does not enqueue. A remote Bot is refused. `activeCount` is `runningOf(lane)` — live runs with `live === true` and `status === 'running'`. The stage aggregate chip and the Runs tab (运行记录) must agree: every such run appears as its own running row and progress card, and the chip count does not exceed that list. +3. **A live run advances only when its ephemeral Bot replies, following the graph's edges in order.** That reply advances the run and may appear on the workflow stage log; it is not delivered into a persistent sidebar Bot conversation (for example Chief). The `workflow-step` send middleware denies those replies after `stepResult` runs. Nodes start only when their forward predecessors finish and edges reach them. **Early termination (#1537, #1538):** When a node completes with `needs-human` or `already-claimed` detected in its outputs or reply text (see `checkEarlyEnd` in Key files), the run ends immediately with status `done`, sets `run.earlyEnd` to the condition name, and does not advance to or spawn successor nodes. This allows Triage, Claim, or any step to terminate the pipeline cleanly when human intervention is required or the issue is already owned by another agent. `workflow/state` exposes only `live === true` runs: the running ones, plus the last run when it is still `live` and ended `failed` — a failed sibling stays on the stage beside whatever is running now, with `run.failure` (`{ code, at, nodeName?, text? }`) naming what ended it, so the room does not have to open Runs to learn that a run failed (#1493, #1494). The failed sibling is left out when a running run carries the same task id: that is a retry, and the live run keeps the row (#1507). History entries carry the run `id` and the same `failure`, so a Runs row shows when the run ended and why, and a row for a run already on the stage is not listed twice. A run does not create a schedule. `scheduleLabel` is display text. When the host can create Bots, starting a live run (`assign`, `run_workflow`, `ingest`) spawns one ephemeral Bot per node from that node's prompt, role, model and tools, sends it the step from the Main Bot, and removes every instance when the run finishes, fails for good, or is reset. A `failOnce` retry keeps them. They are hidden and left out of the sidebar. Loading the plugin does not spawn Bots for the default workflows. Plugin unload (`dispose`) and the load-time `reap` both end a live run still marked `running`: in-flight nodes are failed, the run is archived as `failed`, and any ephemeral Bot ids are released. A run already written as `running` with no ephemeral map (an older unload) is healed the same way on the next load (#1414). Loading is also the one moment a run the host left behind is noticed when its ephemeral map is still present: `reap` releases those instances and records the run as `failed`. Nothing resumes it. A run that already ended keeps its own status (#1388). `workflow/reset` stopping one live run ends it the same way before its Bots are released: nodes still `running` become `failed`, so the strip cannot keep painting a run that is over (#1424). A dispatch that does not land is not marked handed. The chat names the code, and the next hand-off tries that node again. It is not sent twice in the same hand-off. A reply that says `could not answer your message` or `model call failed` fails that node and does not start its successor. A 429 or rate-limit mention in any other reply does not. +4. Each workflow keeps a FIFO queue and up to `maxConcurrent` live runs (default 2, an integer from 1 to 8 on the record, via `workflow/concurrency`, or in Settings → Workflow graphs). A task that does not fit is enqueued. When a run finishes, every workflow with a free slot is drained, oldest enqueue first. A new assign on a workflow that already has a queue waits behind it. A task does not replace a running run or release that run's Bots. Migrating an old store keeps only runs that were still `running` with ephemeral Bot ids; other legacy scripted rows are dropped from active state (finished tasks may remain in lane history). History records a finished or failed live run. Lowering the cap does not stop a run that has already started. Assigning `{ botId }` claims the card for that local Bot and does not enqueue. A remote Bot is refused. `activeCount` is `runningOf(lane)` — live runs with `live === true` and `status === 'running'`. The stage aggregate chip and the Runs tab (运行记录) must agree: every such run appears as its own running row and progress card with its own `status` (never a hardcoded Running), the chip count does not exceed that list, and a failed sibling appears with its time and reason. 5. A node declares up to eight inputs and eight outputs. A required input is an output of the same name on an ancestor, or `task`, `title`, or `brief` on a root. A start with no provider is refused, and Bots spawned for that attempt are released. On a Bot-backed run a successor stays queued until its required inputs are present. If every forward predecessor is done, failed or skipped and an input is still missing, that node fails and the run stops. The dispatch text carries `Input : `. The Bot reports `OUTPUT : `; a single declared output may be the reply body. Nodes start in edge order. 6. Each node is a real ephemeral Bot for that run, then it is removed. A permanent sidebar Bot (`botId`) is optional and is not required for the step to run. **File and shell rights are the same as a normal sidebar Bot's:** every node Bot gets the `danger-full-access` preset, and `EPHEMERAL_DENIED_TOOLS` still refuses the team-admin tools at the guard and approval layers (#1466, 29 names). **`node.tools` selects connectors only:** `spawn` passes the node's cleaned list to `fb.bot.create`, the host records it as `stepTools`, and `connector-access.js` allows `mcp__github__*` for `github` and `mcp__composio__*` for `composio` — a connector the node did not name is refused, and a node that names none reaches none. The list is also stored in `fields.tools` for the UI. A hidden run Bot cannot show an approval card, so the approval seam answers the same way the guard does. The host stores this on the run Bot as `stepTools` and `permission`. 7. A loop edge (`loop: true`, `max` from 1 to 8, default 3, optional `until: { output, eq }`) is the only cycle for success paths. The body runs again until `until` matches or `max` runs have finished, and the forward edges wait until then. Restarting the body drops the branch choice made inside it and queues a sibling that the previous pass skipped, so the next pass can take the other side. A `when: { output, eq }` edge is taken only when that output equals `eq`. `when: { else: true }` is taken only when no other `when` matched. A node whose forward edges are all closed is `skipped` and does not block a join. A run cannot spin: every loop has a max. An `onFail` edge (`onFail: true`, `max` from 1 to 8, default 3) provides failure recovery: when a node fails, if an `onFail` edge points from that node back to an upstream target and recovery attempts remain (tracked in `failAttempts` per edge), the workflow resets both nodes to their pre-failure states (the target to `running`, the failing node to `queued`) and re-runs from the target through the failing node. After `max` attempts are exhausted, the node fails permanently and the run stops. An `onFail` edge does not interfere with `failOnce` retry (which restarts the same node once) or success-path `loop` edges. Settings → Workflow graphs can add a link, a condition, a loop, or a failure recovery edge. diff --git a/plugins/workflow/client.js b/plugins/workflow/client.js index 4c04f4a76..94921555f 100644 --- a/plugins/workflow/client.js +++ b/plugins/workflow/client.js @@ -55,6 +55,11 @@ window.__ModuleLoader__.load({ 'Local Bots': '本机 Bot', 'No workflows yet.': '还没有 Workflow。', 'Waiting for a task': '等待任务', + 'A step failed': '有一步失败', + '{node} failed': '{node} 失败', + Stopped: '已停止', + 'The hand-off failed': '交接失败', + 'The host ended this run': '宿主中断了这次运行', 'Message the workflow': '发消息', Trigger: '触发方式', Manual: '手动，从任务看板分配', @@ -168,6 +173,34 @@ window.__ModuleLoader__.load({ } const STATUS_KEY = { queued: 'Queued', running: 'Running', done: 'Done', failed: 'Failed', skipped: 'Skipped', idle: 'Idle' } + // Why a run failed, from the record the host keeps (#1493). A failure it did not + // name still reads Failed: the reason is added information, never a requirement. + const FAILURE_KEY = { + stopped: 'Stopped', + 'dispatch-failed': 'The hand-off failed', + 'host-ended': 'The host ended this run', + } + function failureText(run) { + const failure = run?.failure + if (!failure) return tr('Failed') + if (failure.code === 'step-failed') { + const head = failure.nodeName ? tr('{node} failed', { node: failure.nodeName }) : tr('A step failed') + return failure.text ? `${head} · ${failure.text}` : head + } + return tr(FAILURE_KEY[failure.code] ?? 'Failed') + } + // A row without a time (an older history entry) stays a row; it just shows none. + function runTime(at) { + const ms = Number(at) + if (!Number.isFinite(ms) || ms <= 0) return '' + const when = new Date(ms) + if (Number.isNaN(when.getTime())) return '' + const pad = (value) => String(value).padStart(2, '0') + const clock = `${pad(when.getHours())}:${pad(when.getMinutes())}` + const today = new Date() + const sameDay = when.getFullYear() === today.getFullYear() && when.getMonth() === today.getMonth() && when.getDate() === today.getDate() + return sameDay ? clock : `${pad(when.getMonth() + 1)}-${pad(when.getDate())} ${clock}` + } const COLUMN_KEY = { 'not-started': 'Not started', 'in-progress': 'In progress', done: 'Done' } const COLUMN_OF = { open: 'not-started', claimed: 'not-started', working: 'in-progress', blocked: 'in-progress', 'result-ready': 'in-progress', accepted: 'done', cancelled: 'done', failed: 'done' } const COLORS = { Planner: '#8b5cf6', Writer: '#f0509a', Coder: '#1fbf6a', Reviewer: '#f5a524', Tester: '#14b8a6', Deployer: '#2f7cf6', '研究员': '#a78bfa', '数据分析师': '#ec4899' } @@ -388,7 +421,9 @@ window.__ModuleLoader__.load({ const running = (run.running ?? []).map(id => nodeOf(workflow, id)).filter(Boolean) const chipClass = run.status === 'done' ? 'done' : (run.status === 'failed' ? 'failed' : (run.paused ? 'queued' : (running[0] ? 'running' : 'queued'))) const chipLabel = run.status === 'done' ? tr('Done') : (run.status === 'failed' ? tr('Failed') : (run.paused ? tr('Pause') : (running[0] ? tr('Running') : tr('Queued')))) - const meta = run.paused + const meta = run.status === 'failed' + ? failureText(run) + : run.paused ? `${tr('Pause')} · ${running.map(node => node.name).join(' ‖ ') || '—'} · ${run.doneCount}/${run.nodeCount}` : (running[0] ? `${tr('Running')} · ${running.map(node => node.name).join(' ‖ ')} · ${run.doneCount}/${run.nodeCount}` @@ -551,12 +586,20 @@ window.__ModuleLoader__.load({ const liveRuns = liveStageRuns(workflow) if (liveRuns.length === 0 && history.length === 0 && queued.length === 0) return null const rows = liveRuns.map(run => ({ + id: run.id, task: run.task, - status: 'running', + status: run.status, + at: run.updatedAt, + failure: run.failure, current: workflow.run?.id === run.id, })) for (const task of queued) rows.push({ task, status: 'queued', current: false }) - for (const item of history) rows.push({ ...item, current: false }) + const seen = new Set(rows.map(row => row.id ?? `${row.task?.id ?? ''}|${row.at ?? ''}`)) + for (const item of history) { + const key = item.id ?? `${item.task?.id ?? ''}|${item.at ?? ''}` + if (seen.has(key)) continue + rows.push({ ...item, current: false }) + } return rows } @@ -726,8 +769,13 @@ window.__ModuleLoader__.load({ const rows = runHistoryRows(workflow) if (!rows) return h('div', { className: 'wf-muted' }, tr('No runs yet.')) return h('div', null, rows.map((item, index) => h('div', { key: index, className: 'wf-hist' }, - h('span', { title: item.task?.title ?? '' }, taskSummary(item.task, item)), - h('span', { className: `wf-chip ${item.status === 'done' ? 'done' : item.status === 'running' ? 'running' : item.status === 'failed' ? 'failed' : 'queued'}` }, tr(item.status === 'done' ? 'Done' : item.status === 'running' ? 'Running' : item.status === 'failed' ? 'Failed' : 'Queued'))))) + h('span', { className: 'wf-hist-task', title: item.task?.title ?? '' }, taskSummary(item.task, item)), + h('span', { className: 'wf-hist-meta' }, + item.at ? h('span', { className: 'wf-hist-time' }, runTime(item.at)) : null, + item.status === 'failed' + ? h('span', { className: 'wf-hist-reason', title: failureText(item) }, failureText(item)) + : null), + h('span', { className: `wf-chip ${item.status === 'done' ? 'done' : item.status === 'running' ? 'running' : item.status === 'failed' ? 'failed' : 'queued'}` }, tr(STATUS_KEY[item.status] ?? 'Queued'))))) } function RenameModal({ workflow, onClose, onConfirm }) { @@ -1558,6 +1606,10 @@ window.__ModuleLoader__.load({ .wf-kv{display:flex;justify-content:space-between;gap:16px;padding:8px 0;font-size:12px} .wf-kv span:first-child{color:var(--bt-ink-2,rgba(20,20,20,.6))} .wf-hist{display:flex;justify-content:space-between;align-items:center;gap:8px;padding:10px 0;border-bottom:1px solid var(--bt-line,rgba(20,20,20,.08));font-size:12px} +.wf-hist-task{min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap} +.wf-hist-meta{display:flex;align-items:center;gap:8px;min-width:0;margin-left:auto} +.wf-hist-time{color:var(--bt-ink-3,#9a9a9a);font-variant-numeric:tabular-nums;white-space:nowrap} +.wf-hist-reason{max-width:220px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:#e5484d} .wf-ctl{display:flex;gap:6px;flex-wrap:wrap;margin-top:14px} .wf-mask{position:fixed;inset:0;background:rgba(0,0,0,.28);display:flex;align-items:center;justify-content:center;z-index:80;padding:16px} .wf-modal{width:560px;max-width:100%;max-height:calc(100vh - 32px);background:var(--bt-main,#fcfcfc);color:var(--bt-ink,#141414);border-radius:16px;border:1px solid var(--bt-line,rgba(20,20,20,.08));display:flex;flex-direction:column;overflow:hidden} diff --git a/plugins/workflow/src/service.js b/plugins/workflow/src/service.js index 4229f126f..4c8d787e9 100644 --- a/plugins/workflow/src/service.js +++ b/plugins/workflow/src/service.js @@ -81,6 +81,7 @@ function publicRun(workflow, run) { updatedAt: run.updatedAt, ...(run.live === true ? { live: true } : {}), ...(run.paused === true ? { paused: true } : {}), + ...(run.failure ? { failure: { ...run.failure } } : {}), ...(run.earlyEnd ? { earlyEnd: run.earlyEnd } : {}), ...(run.ephemeral && Object.keys(run.ephemeral).length > 0 ? { ephemeral: { ...run.ephemeral } } : {}), } @@ -182,7 +183,7 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no queueDepth: (laneOf(item.id).queue ?? []).length, queue: (laneOf(item.id).queue ?? []).map(task => ({ id: task.id, title: task.title })), history: [...(laneOf(item.id).history ?? [])], - runs: runningOf(laneOf(item.id)).map(run => publicRun(item, run)), + runs: [...runningOf(laneOf(item.id)), ...failedSiblings(laneOf(item.id))].map(run => publicRun(item, run)), preview: rowPreview(item, previewRun(item.id)), run: publicRun(item, displayRun(item.id)), messages: state.logs[item.id] ?? [], @@ -203,6 +204,18 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no return (lane?.active ?? []).filter(run => run && run.live === true && run.status === 'running') } + // A failed run is still the room's business: the stage and the Runs tab show it + // beside whatever is running now, naming what ended it (#1493). A failure of the + // task that is live again is a retry, not a sibling, so the live run keeps the + // row and the older attempt stays in history (#1507). + function failedSiblings(lane) { + const failed = lane?.last + if (failed?.live !== true || failed.status !== 'failed' || !failed.task) return [] + const running = runningOf(lane) + if (running.some(run => run.task?.id && run.task.id === failed.task.id)) return [] + return [failed] + } + function maxOf(item) { const value = Number(item?.maxConcurrent) if (!Number.isInteger(value) || value < 1) return DEFAULT_CONCURRENT @@ -324,6 +337,19 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no ...(earlyEnded ? { earlyEnd: result.earlyEnd } : {}), } if (Object.keys(handed).length === 0) delete next.handed + if (next.status === 'failed') { + const hole = blocked ? gated.blocked[0] : null + const node = nodeById(item, hole ? hole.id : op.nodeId) + const text = hole + ? `Missing input ${hole.missing.join(', ')}` + : String(op.text ?? '').replace(/\s+/g, ' ').trim() + next.failure = { + code: 'step-failed', + at: next.updatedAt, + ...(node ? { nodeName: node.name } : {}), + ...(text ? { text: text.length > 200 ? `${text.slice(0, 197)}…` : text } : {}), + } + } const lines = linesFor(item, working.task, { type: op.type, nodeId: op.nodeId, @@ -401,7 +427,7 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no lane.active = lane.active.filter(item => item.id !== run.id) lane.last = run if (run.status === 'done' || run.status === 'failed') { - lane.history = [...lane.history, { task: run.task, status: run.status, at: run.updatedAt }].slice(-20) + lane.history = [...lane.history, historyRow(run)].slice(-20) } await releaseAll(ids, release, warn) write() @@ -412,8 +438,9 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no // dispose(), the catch around a step that threw, and reset() stopping one run — // so run.status and run.nodeStatus cannot drift apart and a client stops painting // "Running now" (#1424). A run that already ended keeps its own ending (#1388); - // ephemeral ids stay on the record until release strips them. - function endRun(item, run) { + // ephemeral ids stay on the record until release strips them. `failure` names the + // ending so the stage and the Runs tab can say why, not only Failed (#1493). + function endRun(item, run, failure = { code: 'host-ended' }) { if (run.status === 'done' || run.status === 'failed') return run for (const nodeId of item ? orderedIds(item, run.nodeStatus, 'running') : []) { try { @@ -424,14 +451,35 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no } run.status = 'failed' run.updatedAt = now() + run.failure = { ...failure, at: run.updatedAt } return run } + // A run that ended failed says so once in the stage log: the step-failure paths + // already write their own line, these are the endings that wrote none (#1493). + function failureLine(run) { + const label = summarizeTask(run.task) + const detail = run.failure?.text ? ` · ${run.failure.text}` : '' + return { kind: 'system', tone: 'bad', text: `✕ 失败 · ${label}${detail}` } + } + function archiveCutoffRun(workflowId, run) { const lane = laneOf(workflowId) lane.active = lane.active.filter(item => item.id !== run.id) lane.last = run - lane.history = [...lane.history, { task: run.task, status: run.status, at: run.updatedAt }].slice(-20) + lane.history = [...lane.history, historyRow(run)].slice(-20) + } + + // The Runs tab reads this: a finished run, when it ended, and — for a failure — + // the reason, so the row says more than "Failed" (#1493). + function historyRow(run) { + return { + id: run.id, + task: run.task, + status: run.status, + at: run.updatedAt, + ...(run.failure ? { failure: { ...run.failure } } : {}), + } } // The oldest queued task whose own workflow still has a slot, or null. @@ -552,7 +600,7 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no try { run = await handOff(item, run) } catch (error) { - endRun(item, run) + endRun(item, run, { code: 'dispatch-failed', text: String(error?.message ?? error).slice(0, 200) }) await releaseRun(item.id, run) return refuse(error) } @@ -884,12 +932,12 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no const ids = Object.values(run.ephemeral ?? {}) // Stopping one run ends it the same way a reap does: the node map settles // with the run, or the strip keeps painting the node it was on (#1424). - const ended = { ...endRun(item, run) } + const ended = { ...endRun(item, run, { code: 'stopped' }) } delete ended.ephemeral delete ended.handed lane.active = lane.active.filter(itemRun => itemRun.id !== runId) lane.last = ended - lane.history = [...lane.history, { task: ended.task, status: ended.status, at: ended.updatedAt }].slice(-20) + lane.history = [...lane.history, historyRow(ended)].slice(-20) push(workflowId, [{ kind: 'system', tone: 'bad', text: `■ 已停止 · ${summarizeTask(run.task)}` }], now()) write() return { ok: true, ids } @@ -1059,7 +1107,8 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no for (const { workflowId, run } of abandoned) { try { if (run.ephemeral) stale.push(...Object.values(run.ephemeral)) - endRun(workflow(workflowId), run) + endRun(workflow(workflowId), run, { code: 'host-ended' }) + push(workflowId, [failureLine(run)], now()) delete run.ephemeral delete run.handed archiveCutoffRun(workflowId, run) @@ -1082,7 +1131,7 @@ export function createWorkflowService({ dir, persist = true, now = () => Date.no for (const [workflowId, lane] of Object.entries(state.lanes ?? {})) { for (const run of [...(lane.active ?? [])]) { if (run?.live === true && run.status === 'running') { - endRun(workflow(workflowId), run) + endRun(workflow(workflowId), run, { code: 'host-ended' }) archiveCutoffRun(workflowId, run) endedLive = true } diff --git a/plugins/workflow/test/client.test.js b/plugins/workflow/test/client.test.js index c93814c65..f1e2a895d 100644 --- a/plugins/workflow/test/client.test.js +++ b/plugins/workflow/test/client.test.js @@ -1067,3 +1067,77 @@ test('a stage control that never answers settles as an error', async () => { const value = await client.settledCall(async () => new Promise(() => {}), 'workflow/pause', { workflowId: 'delivery' }, 'The run did not answer', 5) assert.deepEqual(value, { ok: false, error: 'The run did not answer' }) }) + +// #1493: the header lists the runs that are still the room's business, so a +// sibling that failed is on the stage beside the one that is running — and the +// Runs tab says when it ended and why. The sibling is not listed twice: the +// failed run is both a live-stage row and a history row. +test('#1493: a failed sibling run stays on the stage, and its row carries time and reason', async () => { + const graph = delivery() + const running = graph.run + running.id = 'run-1445' + running.task = { id: '#1445', title: 'issue #1445: a Workflow run keeps the team task' } + running.taskSummary = '#1445 · issue #1445: a Workflow run keeps the team task' + running.updatedAt = 1_700_000_000_000 + const failed = { + id: 'run-1460', + live: true, + status: 'failed', + task: { id: '#1460', title: 'issue #1460: every stage chat line is capped' }, + taskSummary: '#1460 · issue #1460: every stage chat line is capped', + nodeStatus: { planner: 'done', writer: 'failed' }, + running: [], + doneCount: 1, + nodeCount: 6, + marks: ['done', 'failed', 'queued', 'queued', 'queued'], + steps: 2, + stepBudget: 10, + startedAt: 1_699_999_996_000, + updatedAt: 1_700_000_000_000, + failure: { code: 'step-failed', nodeName: 'Writer', text: 'the reply came back empty' }, + } + graph.runs = [running, failed] + graph.history = [ + { id: 'run-1460', task: failed.task, status: 'failed', at: failed.updatedAt, failure: failed.failure }, + { task: { id: '#1395', title: 'an older finished run' }, status: 'done', at: 1_699_999_000_000 }, + ] + const call = async (name) => { + if (name === 'workflow/state') return { workflows: [graph], templates: [{ id: 'delivery', name: '发布交付流水线' }] } + return { ok: true, workflows: [graph] } + } + const focused = [] + const view = mount(client.Section, { roster: { bots: [] }, call, actions: {} }) + await settle() + findAll(view.tree, node => node.type === 'button' && textOf(node).includes('发布交付流水线'))[0].props.onClick() + await settle() + + const strips = findAll(view.tree, node => typeof node.props?.['data-run-id'] === 'string') + assert.deepEqual(strips.map(node => node.props['data-run-id']), ['run-1445', 'run-1460']) + const failedStrip = strips[1] + assert.equal(textOf(failedStrip).includes('Failed'), true, 'the sibling strip does not say Failed') + assert.equal(textOf(failedStrip).includes('Writer failed · the reply came back empty'), true, 'the sibling strip does not say why it failed') + assert.equal(textOf(failedStrip).includes('Waiting for a task'), false) + + // Focusing the failed sibling is how the drawer shows its flow, so its row must be clickable. + failedStrip.props.onClick() + await settle() + assert.equal(findAll(view.tree, node => node.props?.['data-run-id'] === 'run-1460' && node.props.className.includes('wf-strip-focus')).length, 1) + + // The strip's own Progress button is the door to its Runs row. + const progress = findAll(failedStrip, node => node.type === 'button' && textOf(node) === 'Progress')[0] + assert.ok(progress, 'the failed strip has no way into its Runs row') + progress.props.onClick() + await settle() + + const rows = findAll(view.tree, node => node.props?.className === 'wf-hist') + assert.equal(rows.length, 3, `the Runs tab listed ${rows.length} rows for two live rows and two history rows, one of which is the failed run itself`) + const row = rows.find(node => textOf(node).includes('#1460')) + assert.ok(row, 'the failed run has no Runs row') + assert.equal(textOf(row).includes('Failed'), true) + assert.match(textOf(row), /\d{2}-\d{2} \d{2}:\d{2}/, 'the Runs row has no time') + assert.equal(textOf(row).includes('Writer failed · the reply came back empty'), true, 'the Runs row has no reason') + const older = rows.find(node => textOf(node).includes('#1395')) + assert.ok(older, 'a history row was dropped') + assert.equal(textOf(older).includes('Done'), true) + view.unmount() +}) diff --git a/plugins/workflow/test/failed-sibling.test.js b/plugins/workflow/test/failed-sibling.test.js new file mode 100644 index 000000000..8fcd0e5d1 --- /dev/null +++ b/plugins/workflow/test/failed-sibling.test.js @@ -0,0 +1,115 @@ +// #1493: a failed run is not a Runs-tab secret. While another run is live, the +// stage keeps the failed sibling, the timeline says the run ended, and the Runs +// row carries the time and the reason. A failure of the task that is live again +// is a retry, not a sibling, so it stays in history and leaves one row (#1507). +// The silent ending the screenshot shows is the reap: a host that came back with +// the run still marked running archived it as failed and said nothing (#1424). +import assert from 'node:assert/strict' +import { mkdtempSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { test } from 'node:test' +import { createWorkflowService } from '../src/service.js' +import { DELIVERY_ID } from '../src/template.js' + +const CLOCK = 1_700_000_000_000 + +const roster = [ + { id: 'main', name: 'Chief', role: 'lead', isMain: true, remote: false }, + { id: 'bot-coder', name: 'Coder', role: '实现', remote: false }, +] + +function harness({ dir = null, clock = CLOCK } = {}) { + let seq = 0 + let now = clock + const svc = createWorkflowService({ + ...(dir ? { dir, persist: true } : { persist: false }), + now: () => now, + bots: () => roster, + spawn: async () => { + seq += 1 + return `eph-${seq}` + }, + release: async () => {}, + dispatch: async () => ({ ok: true }), + }) + return { svc, tick(ms = 1000) { now += ms; return now } } +} + +function delivery(svc) { + return svc.state().workflows.find(item => item.id === DELIVERY_ID) +} + +const label = (run) => `${run.task.id} ${run.status}` + +test('#1493: a failed sibling stays on the stage with the reason it ended', async () => { + const { svc, tick } = harness() + svc.setConcurrency({ workflowId: DELIVERY_ID, maxConcurrent: 2 }) + await svc.assign({ workflowId: DELIVERY_ID, taskId: '#1460', title: 'issue #1460: every stage chat line is capped' }) + tick() + await svc.assign({ workflowId: DELIVERY_ID, taskId: '#1445', title: 'issue #1445: a Workflow run keeps the team task' }) + tick() + const before = delivery(svc) + assert.deepEqual(before.runs.map(label), ['#1460 running', '#1445 running']) + const victim = before.runs.find(run => run.task.id === '#1460') + const stopped = await svc.reset({ workflowId: DELIVERY_ID, runId: victim.id }) + assert.equal(stopped.ok, true) + tick() + + const after = delivery(svc) + assert.equal(after.activeCount, 1, 'activeCount still counts only running runs') + assert.equal(after.run.task.id, '#1445', 'the running run stays the focused one') + assert.deepEqual(after.runs.map(label), ['#1445 running', '#1460 failed'], 'the failed sibling is missing from the stage') + const failed = after.runs.find(run => run.status === 'failed') + assert.equal(failed.id, victim.id) + assert.equal(failed.failure?.code, 'stopped', 'the failed run does not say what ended it') + assert.equal(failed.updatedAt, failed.failure.at) + + const row = after.history.find(item => item.task.id === '#1460') + assert.equal(row.status, 'failed') + assert.equal(row.id, victim.id, 'a history row cannot be matched to the run it came from') + assert.equal(row.at, failed.updatedAt) + assert.equal(row.failure?.code, 'stopped', 'the history row dropped the reason') +}) + +test('#1493: a retry of the same task leaves the live run as its only row', async () => { + const { svc, tick } = harness() + svc.setConcurrency({ workflowId: DELIVERY_ID, maxConcurrent: 2 }) + await svc.assign({ workflowId: DELIVERY_ID, taskId: '#1460', title: 'issue #1460: retried' }) + tick() + const first = delivery(svc).runs[0] + await svc.reset({ workflowId: DELIVERY_ID, runId: first.id }) + tick() + await svc.assign({ workflowId: DELIVERY_ID, taskId: '#1460', title: 'issue #1460: retried' }) + tick() + + const after = delivery(svc) + assert.deepEqual(after.runs.map(label), ['#1460 running'], 'the failed attempt of a task that is live again must not add a row (#1507)') + const row = after.history.find(item => item.task.id === '#1460') + assert.equal(row?.status, 'failed', 'the failed attempt left no history row') +}) + +test('#1493: a run the host lost on reload is archived failed with a reason and a line', async () => { + const dir = mkdtempSync(join(tmpdir(), 'wf-failed-sibling-')) + try { + const first = harness({ dir }) + await first.svc.assign({ workflowId: DELIVERY_ID, taskId: '#1460', title: 'issue #1460: the host went away' }) + first.tick(5000) + // A host that came back without disposing: the store still says the run is running. + const after = harness({ dir, clock: CLOCK + 10_000 }) + await after.svc.reap() + const lane = delivery(after.svc) + assert.deepEqual(lane.runs.map(label), ['#1460 failed']) + const failed = lane.runs[0] + assert.equal(failed.failure?.code, 'host-ended') + assert.equal(failed.failure.at, failed.updatedAt) + const line = [...lane.messages].reverse().find(message => message.tone === 'bad') + assert.ok(line, 'the timeline never says the run was lost') + assert.match(line.text, /失败 · #1460/) + const row = lane.history.find(item => item.status === 'failed') + assert.equal(row.id, failed.id) + assert.equal(row.failure?.code, 'host-ended') + } finally { + rmSync(dir, { recursive: true, force: true }) + } +}) From 4a0ed7fcb330616b758c0f7bec8863cf7bb59fb7 Mon Sep 17 00:00:00 2001 From: Danyang Chen Date: Wed, 28 Oct 2026 14:51:19 +0800 Subject: [PATCH 2/2] test(workflow): drop an unused local the plugin lint refuses --- plugins/workflow/test/client.test.js | 1 - 1 file changed, 1 deletion(-) diff --git a/plugins/workflow/test/client.test.js b/plugins/workflow/test/client.test.js index f1e2a895d..15c2c26f8 100644 --- a/plugins/workflow/test/client.test.js +++ b/plugins/workflow/test/client.test.js @@ -1105,7 +1105,6 @@ test('#1493: a failed sibling run stays on the stage, and its row carries time a if (name === 'workflow/state') return { workflows: [graph], templates: [{ id: 'delivery', name: '发布交付流水线' }] } return { ok: true, workflows: [graph] } } - const focused = [] const view = mount(client.Section, { roster: { bots: [] }, call, actions: {} }) await settle() findAll(view.tree, node => node.type === 'button' && textOf(node).includes('发布交付流水线'))[0].props.onClick()