feat(workflow): on node failure, return upstream and retry (e.g. Review fail → Rework) #1526

Written-by: FlyBot Product

Ask (owner, 2026-10-28)

Workflow feels “dead” when a node fails: there is no way to recover by going back. Owner example: Review fails → go back to Rework, rework, then Review again.

Wanted: a failed node can route back upstream and re-run that path, instead of ending the whole run as Failed with no recovery.

Current behaviour (main)

• Live fail (Bot ok: false, model-call failure text, etc.) calls failNode; unless the node has demo-only failOnce, the run stops (service.js: “Any other failure stops the run”).
• Graph already has success-path loops (loop: true + max + optional until on outputs) and Settings can add loop links — those are not failure recovery.
• retryNode only restarts the same failed node (demo / scripted path); it does not reset upstream Rework → Review.
• Model-call failure replies are explicitly not retried in a loop (README).

Expected

• Owner can express “when this node fails, go back to node X and continue” on the graph (e.g. Review → Rework on failure), editable in Settings like other edges/loops.
• On a live fail that has such a recovery edge: re-queue / re-run from the upstream target through the failing node again (clear or re-queue intermediate statuses as needed), with a max attempts bound so it cannot spin forever.
• Stage / run chrome shows the recovery (retry / back-edge), not a silent dead Failed.
• If no recovery edge (or attempts exhausted), keep today’s stop-as-failed behaviour.
• Optional but welcome: a stage control to Retry from … / re-run from an upstream node on a failed or stopped live run when a recovery path exists.

Non-goals

• Do not require failOnce demo flag for real recovery.
• Do not remove existing success-path loop / when edges.
• Prod upgrade out of scope. Do not mix fix(workflow): ephemeral node Bot replies still land in Chief / persistent Bot chat as queued 「Reply from …」 #1524 / feat(workflow): Settings control for per-Workflow concurrency; remove host-wide live-run cap #1525.

Acceptance

• Graph + Settings can define a failure recovery back-edge (Review fail → Rework) with max attempts.
• Live Review failure with that edge restarts from Rework and can reach Review again; run does not die on first fail.
• Without the edge (or after max attempts), run still fails cleanly.
• Tests + changelog + subsystem/README update per repo rules.