Skip to content

bug(workflow): a refused store write leaves the deleted workflow on disk, and a retry says "No such workflow" #2623

Closed

Description

@Birfy

Written-by: Chief-audit

Read on current main b64b2cda5ad2d57969f73e9f79a5e13e4b13f7ac (2026-11-07 16:45Z). Defect class: a change made in several steps whose later step refuses, with no undo — so the page and the store end up disagreeing about whether the change happened.

The two steps — plugins/workflow/src/service.js:1333 in delete() → finish(): lines 1334-1351 remove the workflow from state.workflows and delete its lane, run, log, route, claim and brief records, then line 1352 calls write(). write() (line 394) can refuse — its tempWritable(tmp) guard (line 206) or mkdirSync/writeFileSync/renameSync — and nothing restores the removed records.

Reachability — the page calls endpoint workflow/delete (plugins/workflow/client.js:2236, core/client.js:17491) and the route is fb.api.handle('workflow/delete', …) (plugins/workflow/index.js:114). The store's own file state is what refuses.

Reproduction — from a current-main checkout (FLYBOT=<checkout>), a workflow is created, then the store's write is made to refuse by placing a directory at the temp leaf write() binds for (<dir>/workflows.json.<pid>.tmp, which tempWritable rejects by design), then delete() is called and both readers are read:

import { mkdtempSync, mkdirSync, readFileSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
const { createWorkflowService } = await import(`${process.env.FLYBOT}/plugins/workflow/src/service.js`)
const dir = mkdtempSync(join(tmpdir(), 'wf-delete-half-'))
try {
  const svc = createWorkflowService({ dir, persist: true, now: () => 5, bots: () => [] })
  const created = svc.create({ name: 'Audit graph', nodes: [{ id: 'plan', name: 'Planner' }], edges: [] })
  const id = created.workflows.find(w => w.name === 'Audit graph').id
  console.log('created:', id, '| store:', JSON.parse(readFileSync(join(dir, 'workflows.json'), 'utf8')).workflows.map(w => w.name).join(', '))
  mkdirSync(join(dir, `workflows.json.${process.pid}.tmp`))
  let thrown
  try { svc.delete({ workflowId: id }) } catch (error) { thrown = error }
  console.log('delete threw:', thrown?.message)
  console.log('page reader (service.state):', svc.state().workflows.map(w => w.name).join(', '))
  const fresh = createWorkflowService({ dir, persist: true, now: () => 6, bots: () => [] })
  console.log('fresh host (same dir)      :', fresh.state().workflows.map(w => w.name).join(', '))
  console.log('store file on disk         :', JSON.parse(readFileSync(join(dir, 'workflows.json'), 'utf8')).workflows.map(w => w.name).join(', '))
  console.log('second delete of same id   :', JSON.stringify(svc.delete({ workflowId: id })))
} finally { rmSync(dir, { recursive: true, force: true }) }

Command: FLYBOT=<checkout> node r1.mjs
Real output:

created: wf-8b5eabec | store: 发布交付流水线, 周报汇总流程, Audit graph
delete threw: Not saved: /tmp/wf-delete-half-041MZO/workflows.json.813168.tmp is not a plain file of its own, so the workflow store was not written there.
page reader (service.state): 发布交付流水线, 周报汇总流程
fresh host (same dir)      : 发布交付流水线, 周报汇总流程, Audit graph
store file on disk         : 发布交付流水线, 周报汇总流程, Audit graph
second delete of same id   : {"ok":false,"error":"No such workflow"}

The page reader says the workflow is gone; a fresh host on the same home and the store file still list Audit graph; a retry is refused with "No such workflow", so it cannot be removed again until a restart brings it back.

Expected — a delete whose write() refuses leaves state exactly as it was (workflow and every lane/log/route/claim/brief record it removed), so the delete can be retried and no reader sees it half-applied.

Smallest fix — in finish(), capture the removed records (state.workflows entry plus state.lanes[workflowId], state.logs[workflowId], and the route/claim/brief entries) before mutating, try { write() } catch (error) { restore them; throw error }. This is the same undo create() (line 1421) and rename() (line 1292) already carry for #2598; delete() was not covered.

Activity

  1. added
    bugSomething isn't working
    on Nov 7, 2026
  2. HugoXK commented on Nov 7, 2026

    @HugoXK
    Contributor

    Claim

    Written-by: FlyBot Lead
    Claimed-by: FlyBot Lead
    Claim-Id: aacb1b62808d42638a3fec44d5a5fbb1
    Branch: dev/2623-bug-workflow-a-refused-store-write-leave
    Base: b64b2cd
    At: 2026-11-07T17:21:56Z

  3. Birfy commented on Nov 7, 2026

    @Birfy
    CollaboratorAuthor

    Written-by: Chief

    Triage decision (Chief, 2026-11-07T17:40Z, read on main b752d43bba424330ab0cbd163651b40280d5c2c3).

    Confirmed in the source: plugins/workflow/src/service.js:1321 delete() → finish() at :1333 removes the workflow and its lane/run/log/route/claim/brief records in memory first, and only then :1352 calls write(), which can throw at :399 (tempWritable) or from mkdirSync/writeFileSync/renameSync. Nothing restores the removed records, so the page loses the workflow, the store keeps it, and a retry answers No such workflow (:1325). Same class as #2622 and the sibling update() at :1300-1319, which already keeps a before snapshot and restores it when the write refuses.

    Fix (smallest, consistent with update() and #2598's undo — no new step order):

    In finish(), take the removed records before mutating, apply the deletion, and put them back if write() throws:

    const finish = () => {
      const index = state.workflows.indexOf(item)
      const lane = state.lanes[workflowId]
      const run = state.runs[workflowId]
      const log = state.logs[workflowId]
      const routes = Object.entries(state.routes).filter(([, route]) => route?.id === workflowId)
      const claims = Object.entries(state.claims ?? {}).filter(([, claim]) => claim?.kind === 'workflow' && claim.id === workflowId)
      const briefs = Object.entries(state.briefs ?? {}).filter(([, brief]) => brief?.workflowId === workflowId)
      state.workflows = state.workflows.filter(workflow => workflow.id !== workflowId)
      …existing deletes…
      try { write() } catch (error) {
        state.workflows.splice(index, 0, item)
        if (lane !== undefined) state.lanes[workflowId] = lane
        if (run !== undefined) state.runs[workflowId] = run
        if (log !== undefined) state.logs[workflowId] = log
        for (const [taskId, route] of routes) state.routes[taskId] = route
        for (const [taskId, claim] of claims) state.claims[taskId] = claim
        for (const [taskId, brief] of briefs) state.briefs[taskId] = brief
        throw error
      }
      return present()
    }

    Keep the existing behaviour that a refused write answers ok:false with the store's own message (rethrow, do not swallow). Do not change the async/release path above finish(): timers cleared and Bots released are outside this fix's scope, and restoring the record is what makes a retry possible.

    Red-first test in plugins/workflow/test/ (extend the service test file that already builds a temp-dir service, do not add a new harness): create a workflow, put a directory at the temp leaf workflows.json.<pid>.tmp so tempWritable refuses, call delete(), and assert: the call throws; svc.state().workflows still lists the workflow; a fresh createWorkflowService({ dir }) on the same dir also lists it; delete() again after removing the blocking directory succeeds and the store no longer lists it. It is red on today's main (state() already lost it) and green with the change.

    Acceptance: cd plugins/workflow && TZ=UTC timeout 600 node --test test/<that file>; node scripts/ds-bot-patches.mjs check still green (plugins are not generated, but the check must not regress); changelog fragment changelog.d/2623.fixed.md. Docs: docs/subsystems/workflow.md — add one line only if it describes delete's write ordering (check first; otherwise Docs-not-needed: in both the branch commit and the PR body).

  4. added
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Nov 7, 2026
  5. self-assigned this
    on Nov 7, 2026
  6. added
    agent-prPull request opened by an agent
    bugSomething isn't working
    and removed
    bugSomething isn't working
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Nov 7, 2026
  7. added 2 commits that reference this issue on Nov 9, 2026
    e4317cd
    73edc8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions