Skip to content

[coordination][deferred] Retire the workflow closed enum → declarative behavior #160

Closed

Description

@serge-ivo

DEFERRED (the larger 'open the vocabulary' frontier). Today capabilities.workflow is a closed enum (JOB_APPLY/CODING_SESSION/INSURANCE_QUOTES) — a creator can only pick one, not define a new autonomous loop. Make durable behavior a declared composition (steps/triggers + a decide-loop) so a new agent type needs no monorepo PR. Pairs with the connector-manifest work (#143) as the second half of making agents fully data. Part of #154; see agent-platform-strategy.md + third-party-agents-plan.md.

Activity

  1. added
    backendBackend / Worker / API work
    coordinationAgent coordination / delegation primitives
    on Aug 3, 2026
  2. added a commit that references this issue on Aug 3, 2026
  3. serge-ivo commented on Aug 3, 2026

    @serge-ivo
    ContributorAuthor

    Plan refinement (2026-08) — the substrate is emerging. Two durable runners now exist — PipelineRunWorkflow (#97, declarative data pipelines) and CodingSessionWorkflow/Pilot (durable coding objective-runner) — plus the pipeline attach write path (2eb08cd) and the trigger form to start pipelines on cron/webhook (2f55433, #134). These are the same shape: a durable, observable, objective/goal runner.

    So retiring the workflow closed enum isn't a from-scratch build — it's converging those two workflows into one declared durable-behavior capability (steps + triggers + a decide-loop), with the pipeline attach/trigger UI as the authoring surface. Pairs with the connector-manifest work (#143) as the second half of "agents as data": tools ✅, connectors ✅, behavior = this. Sequence after coordination #155 proves the delegate→durable-runner path.

  4. serge-ivo commented on Aug 3, 2026

    @serge-ivo
    ContributorAuthor

    Cross-reference: this is the actual blocker on "configure a Coder agent", and it is independent of the supervision work.

    Status of the authoring path:

    So "configurable Coder" decomposes into two independent tracks that can run in parallel:

    1. Its behaviour — this ticket (declarative durable loop instead of the enum).
    2. Its hierarchy — [coordination] Supervision graph as configured platform data — hierarchy belongs to the platform, not inside an agent #183 supervision as platform data, per docs/supervision.md.

    Neither subsumes the other: retiring the enum without supervision gives you a configurable brain with a hardcoded org chart; supervision without the enum gives you a configurable org chart of un-configurable brains.

  5. serge-ivo commented on Aug 7, 2026

    @serge-ivo
    ContributorAuthor

    Reassessment at HEAD (9007758) — the premise has expired; the residual is real but is not "an enum"

    Measured, not assumed. Three findings, two of which cut against doing this ticket as written.

    1. The ticket's premise is no longer true: a creator CAN define a new autonomous loop today

    "a creator can only pick one, not define a new autonomous loop"

    workers/api/wrangler.toml has five [[workflows]] bindings. Two of them are not in the enum at all — PIPELINE_RUN and AGENT_LOOP — and both are selected by data:

    • AGENT_LOOP is the fallthrough. loop-drivers.ts:246 const DRIVERS = { CODING_SESSION: codingDriver } — one entry. JOB_APPLY, BROWSER_TASK, INSURANCE_QUOTES and null all resolve to chatDriver → env.AGENT_LOOP.create(...). So an agent declaring workflow: null already gets a durable, budgeted, cancellable, escalating loop over its own declared tools[], with a run row a supervisor can read. The enum can only route away from that, to exactly one alternative.
    • PIPELINE_RUN is selected by pipeline name from instance config. triggers.ts:306-309 already states this ticket's thesis as shipped fact: "the pipeline is DATA — loaded by name from instance config via startPipelineRun — so adding a new pipeline type never touches the closed workflow capability union."

    And there are two existence proofs in the catalog:

    • site-builder (migration 0057) — runtime: null, workflow: null, entire behaviour in lib/pipelines/site-builder.json + site-deploy.json, with seed-drift.test.ts pinning both to null.
    • portal-watch (migration 0087) — its own seed says it best: "There is no portal-watch code anywhere in the monorepo — no Worker, no pipeline, no slug in a conditional. The agent is this row." Objective, start URL, read-only flag and settings are all data; only the engine is the enum value.

    Six of the ten seeded agents declare workflow: null. Non-null is four slugs across three values.

    2. The enum's actual decision surface is two tables and a lint — much smaller than when this was written

    Complete map of what reading capabilities.workflow decides at HEAD:

    site decides
    loop-drivers.ts:254 loopDriverFor which durable loop the Loop button / start_work / delegate_goal starts. One entry: CODING_SESSION
    trigger-capability.ts:56 REQUIREMENTS whether a run_browse trigger may be wired and a browse run may start. One entry: BROWSER_TASK (#358)
    agent-claims-lint.ts:31 whether catalog copy claiming runtime capability is an integrity violation
    BoardTab.tsx:104 whether the "Run browser task" control renders
    admin.ts, routes/agents.ts, two AgentDetail.tsx read-back and display

    Two negatives worth stating plainly:

    So the mechanical answer to "is retiring the enum smaller now?" is yes, much — it is two one-line tables, and loop-drivers.ts:17-19 already nominates itself as the single place that has to learn a new form.

    3. And that is exactly why it should not be done — the residual is load-bearing in the #352 way

    Being small is not the same as being available. Per #352's finding that sanitizeConnectorManifest rejects the very connector it was meant to absorb, I checked whether the declarative layer can express what the two remaining enum values name. It cannot, and the gap is not a missing step type.

    workflows/pipeline-run.ts:90 is for (let i = 0; i < pipeline.steps.length; i++) — a linear walk over a fixed array. There is no back-edge, no conditional jump, and no step.sleep anywhere in the file. ai_generate is the only model-calling step and it writes a field; nothing reads a model verdict to decide whether to run another round. The one human gate that exists — create_ticket → approve → executeTriggerAction → run_pipeline — starts a fresh run; loop-carried state does not survive.

    The three brains the enum still names (CodingSessionWorkflow, JobApplyWorkflow, BrowserTaskWorkflow) share three properties, and all three are outside that vocabulary:

    1. A model decides each step over a live relay session — decideCodingAction / decideAction / decideBrowserTask inside a bounded round loop, each step reading a snapshot the previous act changed.
    2. A mid-run pause that resumes the SAME run. HANDOFF_WAIT_POLLS = 180 × step.sleep(…, "5 seconds") in all three, resuming at the same step with the same in-memory job / transcript / solvedChallengeUrl. coding-session.ts:322 puts the point exactly: the runner probe sleeps through step.sleep "so a ten-minute pause survives the workflow being evicted."
    3. Exclusive ownership of a physical resource — a claimed coding_sessions lease with sessionOpenedByRun semantics and mid-run machine-switch reclaim; a single browser page held across the whole run.

    The declarative layer can reach the local machine — repo_*, terminal_*, browser_* are ordinary registry tools calling callRunner, and validatePipeline gates on nothing but tool existence. What it cannot do is the loop around them, with a resumable pause. That is the field a manifest has no key for, the same shape as grantModel: "instance-resource" in #352.

    So workflow is not really naming a behaviour. It is naming which physical executor the platform must reach, and that the run may park itself mid-flight and come back. Replacing that string with a JSON document does not make either fact declarative; it relocates it.

    Recommendation

    Reframe, don't retire. The honest remaining frontier is not "kill the enum" — it is "give AGENT_LOOP a declared front", which is a different and much better-shaped ticket: let a creator declare the loop's objective source, its tool set and its decide-prompt, so a new cloud autonomous loop is config. That is genuinely reachable, because AGENT_LOOP already has the loop, the budget, the cancel, the escalation ladder and the run row; it is missing only a declaration surface. portal-watch shows how close this is — it is one pipelineRef-shaped field away from needing no enum value at all.

    The two runtime brains would stay a closed set, and that is the correct answer, not a compromise: they are Tier 0 in agent-platform-strategy.md, they hold an exclusive lease on the owner's machine, and #142 (still open) says a creator should not be able to declare runtime/workflow without trust gating anyway. Retiring the enum before #142 lands would widen the very field #142 exists to narrow.

    Concretely, I'd suggest:

    1. Do now, cheaply: [bug] The creator's Workflow picker offers a value that is bound to nothing, and omits the only one the platform enforces #375 — drop the dead member, add the live one. Two-line tables are only reasonable-sized if their contents are true.
    2. Retitle this ticket to "declarative front for AGENT_LOOP", or close it and open that, keeping this one's analysis as the reason.
    3. Do not convert CODING_SESSION / BROWSER_TASK / JOB_APPLY to a manifest. create_ticket → approve → run_pipeline is the platform's human gate and it is a re-entry, not a pause; until the declarative layer has a resumable wait, a manifest for these three would either lie or be unusable — the same two outcomes [design] Stage 2: register the generic connector OAuth redirect for Drive/WorkDrive/Gmail and retire the dedicated flows (Stage 1 — the registry declaration — has shipped) #352 found.

    Happy to be argued out of (3) if someone wants to add a wait/resume primitive to the pipeline runner first — that is the change that would actually make the enum retirable, and it is worth its own ticket rather than being smuggled in under this one.

  6. serge-ivo commented on Aug 10, 2026

    @serge-ivo
    ContributorAuthor

    Recovered worktree work found for this issue — deliberately NOT landed. Here is what it is, so it is not lost.

    An uncommitted worktree (.claude/worktrees/issue-160, based on 8c6f75b, +343/−11 across lib/agent-workflows.ts, lib/trigger-capability.ts, routes/agents.ts, their tests and docs/agent-platform-strategy.md) was found during a recovery sweep. I read it, and I am leaving it unlanded for two reasons.

    1. The deferral above is substantive, not a scheduling note. The analysis comment on this issue does not say "later" — it says "Reframe, don't retire", and specifically "Do not convert CODING_SESSION / BROWSER_TASK / JOB_APPLY to a manifest", on the grounds that workflows/pipeline-run.ts has no back-edge and no step.sleep, so a manifest for those three "would either lie or be unusable". Landing enum-retirement work against that would be overturning a recorded decision by implementation rather than by argument.

    2. The worktree does not implement this issue anyway. It does not retire the enum. It adds a requiresRuntime field to each WorkflowSpec, a pure workflowRuntimeDenial(), and a 400 at the three declaring doors (POST /agents, PUT /agents/:id, PUT /agents/:id/capabilities) plus triggerActionDenial. Its own doc comment argues the opposite of this ticket — that workflow names "which physical executor the run reaches, and that the run may park itself mid-flight", i.e. the deferral comment's conclusion.

    The invariant in it is real and worth its own ticket

    The finding is that capabilities.workflow and capabilities.runtime are validated independently and never against each other, so {workflow: "BROWSER_TASK", runtime: null} is accepted at all three doors. packages/cli/src/commands/up.ts registers a runner only for an agent with capabilities.runtime != null, so nothing can ever answer that brain: requireLiveRuntime throws 503, executeTriggerAction reads 503 as a transient skip with failure_count untouched, and the owner is told to run pags up on every fire, forever.

    I have not re-verified that trace against current main (35 commits have landed since that worktree was cut), and it is a new 400 on three write paths, so it needs its own scoping — including whether any seeded agent in migrations 0022/0057/0087 would now be refused on update. That is a pags-ba issue, not a rider on a deferred one.

    The worktree has been left untouched as the only copy. Nothing was landed for this issue.

  7. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Re-verified 2026-08-18: the reassessment holds, #375 landed, and the invariant left in a comment is still true and still unfiled

    Three checks against origin/main @ 6ceec4f.

    1. The 2026-08-07 reassessment's premise is unchanged — the enum still is not what this ticket says it is

    workflows/pipeline-run.ts still walks pipeline.steps linearly with no back-edge and no step.sleep. Nothing has been added to the declarative layer that could express a resumable, machine-holding loop. So "reframe, don't retire" stands, and so does the sharper statement it produced: workflow names which physical executor the run must reach, and that the run may park itself mid-flight — not a behaviour.

    2. Item 1 of that recommendation is done

    #375 is closed. INSURANCE_QUOTES is gone from the live vocabulary — lib/agent-workflows.ts is now a spec table whose header states the check that would have caught it ("everything here IS bound"), and agent-capabilities.ts:481-484 filters on read as well as on write so a row persisted while the enum was wider cannot resolve to a workflow nothing runs.

    That was the "do now, cheaply" half. It is done.

    3. Item 2 — retitle or close-and-reopen — has not happened, and that is the live decision

    This ticket still carries the title "Retire the workflow closed enum → declarative behavior", which its own top comment establishes is the wrong goal. The recommended replacement — "a declarative front for AGENT_LOOP" — is a real and much better-shaped ticket: AGENT_LOOP already has the loop, the budget, the cancel, the escalation ladder and the run row, and is missing only a declaration surface for its objective source, tool set and decide-prompt. portal-watch is one pipelineRef-shaped field away from needing no enum value at all.

    Recommend: retitle this issue, or close it and open that one with this analysis carried across. Leaving a ticket titled "retire the enum" on the board, when the recorded decision is not to retire the enum, is how the recovered worktree happened — someone picked it up and started building against the title.

    4. The invariant the 2026-08-10 comment found is still true, and I have now verified it on current main

    That comment said the finding "needs its own scoping" and had not been re-verified. It has now, at HEAD, and it holds exactly:

    capabilities.workflow and capabilities.runtime are validated independently and never against each other. workers/api/src/lib/agent-capabilities.ts:394-422:

    if ("runtime" in o) {
        out.runtime = KNOWN_RUNTIMES.has(o.runtime as …) ? (o.runtime as AgentRuntimeKind) : null;   // :413-415
    }
    if ("workflow" in o) {
        out.workflow = isAgentWorkflow(o.workflow) ? o.workflow : null;                              // :416-418
    }

    Neither branch reads the other, so {workflow: "BROWSER_TASK", runtime: null} is accepted at every declaring door. The downstream consequence is verified too:

    • packages/cli/src/commands/up.ts:89 — instances = instances.filter((i) => i.capabilities?.runtime != null); — so no runner is ever registered for such an agent, no matter how many times the owner runs pags up.
    • workers/api/src/lib/triggers.ts:269-270 — if (status !== 503 && status !== 409) throw e; then const offline = status === 503; — a 503 is read as a transient skip, so failure_count is never incremented and the trigger never auto-disables.

    Net: the agent tells the owner to run pags up on every fire, forever, and nothing on the platform records it as a failure.

    Recommend filing this separately. It is a 400 at three write paths (POST /agents, PUT /agents/:id, PUT /agents/:id/capabilities) plus triggerActionDenial, it needs a check of whether any seeded agent in 0022/0057/0087/0123 would now be refused on update, and it is a normal bug that has nothing to do with retiring an enum. Carrying it as a paragraph inside a deferred design ticket is why it has sat for eight days. The uncommitted worktree at .claude/worktrees/issue-160 is the only copy of a partial implementation and should be treated as a reference for the ticket, not landed against this one.

    Verified: agent-capabilities.ts:394-422 and :481-484, up.ts:89, triggers.ts:269-270, agent-workflows.ts's table and header, pipeline-run.ts's linear walk, the closed state of #375, and that no open or closed issue on this repo covers the runtime/workflow mismatch. Not attempted: I did not touch the worktree and landed nothing.

  8. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Filed as #705 — the runtime/workflow invariant now has its own ticket, so it stops riding on a deferred design issue. This one is free to be retitled or closed on its own merits.

  9. added
    P3: laterDesign, strategy, or deferred pending a decision or demand
    enhancementNew feature or request
    pipelinesPipelines, steps, triggers and the event pump — what CALLS a connector
    on Aug 18, 2026
  10. serge-ivo commented on Aug 23, 2026

    @serge-ivo
    ContributorAuthor

    NOT-ADDRESSED by 49a1bacc — the premise is intact. Recording it so the next audit does not re-litigate the commit.

    Delivered-work audit, 2026-08-23. 49a1bacc ("feat(coder2): declare the Coder as data — two templates, no bespoke code (#160, epic #58)") names this issue and is real progress in the same direction, but it declares a Coder using the EXISTING vocabulary. It does not open the vocabulary, which is what this ticket asks for.

    Verified on main today: capabilities.workflow is still a closed enum. workers/api/src/lib/agent-capabilities.ts:101 declares workflow: AgentWorkflow | null, validated by isAgentWorkflow from lib/agent-workflows.js (:24, :486), and the fallback branches still hand out the same two literals — :498 workflow: "JOB_APPLY", :500 workflow: "CODING_SESSION". (INSURANCE_QUOTES was removed as a derivation at :501 under #375, which narrows the enum rather than retiring it.) A creator still cannot define a new autonomous loop without a monorepo PR.

    What HAS moved underneath this ticket, and is worth knowing before it is picked up: the composition layer it depends on is now substantially built and in production — lib/steps.ts + pipeline*.ts (#94–#99), triggers→pipeline (#92), the delivery outbox and correlation (#239), run_pipeline as a trigger/connection action, and two catalog agents (site-builder, lead-outreach) whose entire behaviour IS their declarative pipelines with no bespoke Worker. So the question this ticket poses is no longer "can durable behaviour be composed" — it demonstrably can — but "what replaces the workflow field for JOB_APPLY and CODING_SESSION, whose loops are Cloudflare Workflows with resumability that a pipeline run does not have."

    That reframing is the useful next step and it is a design decision. #705 already took the runtime/workflow invariant out of here, as the last comment records.

    Staying open, staying P3: later, premise re-verified rather than assumed.

  11. serge-ivo commented on Sep 26, 2026

    @serge-ivo
    ContributorAuthor

    Closing: reframed, not retired. The decision is recorded here, what it asked for maps to shipped work, and the one real gap belongs in its own issue.

    The decision: "reframe, don't retire" (2026-08-07, re-verified 2026-08-18, 2026-08-23 and today)

    capabilities.workflow does not name a behaviour. It names which physical executor a run must reach, and that the run may park mid-flight and resume the same run. CodingSessionWorkflow, JobApplyWorkflow and BrowserTaskWorkflow each:

    • run a model-decided round loop over a live relay session;
    • pause with step.sleep and resume at the same step with the same in-memory state;
    • hold an exclusive lease on the owner's machine or browser.

    The declarative layer can reach the same machine through registry tools, but it can't express that loop. workflows/pipeline-run.ts still walks its steps linearly, with no back-edge and no step.sleep (checked today). A manifest for those three would either misrepresent them or be unusable. So the enum stays a closed set of executor-backed brains, which is the correct end state rather than a compromise. #142 (trust gating before a third-party creator may declare runtime/workflow) is also still open, and loosening the field first would widen exactly what it exists to narrow.

    What this issue asked for, mapped to what exists

    The ask What exists
    "A creator can only pick one [workflow], not define a new autonomous loop" Expired. An agent declaring workflow: null gets the platform's durable loop, AGENT_LOOP (60b59fc6, #158): budgeted, cancellable, escalating, with a run row, over its own declared tools. loop-drivers.ts routes only CODING_SESSION elsewhere. Most seeded agents declare null.
    Behaviour as declared composition (steps, triggers) Pipelines are data (#94–#99), started by triggers and connections (run_pipeline). site-builder and lead-outreach are entirely pipelines, and portal-watch is entirely a row.
    A decide-loop a creator configures: the "declarative front for AGENT_LOOP" this thread recommended instead Built piece by piece:
    • Objectives: loop presets, creator default under subscriber override (53f78060, #234; MCP 2a90ceb5).
    • Run length: limits on the agent (ca866695, #820).
    • Objective queue: follow-ups that auto-start when a run ends (fbfe8097, #788, migration 0149).
    • Tools and decide context: the loop acts through the instance's declared tools[] and identity.
    • The Coder itself declared as data over this vocabulary (49a1bacc).
    Keep the enum honest while it exists #375 (c9ea1ebe): the picker is served from the one table the platform enforces, and the dead INSURANCE_QUOTES is gone. #705 (8d43f06f): a workflow that needs hands is refused at all three declaring routes without the runtime that supplies it.
    Today, 1cbdb09a Regression tests pinning the closed set at every entry point, not only in the table:
    • the write sanitizer drops PIPELINE_RUN/AGENT_LOOP, and the picker never offers them;
    • a stored platform-driven or dead value resolves to no workflow on read;
    • every declarable workflow round-trips unchanged;
    • the legacy fallback (job-application-assistant → JOB_APPLY + browser, coder/category code → CODING_SESSION + coding, anything else → none) is asserted and checked against the #705 guard, which it otherwise never passes through.

    On a "central validator": none was added, because one already exists. workflowRuntimeDenial in lib/agent-workflows.ts is the single pure check, and the three routes call it on the capabilities block each one merges. The catalog claims lint (agent-claims-lint.ts) is deliberately separate: it warns about catalog copy and never refuses a write. Merging it into a write-time validator would turn those warnings into 400s on save.

    Test coverage now:

    • matched and mismatched pairs for every workflow, and null-workflow acceptance (agent-workflows.test.ts);
    • the 400 at all three routes (agents.integration.test.ts);
    • every migrated row against the guard (seed-workflow-runtime.test.ts);
    • the entry-point and fallback tests above.

    D1 migrations have no down step, so there's no rollback path to test. Full suite: 714 files, 12,825 tests passed.

    The remaining gap, and where it belongs

    Making the enum genuinely retirable needs one thing the platform doesn't have: a resumable wait in the pipeline runner, a step.sleep-based park and resume within the same run, with loop-carried state. That's a design change to workflows/pipeline-run.ts, not part of this issue. If the owner wants it, it should be its own issue, sequenced after #142.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3: laterDesign, strategy, or deferred pending a decision or demandbackendBackend / Worker / API workcoordinationAgent coordination / delegation primitivesenhancementNew feature or requestpipelinesPipelines, steps, triggers and the event pump — what CALLS a connector

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions