Skip to content

[Epic] Coordination primitives — delegate goals, not one-shots (Coder as first consumer) #154

Closed

Description

@serge-ivo

Make agent coordination correct — as a platform primitive Coder is the first consumer of, NOT a refactor of Coder's hardcoded internals. Design: docs/coordination-primitives.md. Companion to agent-platform-strategy.md + connector-manifest.md (#143).

Problem (verified): the Overseer (drive_claude) and send_to_cli are fire-and-forget one-shots — no follow-through, no typed task/status, three overlapping drive-paths, and the console Loop is client-driven (dies when the tab closes) while the durable Pilot (CodingSessionWorkflow) overlaps it.

What stays (best practice, keep): Engine=executor / brain=decider; deterministic target routing (repo → one active session → live node); per-instance isolation; declarative consent-gated tools. This is not a rewrite — only the coordination policy on top changes.

Principle: delegate at the right abstraction, through a durable handoff. Commands → Engines; goals → autonomous workers (Pilot, later another agent) that own loop/retry/escalate; the coordinator delegates goals and tracks status.

We build ONE thing now (the fix as the primitive) and defer the generalizations until they have a proven primitive and/or a real second consumer — avoiding both "polish the hardcoded thing" and "build the framework too early."

Activity

  1. added
    backendBackend / Worker / API work
    coordinationAgent coordination / delegation primitives
    on Aug 3, 2026
  2. added a commit that references this issue on Aug 3, 2026
  3. serge-ivo commented on Aug 3, 2026

    @serge-ivo
    ContributorAuthor

    Near-term build #155 shipped (1a8c40d): Overseer → durable Pilot goal delegation + observable task. The epic's substrate is now proven with Coder as first consumer. Remaining are the deferred generalizations: #156 (first-class delegate verb), #157 (typed task model), #158 (Loop→Pilot convergence), #159 (cross-agent), #160 (retire workflow enum — now a convergence job per its updated note).

  4. serge-ivo commented on Aug 3, 2026

    @serge-ivo
    ContributorAuthor

    Status update 2026-08-04 — the deferral gate is met

    Near-term #155 (Overseer delegates a GOAL to the durable Pilot) is closed and shipped — delegateGoal + the board task record in routes/coding.ts / lib/delegation.ts.

    The sequencing note said: "THEN cross-agent + workflow-enum retirement once a second consumer exists." That consumer has arrived — configurable multi-layer supervision, where a user wires agents to oversee other agents at arbitrary depth, as configuration rather than code.

    New design: docs/supervision.md, which carries #154's reasoning one level up. The same argument this epic made about delegate applies to the hierarchy itself: Coder's Overseer → Pilot → Engine structure is correct, but it lives inside the agent. Three things hardcode it — the target type (drive_claude(repoId, …)), the coordinator being a route, and depth being fixed at 2 by construction. No table describes who supervises whom, so supervision cannot be configured, inspected, or reused.

    Un-deferred: #156 (generalize the delegate target — the smallest useful step, everything depends on it) and #159 (cross-agent, which should now fall out rather than be built directly).

    New children, covering hazards that only appear once hierarchy is configurable rather than written by a human:

    Sequence: #156 → #183 + #184 + #185 → #157 (result channel + escalation ladder) → #159 falls out → #160 (retire the workflow enum, the actual blocker on a configurable Coder).

    Coder stays first consumer throughout: at each step its Overseer should be re-expressible in the primitive, and when it is, the hardcoded route comes out.

  5. serge-ivo commented on Aug 7, 2026

    @serge-ivo
    ContributorAuthor

    Reassessment at HEAD (9007758) — the epic is served. Closing.

    Read in code, not from commit messages. Taking the four problems this epic opened with, one at a time, because they are the acceptance criteria it actually wrote down.

    1. "no follow-through — the Engine may get stuck / half-finish and the coordinator never knows" — served

    routes/coding-brains.ts delegateToTarget() (#155, 1a8c40d): takes the single-flight session claim before anything observable is written, mirrors a delegation board card, logs a delegate trace event, writes the handoff into the target session's own thread, and hands a CodingGoal to CODING_SESSION — the durable Pilot, which owns the loop and the escalation and closes the card it was given. It returns a task handle, not a reply string. The fire-and-forget one-shot is gone from the path the ticket named.

    2. "no typed handoff — a reply string, not a task with owner + status + result" — served, in a shape the ticket did not name

    The record is agent_loop_runs (migration 0062): instance_id + delegated_by (owner, migration 0090 / #318 792d7a0), status, stop_reason deliberately distinct from status, detail, iteration, budget_id, cooperative cancel_requested. Around it:

    The invariant that makes this a model rather than a pile of tables is stated in lib/loop-drivers.ts:14-16: every driver opens an agent_loop_runs row, so check_delegation, subordinate_status and the console's /loop/:runId poll need no idea which driver ran.

    3. "three overlapping ways to drive an Engine, with only implicit rules about which to use" — served, and better than picking one

    Three separate changes did it:

    • claimSessionDriver (migration 0069, [docs] Reconcile runtime docs with relay-only pags up #208) makes them mutually exclusive at the pane — the rule is now enforced rather than documented. Guarded at coding.ts:548 (/run), coding-brains.ts:132 (drive_claude) and loop-drivers.ts:159 (the coding driver).
    • lib/loop-drivers.ts ([docs] Add docs drift checks for generated docs, llms files, and API surface #210) turns "work on this autonomously" into one verb with a driver table, reachable identically from the owner's Loop button, start_work and delegate_goal. That if (workflow === "CODING_SESSION") had already been duplicated and had already drifted (one copy passed onBehalfOf, the other didn't) — the table is why.
    • overseerDisabled / copilotDisabled (coding-brains.ts, reading surface-options.ts) make which paths exist per agent DATA, which is how the second coordination layer came off a Repo Coder without forking the code.

    4. "autonomy that isn't durable — the Loop dies when the tab closes" — served for the Assistant tab; NOT for the Coding tab

    AgentLoopWorkflow + migration 0062 + the console migration 67eb915 fixed the Assistant Loop, and #158's own closing comment is precise about what it covered.

    agents/coder/web/src/use-coding-loop.ts at HEAD still self-schedules with setTimeout and drives the Engine through POST …/sessions/:id/message — the route documented as "manual drive, no brain." Filed as #374, not left here, because the interesting part is not durability: on that path merge authority (#314), the driver claim (#208) and the spend budget (#184) all miss. readMergePolicyForRun has exactly one call site (coding-session.ts:84) and screenInstruction exactly one enforcement site (coding-loop.ts:150), so coding-session.ts:79-83's stated assumption — "the Pilot is the only thing that drives the Engine autonomously" — is false at HEAD. That is a safety defect that stands on its own whatever happens to this epic.

    The five deferrals, plus the three added in August

    All closed, and each is real enforcement rather than a status change:

    what serves it
    #156 delegate primitive lib/delegate-target.ts — the target is an addressable thing ({kind,id}), not a repo id, with an explicit refusal for a parsed-but-unrunnable kind so a silent no-op can't look like success on the board
    #157 typed task/handoff agent_loop_runs + 0090 + engine-acts.ts + escalation.ts + run-sweeper.ts (see below on what it did not build)
    #158 Loop → durable AgentLoopWorkflow; lib/agent-loop.ts pure policy — terminal decisions honoured before the iteration cap, plus a no_progress guard that doesn't depend on the model noticing
    #159 cross-agent lib/delegate-instance.ts — and it did "fall out" as the sequencing note predicted: 117 lines, almost all of it enforcing #183/#184/#185 rather than implementing delegation
    #183 graph as data lib/supervision-graph.ts — pure, validated at wiring time when the human is present, MAX_DEPTH 4 / MAX_SUBORDINATES 12
    #184 budget inherited from the parent run, never a fresh pool per hop — the specific thing that stops allowance × fanout^depth
    #185 authority containment lib/execution-authority.ts — the executor is the only authority, the asker reaches onBehalfOf and nothing else, both directions tested

    #160 is the one child still open, and it has its own life — separate comment there.

    What I would NOT build

    #157's unified task record. Its closing comment says the typed-record half was not built and offers to reopen. It shouldn't be. The need it named is met — every driver opens the same run row, and that row already carries owner (instance_id + delegated_by), status, stop reason, result detail and progress. What remains is unifying it with pipeline_runs and the board cards, and run-sweeper.ts argues the counter-case in its own header: "more write paths into one table means more ways to strand a row, and a stranded row is permanent data." That trade is vocabulary tidiness bought with a class of permanent bad data, and the sweeper exists because the class is not hypothetical.

    Assumption I tested

    "The Coder's coordination is now durable end to end." It is not, and the failure is not where the epic would have looked. The delegated path is durable, claimed, budgeted and screened; the path a human actually presses — the Loop button in the Coding tab — is none of those. Two buttons, both labelled Loop, on the same agent, with opposite properties (table in #374). That the epic's own fourth bullet is the surviving one is worth recording: it survived because #158 fixed the Loop the epic described (the console poll in InstanceDetail.tsx) and there turned out to be a second one.

    Closing

    The near-term build shipped, all five named deferrals and all three August children are closed with enforcement behind them, the one criterion left is now a filed safety bug with a concrete fix, and the one open child (#160) is a strategy ticket that reads better standing alone than as a line item under a coordination epic. Closing.

  6. serge-ivo commented on Aug 7, 2026

    @serge-ivo
    ContributorAuthor

    Closing per the reassessment above: near-term #155 shipped, all five named deferrals plus #183/#184/#185 closed with enforcement behind them, the one surviving criterion is filed as #374, and #160 stands on its own.

  7. proagentstore commented on Oct 10, 2026

    @proagentstore

    New review scope: optional embedded brain on every agent instance (2026-10-10)

    The owner requests a thorough consolidated architecture/design review BEFORE any new brain feature implementation. Each agent should optionally understand its own progress, decide next steps and assist/recover its executor. This belongs inside each individual instance as configuration, not a separate supervisor agent. It must cover ALL store agent types, not only coding/CLI. The owner explicitly authorizes the Platform coder to investigate current implementation and document the epic; feature implementation awaits joint review.

    Historical epic and verified starting evidence

    This is follow-on review under #154; its original delivered scope remains closed. Do not infer that closed means this new desired experience exists, or reopen/close historical tickets without review.

    Required investigation and consolidated review document

    1. Inventory schema/storage, API, MCP, Console UX, prompts/instructions, runtime drivers, model resolution, capability routing, schedules/triggers/events, budgets/limits, consent/tool policy, recovery and durable state. Cite current commit + file/line and linked issues/comments. Separate delivered-and-verified, source-only/unverified, historical/superseded, and genuinely missing.
    2. Propose a coherent per-instance brain ON/OFF contract for every supported runtime (coder/CLI, cloud/chat, pipeline, browser, job-related and creator-defined agents). Brain OFF must retain deterministic authorized queue execution, persistence, status and explicit human controls without hidden LLM decision turns; describe unsupported combinations honestly.
    3. Separate decision-making model/provider from execution-engine model and conversational model. Inventory what is currently shared; present compatibility/inheritance/migration choices, credential selection and attribution, token/spend reservations, failure/fallback behavior. No silent provider/model changes.
    4. Design actions/policies and enforceable boundaries: observe, continue, retry/recover, wait, stop and escalate, with evidence-based progress state, bounded retries/no-progress checks, idempotency, safe interruption and single-driver ownership. Turning a brain on grants no tools, credentials, submission authority or widened consent.
    5. Distinguish alive/working/waiting/stalled from actual progress. Avoid repeated discovery instead of implementation, interrupting an active executor, identical retries after provider-credit failures, fabricated success and replay of uncertain side effects. Include restart/offline/event-deduplication scenarios.
    6. Define instructions/goal precedence, owner limits, audit trail and UI explanation of decisions/actions (concise evidence, not hidden reasoning), pause/stop semantics and what config changes do to in-flight work.
    7. Draft dependency-ordered child tickets ONLY for unmet gaps, each with existing-ticket deduplication, acceptance tests across agent types, rollout/migration risks and review questions. Do not file new issues, alter live instance settings or implement feature code before joint review.
    8. Publish a substantial review-ready documentation artifact linked here, plus a short decision guide with recommended options and unresolved choices. Preserve historical design documents as history, linking the new review where appropriate.

    Scope guard

    This task is documentation and investigation only. No new brain feature implementation, live configuration/model/autonomy changes, application submissions, credential/access changes, runner restarts or production deployment. Do not interrupt the active runner auto-update work. Platform coder should run this after that existing run finishes. Respect denied issue creation; do not bypass permission errors. The owner will review the consolidated design and proposed missing child tickets before implementation starts.

  8. proagentstore commented on Oct 10, 2026

    @proagentstore

    Embedded-brain review package ready for joint review

    The thorough investigation and consolidated design requested in the October 10 review brief is published:

    Read the full architecture review and draft child tickets

    Documentation-only commits: c82a630 and d6d7697. Both changed only docs/embedded-brain-review.md; remote content verified. Inspected source baseline is 60d6380.

    What is already there

    Instance model selection currently couples chat and orchestration; other decision paths vary. Durable runs, queues, driver claims, pause/stop, event wakes and bounded application checkpoint decisions are existing primitives to reuse. They are not a single universal brain switch or separate decision-model contract. The review distinguishes source/test evidence from actual live acceptance and preserves historical issue closure.

    Proposed design for review

    • An optional embedded brain on each instance, across cloud/chat, pipelines, coder/CLI, browser/local application/artifact and creator-defined runtimes. No separate supervisor agent.
    • Decision model/provider distinct from conversational model and execution engine, with explicit owner choice.
    • Enabling grants no extra tools, credentials, consent or submission authority.
    • Deterministic execution/persistence/status remains available. Existing decision-making loops are explicitly legacy_compatibility until reviewed migration; the new layer default-off does not falsely mean legacy decision calls vanished.
    • Typed progress evidence, bounded recovery, single-driver ownership, no interruption of active executors, idempotent wake/restart and no automatic replay of uncertain effects.
    • Separate decision-cost attribution/sublimits still debit and obey inherited shared/root/account budget controls.

    Owner decisions before implementation

    1. Default-off/opt-in and legacy migration semantics.
    2. Separate decision-model/provider selection and template inheritance.
    3. Allowed lifecycle actions and evidence/privacy boundaries.
    4. Adapter delivery order and live-acceptance prerequisites.
    5. Approval/refinement of five dependency-ordered draft child tickets: configuration/model separation; policy/evidence core; progress/recovery; runtime adapters; Console/MCP and conformance.

    The review contains acceptance criteria, existing-issue deduplication and rollout risks for each draft. No new child issues were filed, no brain features were implemented, and no live instance settings changed. This historical epic remains closed for its original delivered scope; the follow-on design awaits joint review.

    Validation: documentation drift, local linked-path checks and Git whitespace passed. Strict docs build was not run successfully because zensical is absent from the checkout; no dependency changes were made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend / Worker / API workcoordinationAgent coordination / delegation primitives

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions