Skip to content

[bug] capabilities.tools is not enforced on pipeline steps — a stored pipeline can call any registry tool the agent never declared #381

Closed

Description

@serge-ivo

The gap

capabilities.tools is the authoritative allowlist of what an agent may reach — PR #59 wired it,
sanitizeToolList validates it, and toolNamesFor is what turns it into a gate. Three surfaces
enforce it:

  • agent-think.ts:657 — the chat runtime builds allowedToolNames and hands the model nothing else
  • instance-tool-policy.ts:123 — /v1/instances/:id/tools marks an undeclared tool not_declared
  • supervision-capability.ts:35 — delegation checks the target's declared set

lib/pipeline.ts does not. runPipelineStep builds its registry context by hand at
pipeline.ts:389:

const registryCtx = { env: ctx.env, userId: ctx.userId, instanceId: ctx.instanceId, traceId: ctx.traceId };

and dispatches at :400 / :408. runRegistryTool (tool-registry.ts:383) enforces exactly two
things: the scope-vs-read-only-connector check (#86, :392) and the per-instance write-consent gate
(#90, :402). Neither consults toolNamesFor, and nothing else on that path does either. The file's
own header comment at pipeline.ts:3 says the dispatch goes through "the single path enforcing
connector auth/grant/consent" — which is accurate, and is the whole point: connector consent and
declared capability are different questions, and only the first one is asked here.

So a stored pipeline definition can call any tool in the registry, including tools the agent
never declared and which its own console would render as not_declared.

Why this is worth more than a shrug

Read-scoped tools are ungated outright. Write-scoped ones still need a standing per-instance
connector write-consent row, which is a real gate — but it is granted per connector, not per tool,
and it is the same row that legitimises the pipeline's intended write. Consent to
create_ticket-style work through a connector is not consent to every other write tool that
connector owns. tmux_run_command (connectors/tmux.ts:79, scope: "write") is arbitrary shell on
the owner's machine and sits behind the same one row as its read-only siblings.

Pipelines are the surface where this matters most, because a pipeline is data. agents.config.pipelines
is copied into every new instance on subscribe, and #141 lets capabilities be declared from config.
The declared-tools allowlist is the mechanism that is supposed to make that safe.

What I am not claiming

This may be deliberate — a pipeline is authored by the agent's creator, not by a model at runtime,
so one reading is that the pipeline definition is the declaration. If so it should be written down
at pipeline.ts:389 and in docs/connector-manifest.md, because every other surface reads the
opposite rule from the same field, and #58's third-party opening is exactly when an unwritten
exemption becomes load-bearing.

Suggested fix

Resolve the instance's capabilities once per run and pass the declared set into PipelineRunCtx, so
runPipelineStep refuses an undeclared tool with the same vocabulary instance-tool-policy.ts
already uses (not_declared). Failing closed on a pipeline that names an undeclared tool is also a
better authoring error than discovering it at run time.

Found while assessing #322 (standing policies), where the relevant question was what a
timer-initiated pipeline can reach without a human in the loop. Verified against main at db8a2af.

Activity

  1. added
    bugSomething isn't working
    backendBackend / Worker / API work
    on Aug 7, 2026
  2. serge-ivo commented on Aug 8, 2026

    @serge-ivo
    ContributorAuthor

    Picked this up for implementation and found it already delivered — shipped in b11a740, which is on main and deployed (API Worker green through fc01c17). The commit subject leads with #381 and closes with #382, and #382 was closed on its own earlier today, which is why this one stayed open.

    Verified against the code on main, not against the commit message:

    • PipelineRunCtx.declaredTools exists (lib/pipeline.ts:461) and is threaded into the hand-built registry context at pipeline.ts:490 — the exact line the issue named (:389 at the time) as the place the allowlist never reached.
    • The rule is one pure function, undeclaredToolRefusal in the new leaf lib/tool-refusal.ts, refusing iff the caller resolved an allowlist, the tool carries a connector, and the allowlist does not name it. explainRefusal moved there too with instance-tool-policy.ts re-exporting it, so the not_declared vocabulary the issue asked for is literally the same code path, not a parallel one.
    • Three statements of the one rule, as the issue's "failing closed is a better authoring error" argument asked for: 400 on attach, refusal at kick before the run row opens (pipeline-run-start.ts), refusal at dispatch (tool-registry.ts). The dispatch arm is the security boundary and covers the nested case the issue did not raise but which matters — {"tool":"enrich","inputs":{"tool":"tmux_run_command"}} would walk past a check that only read steps[].tool.
    • The exemption is written down, which is what the "What I am not claiming" section asked for either way the decision went. It is derived from the tool's own connector field rather than listed, so the step library (map, filter, slice, dedupe_upsert, …) and create_ticket are exempt because no creator can declare them; two tests pin that derivation against the live catalog, so a step tool that later joins TOOL_CATALOG comes under the gate by itself. platform-docs/connectors.md carries the note.
    • The seeded agent was fixed with it — migration 0096 declares http_request / web_search / mcp_call_tool on the Small Business Website Builder, which 0057 had seeded with no tools at all and which would otherwise have been refused on all four of its dispatches. seed-drift.test.ts derives the gated set from the reference JSON through the real registry and fails if they disagree.

    Tests green on current main: pipeline-tool-policy.test.ts (27), pipeline-run-start.test.ts (17), seed-drift.test.ts (7), step-dispatch.test.ts (5), plus pipeline.test.ts — 79 passing across the group.

    Two things the fix deliberately does not do, both stated in the commit and worth carrying forward rather than rediscovering: the gate fails open when the capability lookup returns null (a failed D1 read is not evidence the agent declared nothing — the same asymmetry instanceToolPolicy and delegationDenial already document), and it does not consult the subscriber's config.disabledTools, which answers a different question.

    One consequence that is live and not fixed by code: the small-business-website-lead-finder agent declares no capabilities.tools, so its lead_finder pipeline is refused at kick today. That is live catalog data, it is recorded in the #394 audit with a copy-pasteable declaration and a narrowing warning, and it is the owner's call.

    Closing as delivered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend / Worker / API workbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions