Skip to content

[bug] A typo'd or forward $ref resolves to undefined — pipelines run to completion on missing data #244

Closed

Description

@serge-ivo

Root cause of the class #243 is one instance of.

What happens

resolveInputValue (lib/pipeline.ts:206) resolves references by lookup, with no existence check:

if (typeof obj.$param === "string") { … return scope.params[obj.$param]; }   // :221
if (typeof obj.$ref === "string")   return readPath(scope.outputs, obj.$ref); // :223

and readPath returns undefined the moment a segment is missing (:191-199).

So three distinct authoring mistakes are all indistinguishable from "this value is legitimately
absent":

  1. A typo'd bind — {"$ref": "geo.lat"} when the step bound geocode.
  2. A forward reference — $ref to a bind produced by a LATER step. Steps run in order, so the
    output does not exist yet.
  3. An undeclared param — {"$param": "citty"}, or a param the caller never supplied.

Each yields undefined, which is then handed to the step handler.

Why that is worse than it sounds

Handlers treat absent as default, not as error — asArray(undefined) → [],
Number(undefined) || 0 → 0, and so on. Combined, a single-character typo produces a pipeline
where every step succeeds, the run completes, and the output is empty or wrong. There is no
failed step, no error-log row, nothing to notice. See #243 for a concrete instance where this
silently drops every record.

Why this is cheap to fix

validatePipeline (:158) already collects exactly what is needed and uses it only for
duplicate detection:

const binds = new Set<string>();
…
if (binds.has(s.bind as string)) return `Step ${i}: duplicate bind "${s.bind}"`;
binds.add(s.bind as string);

Walk each step's inputs in the same loop and check the ROOT segment of every $ref against the
binds seen so far — that catches typos and forward references in one test, because "so far" is
precisely the set of outputs that will exist when the step runs.

Two details to get right:

  • Implicit binds. stepBind (:187) defaults to step${index}, so $ref: "step3.foo" is
    legal without an explicit bind. The known-name set must include both.
  • $param. Pipeline defs declare their params (params?: Record<string, {description?}>, the
    same map agent-think renders for the model). A $param naming something undeclared is an
    authoring error and can be rejected the same way. Be careful with item / item.* inside a
    forEach body — those are scope-provided, not params.

Only the first path segment can be checked statically (a $ref into a step's output shape is
not knowable ahead of time), but the first segment is where typos and ordering mistakes live.

Suggested shape

  • Extend validatePipeline as above. It already runs on every stored def
    (:319 if (!def || validatePipeline(def) !== null) return null;), so bad defs are rejected at
    save/seed time rather than discovered as an empty run months later.
  • seed-drift.test.ts already compares seeded defs to the reference JSON; this validation would
    also protect the first-party pipelines (lead-finder, site-builder, site-deploy) from a rename
    quietly breaking a $ref.

Verification

  • A $ref to an unknown bind is rejected at validate time, naming the step and the bad reference.
  • A $ref to a bind declared by a LATER step is rejected.
  • $ref: "step2.x" is accepted when step 2 has no explicit bind.
  • $param: "item.lat" inside a forEach body is still accepted.
  • Every first-party seeded pipeline still validates — if one doesn't, that is a real bug this
    found, not a test to relax.

Activity

  1. serge-ivo commented on Aug 6, 2026

    @serge-ivo
    ContributorAuthor

    Shipped in fa7b00e.

    Verified: true. resolveInputValue (pipeline.ts:206) and readPath (:191-199) are exactly as described, and validatePipeline was collecting binds only for duplicate detection.

    What shipped

    • collectReferences(value) — pure, exported: every $ref/$param in an input tree. It mirrors resolveInputValue's dispatch exactly, including $param winning over $ref on an object carrying both, so validation can never police a reference the runner doesn't resolve. A test pins the two together.
    • stepReferenceError(step, index, knownBinds, declaredParams) — pure, exported. validatePipeline calls it per step with the binds seen SO FAR, before adding the step's own bind, so a step reading its own output is caught too. Implicit step{i} binds are in the known set. forEach is checked alongside inputs.
    • item / item.* is excused only when the step actually has a forEach — outside one it's an undeclared param, and there's a test for that.

    One deliberate narrowing. The $param check runs only when the def declares params at all. params is optional and predates this check, so a def declaring none isn't asserting a complete set — enforcing there would invalidate stored user defs rather than find bugs. A def that does declare them gets the full typo protection, which covers every first-party pipeline (all four params of site-builder, eleven of site-deploy, three of lead-finder are declared and used).

    No first-party pipeline broke — lead-finder, site-builder and site-deploy all still return null. So no latent rename bug was hiding there.

    Left open: a deeper-path typo (geo.latitude for geo.lat) is still undetectable statically. I considered a runtime warning on any reference resolving to undefined and rejected it: site-builder legitimately $refs email/instagram/facebook which are usually absent, so it would fire on healthy runs and become noise. The root segment is where typos and ordering mistakes live, and that's now closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions