Skip to content

The vocabulary inventory's own citations are unchecked — three name files or symbols that do not exist #600

Closed

Description

@serge-ivo

Three citations in the vocabulary inventory point at code that does not exist

state-vocabulary.ts (#593, 927f963) inventories MCP tool descriptions that publish a value set, each entry carrying a reason naming where the vocabulary is defined. Nothing validates those citations, and three are false:

entry cites reality
workers/api/src/lib/tool-listing.ts the file does not exist
lib/connector-consent.ts for granted / n/a / per_call / required zero occurrences of any of the four
lib/coding-engines.ts declares a different four-member set (EngineAuth)

An entry naming a nonexistent file passes the guard exactly as an accurate one does.

Why this matters more than three stale strings

The inventory's purpose is to record why a claim is unbacked, so a future reader can decide whether it can now be backed. A false citation sends that reader to a file that isn't there and, finding nothing, they will reasonably conclude the claim is unbackable — the opposite of what the inventory exists to enable.

It is also the same defect class the guard itself was built to catch: a stated fact that nothing checks. #585 was prose recommending a field the code had replaced; #594 was a legend naming a payload key that was never sent. This is a guard's own metadata with the same problem, which makes it the most ironic instance and arguably the most important — a verification artefact that is itself unverified erodes trust in the verification.

Acceptance criteria

  1. Every reason citation resolves: the file exists and contains the vocabulary named. Fail the guard when it does not.
  2. The three false entries are corrected to their real location, or reclassified if none exists.
  3. Denominator stated per ADR 0002: how many citations were checked, out of how many entries.
  4. Red demonstrated by pointing an entry at a nonexistent path.
  5. Consider whether the same check belongs on other guards that carry explanatory metadata — PINS entries in check-file-size.mjs and the UNBACKED_CLAIMS reasons are the same shape. Do not widen without measuring; say what you found.

Verified vs inferred

  • Verified: all three citations checked against the tree by the agent that reported them; that nothing enforces the reason string.
  • Inferred: that the remaining six citations are accurate. They were not individually checked — three of nine were found false, so the base rate is poor and AC1's guard is what settles it rather than a spot check.

Activity

  1. serge-ivo commented on Aug 15, 2026

    @serge-ivo
    ContributorAuthor

    Delivered in e36e713, CI and the MCP deploy both green. Done as ONE design with #603 — see AC5 below.

    AC1/AC2 — citations now resolve, and all nine were re-derived

    The citation is a field (source + symbol), not prose, and state-vocabulary.test.ts resolves every one against the tree: the file exists, it declares the symbol, and every member of the claim appears in a window around that declaration. Windowing is what lets it say no — ok, null and required appear in most files, so a whole-file substring search would agree with everything and measure nothing.

    All nine were re-derived, not just the three known bad: three false in nine is too poor a base rate to spot-check the rest. That turned up two further corrections nobody had flagged:

    entry was is
    disabled_by_owner|not_declared|ok lib/tool-listing.ts (no such file) lib/tool-refusal.ts:20 ToolPolicyReason
    granted|n/a|per_call|required lib/connector-consent.ts (0 of 4 members) lib/instance-tool-policy.ts:56 ToolWriteConsent
    account|default|env|platform lib/coding-engines.ts, "engine auth origin" routes/budget.ts:70 CeilingTier — and the noun was wrong too; it is a budget ceiling tier, and no concept of an engine auth origin exists in the tree
    added|errors|seen|skipped "an ingest tally" a pipeline run tally, lib/pipeline-runs.ts:21 RunCounts

    AC3 — denominator

    ✓ 9 of 9 UNBACKED_CLAIMS citations resolved against the tree (source + symbol + members)

    AC4 — red demonstrated, on all three shapes that really occurred

    Not one invented case: a citation pointing at a nonexistent path; one naming a symbol whose members are absent; and a different set of the same size (EngineAuth). Each was pointed at the real tree through the main arm and watched fail.

    AC5 — measured, and NOT widened

    check-file-size.mjs does not share this defect. Its load-bearing citation is the PINS key — a path — and scripts/check-file-size.mjs:1270-1273 already fails when a pinned path is neither measured nor present. Nothing widened there.

    The shared half with #603 — and a real catch

    The scanner's blindness was written down as a caveat in the success line ("NOT counted: a claim written with parenthesised glosses is invisible"), which is exactly the thing ADR 0002 asks a guard to measure instead of narrate. The design splits detection of presence from extraction of members:

    • enumAnnouncements is broad — it asks only whether a description publishes a value set, never what the members are, so it may match shapes the extractor must not.
    • stateEnumClaims is untouched, because your measurement stands: 6 false positives in 8 candidates, recovering two of four members.

    So announced = readable + UNREADABLE, and an unreadable announcement fails with the remedy named: render the vocabulary, never widen the scanner.

    Measured over all 136 descriptions: 3 announce a value set. \bvalues are\b was a candidate marker and was dropped — 2 false positives ("values are never exposed", "Values are clamped server-side") and no true ones.

    Its first catch was real and previously unknown: coding_session_capture has been rendered from a constant since #593 and was invisible to the sweep the whole time — checked by generation and uncounted by detection at once. runStateSentence now leads with a bare chain the way runHealthSentence was made to in #588. That moved the surface from 14 claims to 15, and backed from 4 to 5 — one claim from unchecked to checked. description is outside the surface-lock fingerprint by explicit decision, so no version bump; check-surface-lock and all 237 worker tests confirm it.

    What I did NOT do, and one defect this surfaced

    The citation check proves a citation resolves, not that it is the right enum. cancelled|done|failed cites LoopStopReason, which contains all three — but the only description publishing that set (instance-tools/board.ts:273) is about runtime task status, whose vocabulary is TaskStatus in packages/browser-runner/src/types.ts:10, where the member is completed and done does not appear at all. So a citation that resolves sits above a description that is wrong.

    That is a live defect in a file this change does not own. It is recorded on the entry rather than silently fixed, and it needs its own issue — flagging rather than filing, per the process here.

    Closing: AC1–AC5 all met and verified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions