[hhkaos]

skills/ carries version literals that nothing checked: the npx -p @opentechevents/<pkg>@<minor> lines an organizer actually runs, a pinned embed URL, a specVersion in an example an agent will copy. They had drifted, because the rule asking for the pins to move lives in SKILL.md and the pins live in references/.

This replaces that rule with a test.

What was wrong

	Was	Ships
references/website.md	export-jsonld@0.4 ×2	0.6.0
references/website.md	embed/v0.7.0 ×2	0.8.2
references/writing.md	the canonical minimum writes specVersion: 0.4.0	0.6.0
references/website.md	"application/ote+json — what v0.4 documents"	spec is on 0.6
evals/test/assertions.test.ts	the pins it claims the skill prescribes, on @0.4	0.6.0

And the one that mattered most: fileIsValid measured ground truth with the release whose version bug this kit exists to have fixed. It fetched the validate package on the 0.4 line, which judges a document against the newest release instead of the version it declares — so a correct 0.5 or 0.6 document came out broken, and a green run meant nothing. The eval README warns about exactly this ("an uncalibrated instrument turns a real regression into a green run") one screen above the pin that did it.

The guard

skills/evals/test/version-pins.test.ts — a plain unit test, no agent, no API key, runs in CI with everything else. It lists nothing: every expectation is derived at run time from the workspace's own package.json versions and from packages/validate's support policy, and failures name the file, the line and the value to write.

• npm pins name a package this repo ships, on the line it ships (minor or exact both fine).
• a pinned embed/v<x.y.z> URL matches apps/embed and has a committed versions/v<x.y.z>/ snapshot behind it — the skill tells organizers to put that URL in a live <script> tag.
• a specVersion in anything skills/ote-event/ tells an organizer to write is the latest published version.
• no .md names a bare spec minor in prose: "what v0.4 documents" is read as current by whoever reads it.

No per-line ignore mechanism, deliberately — a marker that silences this scan would be reached for on the day the scan is right. A comment that needs to discuss an old pin spells it in words ("the 0.4 line"). The guard caught its own comments on the first run, and that is how they were fixed.

The failure nobody would have seen

An eval fixture pinned to a spec version changes meaning when the support window moves. corunajug-feed.json was stamped 0.3.0 to be the "older but supported — a notice, never a defect" case. The day 0.5.0 and 0.6.0 shipped together, the window slid past it. The fixture still parsed, the case still ran, and it had quietly become a migration case still asserting the opposite — today it exits 3.

Fixtures now declare a role (latest / older-supported / none) that the test resolves against the live policy, so a window move names the file to re-stamp instead of silently rewriting what a case tests. Re-stamped to 0.4.0 (✓ valid against OTE 0.4.0, with the notice), and its watch_for rewritten — it described the pre-0.6 CLI behaviour as "live today".

fileIsValid now imports validateDocument from the workspace rather than pinning a published release. An instrument must be the code under review, not a copy of it from npm, and an unpinned import cannot drift. The agent-facing pins stay literal on purpose: they say which release the instructions were tested with.

Verification

By simulation, not by reading. Bumping validate to 0.7.0 and embed to 0.9.0 and re-stamping the fixture:

skills/ote-event/SKILL.md:17 pins @opentechevents/validate@0.6, workspace ships 0.7.0
skills/ote-event/references/website.md:92 pins embed v0.8.2, workspace ships v0.9.0
corunajug-feed.json must not be the latest release

Inserting a bare v0.5 into prose trips the prose rule with its file and line. Both reverted. pnpm lint, pnpm build, pnpm typecheck, pnpm test green (31 tests in skill-evals).

CLAUDE.md gets the procedure: the command, the five rules, and the fixture-role rule that matters on a release day — re-stamp the fixture, never move the bar.

What this does not fix

Found while investigating, filed rather than fixed here:

• validate: "not judged" counts as 0/N valid and exits 3 — a healthy kit fork reports 0/3 valid #117 — ote-validate reports a healthy kit fork's events/*.json as 0/3 valid, exit 3.
• ote-event skill: Gate 2 in a kit fork is build-feed, and the validator should name a feed member when it meets one #119 — the skill's Gate 2 sends the agent at those files before ote-build-feed --check.
• ote-event skill: no version handshake — a copied skill cannot tell its pins went stale #122 — nothing keeps a copied skill's pins honest; this guard only reaches the repo.
• A feed member is a third document class with no $id — give it one, and make the inheritable set normative opentechevents-spec#55, Push tag embed-v0.6.0 and create its GitHub release #56 — the modelling gap behind validate: "not judged" counts as 0/N valid and exits 3 — a healthy kit fork reports 0/3 valid #117, and the support window being measured in releases rather than time.

🤖 Generated with Claude Code