Repository navigation
[docs] Add docs drift checks for generated docs, llms files, and API surface #210
Description
Problem
Docs drift is currently easy to introduce: generated docs, llms.txt, llms-full.txt, OpenAPI, package READMEs, and source docs can contradict implementation without a check failing.
Recent audit examples:
- Runtime docs disagree on relay-only vs tunnel fallback.
- Job Application Assistant README describes old
job.apply_basictask flow. - OpenAPI misses many live Hono routes.
- Coder docs say runner-node UI is future work even though it exists.
- README catalog count does not match checked-in manifests/registry.
Scope
Add lightweight validation so future changes catch the highest-risk drift:
- Route inventory check comparing implemented Hono routes to OpenAPI coverage, with an allowlist for intentionally undocumented/admin routes.
- Text assertions that current runtime docs do not mention removed current-path commands such as
pags up --tunnelor FAGS. - Check that
platform-docs/*.mdand generatedplatform/store/docs/**are in sync afterpnpm docs:build. - Optional catalog consistency check across README,
agents/*/agent.json, migration seeds, andstore/registry.jsononce the source of truth is decided.
Acceptance Criteria
- CI or
pnpm test/pnpm docs:buildcan detect the main docs drift classes above. - The check has a small intentional allowlist, not a brittle giant snapshot.
- CONTRIBUTING or README documents how to update docs artifacts when route/docs sources change.
Activity
feat(loop): one Loop, in the one chat — the driver comes from what th…
fix(coding): the single-flight claim covered 1 of 3 paths that start …
fix(loop): say when a Loop starts, where it works, and render its res…
fix(coder): "Work on this" actually starts work, and a coding agent g…
feat(agent): start_work — the chat can finally hand a goal to its own…
fix(coder): issues list was unreadable on a phone (#227)
feat(loop): loop presets are per-instance config, and appear wherever…
test,docs: name the real cause of the "flaky" suite, and check the cl…
serge-ivo commentedon Aug 6, 2026
Shipped in 1640f25 — closing, left open by the delivering agent.
scripts/docs-drift.mjs(pnpm docs:drift), CI-wired, delegating the API half to the existingopenapi-coverage.mjsrather than duplicating it. Checks nav ↔ sources, removed commands in runnable code blocks only (prose may discuss--tunnel; a code block may not), MCP tool table in both directions, tool counts, docs links, API surface.Two real drifts fixed rather than just detected:
/healthreportedtools: 41against 124 registered — andoauth-provider.test.tsasserted the 41, so the test had made the lie load-bearing. Now derived from a real registration run.store/docswas committed and stale (the checked-in MCP page was missing its entire Tool Surface section) and is not byte-reproducible, so no diff-based check could ever have gone green. Now untracked + gitignored; bothci.ymlanddeploy-host.ymlrebuild it first. The "never stage store/docs" hazard is now structurally impossible rather than a convention.It earned itself during its own rebase: it immediately caught
GET/PUT /v1/instances/{id}/loop-presetsfrom #234, landed hours earlier with no spec entry.docs(api): spec the outbound-MCP consent routes (#262)
docs(mcp): document GET pipelines/:name + bump MCP tool count to 131 …
fix(budget): every autonomous coding entry point opens a pool, and MC…
feat(agents): an agent can stop the work it started, and end the sess…