Skip to content

#126 docs: curate the agent context — AGENTS.md under budget, DEPLOYMENT.md split by topic - #128

Merged
gregoryfoster merged 9 commits into
mainfrom
126-curate-context
Nov 8, 2026
Merged

gregoryfoster merged 9 commits into
mainfrom
126-curate-context

Conversation

@gregoryfoster

@gregoryfoster gregoryfoster commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Refs #126. This is the first of two PRs: § Patching and § Node.js stay in docs/DEPLOYMENT.md byte for byte until #113's first window (Tue 2026-11-10, 05:00–10:00Z) closes.

Docs only. No path under processor.drift.RUNTIME_* changes, so no deploy.

Before and after

Counts are measure-context.sh --exact, from count_tokens.

File Before After Budget
AGENTS.md 6,663 5,929 6,000
docs/DEPLOYMENT.md 44,066 17,639 (until PR 2) 10,000
docs/PROVISIONING.md (new) — 8,303 10,000
docs/DEPENDENCIES.md (new) — 8,083 10,000
docs/MONITORING.md (new) — 7,633 10,000
docs/NETWORK.md (new) — 5,801 10,000
  • There are no dead links, dead anchors, orphans or unindexed docs.
  • AGENTS.md's largest section was Infrastructure at 41%. It is now Hard Rules at 17%.
  • Every X.md#anchor in the repo resolves, including those in docs/plans/ and docs/specs/.

The DEPLOYMENT.md split

Each ## section moved verbatim to the doc below. The only edit in transit is links re-aimed at the moved section's new doc.

Section Now in
Prerequisites (operator), /etc/processor/, Containment, Install, Before #2, Create the group PROVISIONING.md
CI pins and Dependabot, Shell lint, The lock audit, Bumping co-core, Extraction dependencies, TLS trust DEPENDENCIES.md
The drift check, Liveness, Disk GC, Memory: no earlyoom MONITORING.md
Tailscale DNS (with its ###s), Tailnet flaps NETWORK.md
Releases, Deploy a change (with The CI gate), Rollback, Operate, Patching, Node.js stay in DEPLOYMENT.md

Stubs. A ## Moved sections block at the end of DEPLOYMENT.md keeps each moved heading as a one-line stub, so every old anchor still resolves. They are needed for two reasons:

  • Runtime files cite DEPLOYMENT.md sections (src/, deploy/, scripts/deploy.sh, pyproject.toml, .skills/patching-hosts), and editing any of them is a deploy.
  • The frozen § Patching links into moved sections.

Pointers repointed (none of these files is a runtime path):

  • scripts/{audit_lock.sh,co-pypi.sh,co_pypi_token.py,shellcheck.sh}
  • .github/{dependabot.yml,workflows/audit.yml}, .githooks/pre-commit
  • .skills/{doc-sections,doc-sensitive-paths}
  • docs/SKILLS.md, docs/SOCRATICODE.md, .socraticodecontextartifacts.json (the four new docs are added)
  • Tests that read a moved section now read its new doc. Two never-checks (/etc/processor/* globs, bus URLs) scan all five runbook docs.

§ references. A bare § X whose section now lives elsewhere names its doc, e.g. MONITORING.md § The drift check. The exception is inside § Patching, which isn't touched.

AGENTS.md

Section Class Disposition
Infrastructure (2,781) B + C Rows become pointers. Each detailed row moved verbatim to a ## At a glance table: CI and Lock audit to DEPENDENCIES.md; Drift check, Liveness and Disk GC to MONITORING.md; Broker to NETWORK.md; Service, Patching and Node.js to DEPLOYMENT.md, ahead of and outside § Patching. The new rows keep the never-rules: never a bare apt-get upgrade, never Docker Hub anonymously, never from a checkout, and no sudo, docker or home for the service user.
(Infrastructure's 5 rule paragraphs) A Kept inline, under a new ## Hard Rules heading.
Parity goldens paragraph B Moved verbatim to DEPENDENCIES.md § Parity goldens. Inline it keeps its three rules: goldens come from co-core directly; regenerate only at a bump, never to pass a test; never edit tests/fixtures/parity/real/.
Environment & Tooling, Environment Files A Kept. Links retargeted to DEPENDENCIES.md and PROVISIONING.md.
Detail Docs A One line per doc, all six indexed.
All other sections A Untouched.

Deletions: none. Phase 2 found 52 TRUE, 25 UNVERIFIABLE and 0 FALSE claims, so no deletion has a warrant.

Proof

  • prove-no-loss.sh --base origin/main: AGENTS.md lost 0 (11 retarget warrants), --claims dropped 0 (9 retarget warrants); --file docs/DEPLOYMENT.md lost 0 (27 retarget warrants). All are in .skills/context-loss-ok and .skills/context-claims-ok.
  • check-seams.sh: 0 open. 89 acknowledged in .skills/context-seams-ok, one line-exact entry per judged hit.
  • check-counts.sh: 0 open. 3 acknowledged; all predate this change.
  • § Patching and § Node.js are byte-identical to origin/main.
  • Full suite: 1032 passed, 15 skipped. ruff and shellcheck are clean.
  • A fresh-eyes review found no lost facts, no runtime-path edits and no weakened tests. This PR addresses its findings in 4cd3296.
  • Telemetry: a baseline row and an after-row in .skills/context-metrics.jsonl. The after-row's note records the interim DEPLOYMENT.md overrun.

Left for #126 (PR 2, after the window closes)

  1. Move § Patching and § Node.js to a docs/PATCHING.md (about 8.2k tokens), behind stubs. That brings DEPLOYMENT.md to about 9.5k.
  2. Repoint the runtime citations of moved sections in that PR's deploy. Two comments are now slightly wrong:
    • pyproject.toml:51 says the § Shell lint stub names the shellcheck version.
    • deploy/tailscaled.service.d/…:11 says the install is in DEPLOYMENT.md; it is in NETWORK.md.
  3. Decide on Phase 8: whether to install curating-context's write guard and weekly cadence.

Phase 8: guard and cadence (operator, 2026-11-08)

  • Write guard: .claude/hooks/context-budget-guard.sh (symlink into the vendored skill) plus a PostToolUse entry in .claude/settings.json matching Edit|Write|MultiEdit. It never blocks. Budgets: .skills/context-budget 6000, .skills/context-doc-budget 10000.
  • Weekly measurement: .github/workflows/context-cadence.yml, generated by install-cadence.sh, runs Tue 16:14 UTC and on dispatch. It records a baseline:scheduled row. Under merge_mode pr it keeps one standing PR open from context/weekly-measurement.
    • actions/checkout is SHA-pinned to ci.yml's v7.0.1, because test_dependabot.py reads every workflow. A re-render keeps the pin.
    • runs-on: ubuntu-latest stays as the template sets it, because a re-render would write it back. The job builds nothing. test_ci.py's image check covers ci.yml and audit.yml only.
  • .gitattributes: the ledger merges by union, the ratio file by ours, the counts file by context-counts. The two drivers are set in the main checkout's git config, which worktrees share.
  • Docs: a new docs/SKILLS.md § Context budget. § Not installed no longer lists these. DEPENDENCIES.md gains one sentence on the third workflow. The Pins line it moved stays verbatim, and no-loss reports 0 lost.
  • Telemetry: the after-row is amended (wire:write-guard, wire:cadence, 90 seams acked) and repo_commit is backfilled.

Operator, before the first run. My token can't read or set either of these:

  1. gh secret set ANTHROPIC_API_KEY -R CannObserv/processor. Without it the job fails at its first step and records nothing.
  2. Settings → Actions → General → Workflow permissions → Allow GitHub Actions to create and approve pull requests.
  3. After merge: gh workflow run context-cadence.yml once by hand. A PR opened with GITHUB_TOKEN starts no CI, so close and reopen it to run the checks.

🤖 Generated with Claude Code

gregoryfoster and others added 9 commits November 8, 2026 03:54
DEPLOYMENT.md was 44,066 tokens against a 10,000 per-doc budget. Its `##`
sections move verbatim (anchor links re-aimed, five warranted in
.skills/context-loss-ok) to:

- docs/PROVISIONING.md: Prerequisites (operator), `/etc/processor/`,
  Containment, Install, Before #2, Create the group
- docs/DEPENDENCIES.md: CI pins and Dependabot, Shell lint, The lock audit,
  Bumping co-core, Extraction dependencies, TLS trust
- docs/MONITORING.md: The drift check, Liveness, Disk GC, Memory
- docs/NETWORK.md: Tailscale DNS, Tailnet flaps

§ Patching and § Node.js stay, byte for byte, until #113's first window
(2026-11-10) closes. A "Moved sections" block keeps each old heading as a
one-line stub: src/, deploy/, scripts/deploy.sh, pyproject.toml and
.skills/patching-hosts cite DEPLOYMENT.md sections and can't change without
a deploy, and § Patching links into moved sections.

Non-runtime pointers (scripts, .github, .githooks, tests, .skills knobs,
SKILLS.md, SOCRATICODE.md, the SocratiCode manifest) now name the new docs.
Baseline telemetry row recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Infrastructure (2,781 tokens, 41%): rows become pointers into the new
  docs. The detailed rows move verbatim to each doc's `## At a glance`
  (CI and Lock audit → DEPENDENCIES.md; Drift check, Liveness, Disk GC →
  MONITORING.md; Broker → NETWORK.md; Service, Patching, Node.js →
  DEPLOYMENT.md, ahead of and outside § Patching).
- The five rule paragraphs under it get their own `## Hard Rules`.
- Parity goldens: the paragraph moves verbatim to DEPENDENCIES.md
  § Parity goldens; AGENTS.md keeps its three rules and a link.
- Pointers into moved DEPLOYMENT.md sections retargeted; Detail Docs lists
  each doc on its own line.
- Two `§` references to moved subsections (GCP provisioning, tailscaled's
  OOM rank) now link the doc they moved to.

Warrants: .skills/context-loss-ok (retarget), context-claims-ok (retarget),
context-seams-ok (each hit judged), context-counts-ok (3, all pre-existing).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Bare `§ X` references in moved and kept sections now name the doc X
  lives in (`MONITORING.md § The drift check`); § Patching untouched.
  20 warranted as retargets.
- AGENTS.md keeps two never-rules on every task: CI's Redis comes from
  `mirror.gcr.io`, never anonymously from Docker Hub (#43); the service
  user has no sudo, no `docker`, no home. Shell lint links its own section.
- tests: the `/etc/processor/*` glob and bus-URL checks scan the whole
  split runbook again, not one doc.
- .skills/context-seams-ok: one line-exact entry per judged hit, so a new
  seam can't hide behind a path-wide pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ment

Phase 8, adopted by the operator 2026-11-08.

- Write guard: PostToolUse hook on Edit|Write|MultiEdit, never blocks;
  budgets in .skills/context-budget (6000) and context-doc-budget (10000).
- context-cadence.yml: weekly baseline:scheduled row, Tue 16:14 UTC; under
  merge_mode pr it keeps one standing PR from context/weekly-measurement.
  actions/checkout SHA-pinned by hand (kept on re-render); runs-on stays the
  template's ubuntu-latest, which a re-render writes back.
- .gitattributes: ledger union, ratio ours, counts context-counts. Drivers
  set in the main checkout's git config.
- docs/SKILLS.md § Context budget; DEPENDENCIES.md notes the third workflow.

Needs, before the first run: the ANTHROPIC_API_KEY repository secret, and
"Allow GitHub Actions to create and approve pull requests".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant