Skip to content

check_docs.py reads the Step 4 substitution table through a fixed 8000-byte window with only 231 bytes of headroom #80

Closed

Description

@dmccoystephenson

What went wrong

scripts/check_docs.py:63 scans for substitution-table rows inside a hardcoded byte window:

table_section = text[table_start:table_start + 8000]

Measured against create-dev-loop.md at c1ac200:

Step 4 section start: 42479
Step 5 section start: 50248
Step 4 section length: 7769 bytes
Window:               8000 bytes
Headroom:              231 bytes

The window currently overshoots the end of Step 4 by 231 bytes (harmlessly — it lands in Step 5's prose,
which has no table rows). But the GENERATED_AT row — the last real row — sits at offset 6558, leaving
roughly 1.2 KB before added rows start falling outside the window.

Why this matters

When a row falls outside the window, declared silently loses it and check_placeholders reports the
placeholder as missing a substitution-table row even though the row is right there. CI fails with a
message that points the contributor at the wrong problem — "add a table row" for a row that already exists —
which is a worse failure mode than not checking at all.

This is precisely the invariant CLAUDE.md calls load-bearing ("Every {{placeholder}} in the template must
have a corresponding row in the Step 4 substitution table"), so the check silently mis-firing as the table
grows is a real risk. Adding 3–4 more placeholders would get there.

Expected vs. actual

Expected: the Step 4 table is parsed by its actual boundaries.
Actual: it is parsed by a byte count that happens to fit today.

Suggested fix

Bound the section by the next ### heading rather than a byte count:

table_start = text.find("### 4 — Fill in the placeholders")
...
next_heading = text.find("\n### ", table_start + 1)
table_section = text[table_start:next_heading if next_heading != -1 else len(text)]

That also removes the current 231-byte bleed into Step 5, which is how the unused-looking
`path/to/file` example row (line 603, inside Step 4's DOC_CHECK_TABLE guidance) ends up counted as a
declared placeholder.

Worth a regression check that fails before the fix: append a dummy row past the 8000-byte mark and confirm
check_docs.py reports it missing, then confirm it passes with the fix applied.

Notes

Found during a Stage A documentation-accuracy sweep on 2026-07-29. Not fixed in that sweep's PR because it is
a code change, not doc drift.

Activity

  1. added 2 commits that reference this issue on Jul 29, 2026
    e933cdb
    a2820e1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions