Skip to content

fix(docs): read counted claims spelled out above twenty (#367) - #381

Merged
Gerrrt merged 2 commits into
mainfrom
gerrrt/check-docs-number-words-367
Sep 7, 2026
Merged

Gerrrt merged 2 commits into
mainfrom
gerrrt/check-docs-number-words-367

Conversation

@Gerrrt

@Gerrrt Gerrrt commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Closes #367.

check_docs.py matched counted claims as digits or as words, and NUMBER_WORDS stopped at twenty. The comment explained the cap as convenience — "Twenty is comfortably above any count here … it is not a claim about the ceiling" — which was true when written and quietly stopped being true.

On 2026-09-07 observability.md read "Coverage is forty-five rules of 67" and two rules were added that day. The digit was caught both times. The word was not, and I corrected it by hand twice. The check exists to prevent exactly that, and the defect was living inside it.

Two changes, and the second matters more

Words are generated through ninety-nine rather than typed — ones, teens, tens and hyphenated compounds. So forty-five and fifty-one now parse:

docs/observability.md:431 claims forty-nine unit-tested rules; the repository has 50

Previously that line was silently skipped.

The ceiling is asserted rather than assumed. Each claim pattern is re-run with the number token swapped for the words this table cannot read — hundred, thousand, dozen and friends — and a match is reported:

docs/observability.md:431 states unit-tested rules as a number this check
cannot read ('Coverage is a hundred rules'). Counts above ninety-nine must be
written in digits, or NUMBER_WORDS needs extending — an unreadable count is an
unchecked one

Extending the table alone would only have moved the silence. This is what makes the ceiling a rule rather than a hope.

The unparseable pattern carries an optional leading word, because these always appear with a determiner or multiplier in prose — "a hundred", "two hundred" — and without it the article sits where the claim pattern's whitespace cannot span it. That was the first version's bug, found by testing the assertion rather than assuming it fired.

A correction, in the second commit

The first commit claimed check_docs "still runs in well under a second". I had not measured it. Measured:

main (20 words) 1.57s
flat 99-word table, unconditional scan 7.53s
+ ceiling scan gated per file 5.54s
+ structured alternation (36 branches) 2.23s

A 4.8× regression on something that runs in make validate and CI. Fixed by expressing the compounds as structure — tens, optionally hyphen ones — rather than flattening them, and by gating the ceiling scan on one search per file with the swapped patterns lru_cached.

2.23s for five times the vocabulary plus an assertion main does not have is a fair trade. 7.53s was not, and asserting it was fast without running it was the same kind of unchecked claim #363 exists to catch.

Both assertions re-verified after the rewrite. make validate passes.

🤖 Generated with Claude Code

Gerrrt and others added 2 commits September 7, 2026 20:20
check_docs.py matched counted claims written as digits or as words, and
its NUMBER_WORDS table stopped at twenty. The comment explained the cap
as convenience — "Twenty is comfortably above any count here ... it is
not a claim about the ceiling" — which was true when written and stopped
being true without anything noticing.

On 2026-09-07 observability.md read "Coverage is forty-five rules of 67"
and two rules were added that day. The digit was caught both times. The
word was not, because forty-five was not in the table, and it was
corrected by hand twice. The check exists to prevent exactly that class
of defect and the defect was living inside it.

TWO CHANGES, and the second matters more.

  * The words are GENERATED through ninety-nine rather than typed —
    ones, teens, tens and the hyphenated compounds. 99 entries, so
    "forty-five" and "fifty-one" now parse. Verified: a stale
    "forty-nine" against a real 50 is now reported, where before it was
    silently skipped.

  * The ceiling is ASSERTED rather than assumed. Each claim pattern is
    re-run with the number token swapped for the ones this table cannot
    turn into an integer — hundred, thousand, dozen and friends — and a
    match is reported. So "a hundred panels" fails loudly instead of
    walking past every assertion, and the next time prose outgrows the
    table it says so rather than going quiet. That is the actual lesson
    of #367: extending the table alone would just move the silence.

The unparseable pattern carries an optional leading word, because these
always appear with a determiner or multiplier in prose — "a hundred",
"two hundred" — and without it the article sits between the verb and the
number where the claim pattern's whitespace cannot span it. That was the
first version's bug, found by testing the assertion rather than assuming
it fired.

Verified: all four forms caught with the file and line, a readable
spelled count above twenty now checked, and check_docs still runs in
well under a second despite the 99-branch alternation.

Closes #367

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CORRECTION TO THE PREVIOUS COMMIT, which claimed check_docs "still runs
in well under a second despite the 99-branch alternation". That was
wrong and I had not measured it. Measured, in this tree:

    main (20 words)                      1.57s
    previous commit (99-word flat list)  7.53s

A 4.8x regression on a check that runs in `make validate` and in CI.
Two causes, both fixed:

  * THE ALTERNATION WAS FLATTENED. Ninety-nine literal branches, applied
    for ~20 claim patterns across ~25 documents. Expressed as structure
    instead — tens, optionally hyphen ones, then teens, then ones — the
    same language needs 36 branches rather than 99.

  * THE CEILING SCAN RAN UNCONDITIONALLY, re-walking every document per
    claim pattern to look for words almost no document contains. It is
    now gated on one search per file, and the swapped patterns are
    compiled once via lru_cache instead of on every call.

    1.57s  main
    7.53s  flat table, unconditional scan
    5.54s  + gated scan
    2.23s  + structured alternation

2.23s for five times the vocabulary and an assertion main does not have
is a fair trade; 7.53s was not, and saying it was "well under a second"
without running it was the same kind of unchecked claim that #363 exists
to catch.

Both assertions re-verified after the rewrite: "forty-nine" against a
real 50 is reported, and "a hundred rules" is reported as unreadable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Gerrrt
Gerrrt merged commit bb4671a into main Sep 7, 2026
3 checks passed
@Gerrrt
Gerrrt deleted the gerrrt/check-docs-number-words-367 branch September 7, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check_docs cannot see a counted claim spelled out above twenty

1 participant