check_docs cannot see a counted claim spelled out above twenty #367

scripts/check_docs.py matches counted claims written as digits or as words, and its NUMBER_WORDS table stops at twenty:

NUMBER_WORDS = {
    "one": 1, ... "nineteen": 19, "twenty": 20,
}

The module docstring explains the cap as a convenience — "Twenty is comfortably above any count here — the cap keeps the alternation short, it is not a claim about the ceiling." That was true when it was written. It is not true now.

The instance

docs/observability.md carries the unit-test coverage claim:

Coverage is forty-five rules of 67 so far

Adding a rule with tests in #360 made both numbers stale. check_docs caught the denominator — 67 is a digit — and silently passed the numerator, because forty-five is not in the table. It was corrected by hand to forty-six; nothing would have caught it going stale again.

This is precisely the failure the whole check exists to prevent, surviving inside its own fix — the same sentence the docstring already uses about the first version of this check.

Why it will keep happening

The coverage numerator is the number here most likely to change, it is written in words, and it is now permanently above twenty. Rule counts crossed twenty long ago; dashboards and panels will follow.

Options

1. Extend the table through the compound forms — twenty-one … ninety-nine, generated rather than typed. The alternation grows but stays mechanical.
2. Fail on an unparsed number word next to a checked noun, so a spelled count the table cannot read is an error rather than a silent pass. Stricter, and it makes the ceiling visible instead of quiet.
3. Require digits above twenty in prose and assert that. Cheapest, but it is a style rule enforced by a checker rather than a fix to the checker.

Option 2 is the one that matches how the rest of this repository handles a check that cannot see something — check_firewall_claims.py stops rather than reporting silence as success, and check_alert_channels.py exits if its parser stops matching.

Related

• Alert-rule and panel counts are stale in 8 places, including a deploy-runbook check #72 — the stale-count defect this checker was built for.
• Patch state is collected on the monitoring host only — oracle, Saruman and morpheus have none #360 — where this surfaced.

🤖 Generated with Claude Code