Repository navigation
Gate the Ruby version the README counts at, and drop a limitation that is gone - #889
Merged
Merged
Conversation
The docs site and the landing page have had conformancefigure for weeks. The
README — the page most people actually read — did not, and it showed.
Pointing the published v0.7.0 tool at this repository reported:
0 total claim(s) and 0 version claim(s) checked
conformancefigure: found no statement of the total; either the vocabulary
no longer matches how the docs phrase it, or the figure was dropped
Zero, on a README that states the total twice. It refused rather than passing,
which is the only reason the gap was visible at all. Three sentences were
phrased so the gate could not reach them, and one of the three was wrong:
* line 16 said **"It runs 23,672"**. 23,672 is what *passes*; rbgo runs
considerably more — the dated sweep ran 24,116 examples. The figure was
right and the verb was not, and a hand-maintained README is exactly where
that survives. Now "It passes", which is also what the docs site says.
* line 1267 said "the total, 23,672, is a derived summary" — an apposition the
pattern cannot cross. Now "their sum is 23,672,".
* line 782 said "| `RUBY_VERSION` | rbgo reports `"4.0.9"`" — the table pipe
and "rbgo" sit between the constant and the verb. Now the cell repeats the
constant.
Nothing was added to the tool's vocabulary for this. The alternative was to
teach it "runs", and that would have taught it to bless a sentence that was
wrong.
After: **2 total claim(s) and 1 version claim(s) checked**, all agreeing.
The new `readme-figure` job passes every input locally
(--baseline/--runtime/--shim). The tool's default is to fetch those three files
from this repository's main branch, which inside this repository is a
self-reference: a pull request that re-records BASELINE and updates the README
in one commit would be refused for disagreeing with the main it is about to
replace.
Verified to bite before committing — README.md:16 set to 23,999:
README.md:16: claims 23999, the gate records 23672
conformancefigure: 1 claim(s) disagree with the ruby repository
exit status 1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t is gone
This branch started on a false premise, which is worth stating because the
correction is the interesting part. I set out to gate the README's conformance
total, believing nothing checked it, and wired a CI job around the published
`conformancefigure`. Then I read
`TestReadmeConformanceCountMatchesTheBaseline`, which has been checking exactly
that for weeks — against BASELINE in the same tree, with no network and no
pinned external version. My job would have duplicated a better check and added
a dependency. It is not in this commit.
What that test does *not* cover is the version the total is counted at, and
that version selects the population counted: the move from 3.4.1 to 4.0.9
changed which `ruby_version_is` guards admit examples (+144 passing, 27 gone
red). The same number means a different thing at a different version.
TestReadmeRubyVersionMatchesTheRuntime reads `rubyVersion` from
internal/vm/kernel_introspect.go and checks it two ways:
* forward — every present-tense claim ("rbgo reports `"4.0.9"`", "already
reflect 4.0.9") must equal it, and fewer matches than shapes is a failure, so
a rewording that escapes the patterns fails instead of silently checking less;
* inverse — a line mentioning RUBY_VERSION and naming a different version must
declare itself historical ("sweep", "the move from"), and the exemptions are
logged rather than applied silently.
Both halves are needed, which a deliberate break showed. With the table row set
to 4.0.8 the forward check named it while the inverse check EXEMPTED the same
4.0.8, because that line also says "the move from": a line-granular marker
meant for the version left behind covered a wrong present-tense claim sitting
beside it.
Two README corrections, both measured rather than read:
* line 16 said "It **runs** 23,672". 23,672 is what passes; rbgo runs
considerably more. The figure was right and the verb was not, which is why
the existing numeric gate was green the whole time — no check on a number can
see a wrong verb.
* *"A backtick command holds the interpreter lock"* no longer reproduces. The
probe the README carried printed `frozen` in October and prints `ran` now,
five runs out of five, and rubydiff reports the program AGREE with MRI 4.0.7.
Removed, with no cause attributed: naming one without a control would be a
guess dressed as a fact.
`Timeout.timeout` was re-checked on the same pass and still fails open — rbgo
prints `returned` then `nil`, MRI 4.0.7 prints `raised` then `0.3` — so it
stays, now with that measurement beside it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Not mergeable, and not because of this change.
Two identical durations are a fixed cap, not a busy pool, so I have stopped |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I set out to gate the README's conformance total, believing nothing checked it,
and added a CI job around the published
conformancefigure. Then I readreadme_conformance_count_test.go: it hasbeen checking exactly that for weeks, against
BASELINEin the same tree, withno network and no pinned external version — strictly better than what I was
adding. The job is gone.
What actually was ungated: the version
TestReadmeConformanceCountMatchesTheBaselinechecks the total. It does notcheck the version that total is counted at, and that version selects the
population counted: the move from 3.4.1 to 4.0.9 changed which
ruby_version_isguards admit examples (+144 passing, 27 gone red). Thesame number means a different thing at a different version.
TestReadmeRubyVersionMatchesTheRuntimereadsrubyVersionfrominternal/vm/kernel_introspect.goand checks it both ways:Both halves are needed, and a deliberate break proved it. With the table
row set to
4.0.8, the forward check named it — while the inverse checkexempted the same
4.0.8, because that line also says "the move from". Aline-granular marker meant for the version left behind covered a wrong
present-tense claim sitting beside it.
Two README corrections, measured not read
It runs 23,672→It passes. 23,672 is what passes; rbgo runsconsiderably more. The figure was right and the verb was not — which is exactly
why the existing numeric gate was green throughout. No check on a number can
see a wrong verb.
"A backtick command holds the interpreter lock" no longer reproduces. The
probe the README itself carried printed
frozenin October and printsrannow, 5 runs out of 5, and
rubydiffreports the program AGREE with MRI4.0.7 — same output, not merely no-longer-compared output. Removed, with no
cause attributed: naming one without a control would be a guess dressed as a
fact.
Timeout.timeoutwas re-checked on the same pass and still fails open —rbgo prints
returnedthennil, MRI 4.0.7 printsraisedthen0.3— so itstays, now with that measurement beside it.
🤖 Generated with Claude Code