Repository navigation
Phase 8 named a conformance gate that no longer exists, and a stale total - #15
Merged
Merged
Conversation
…otal The blurb said the sweep is "gated in CI by a shrink-only floor of 22,475". That floor was replaced by a per-file baseline in go-embedded-ruby/ruby#739, so the sentence described a mechanism that is gone -- in the present tense, which is the part that matters: a dated number going stale is honest, a dated description of how CI works today is not. The total is refreshed with it: 23,481, which is what scripts/conformance/ rubyspec/BASELINE sums to on main today (2,202 of 2,206 spec files produce a result). The repository README reads that figure out of BASELINE mechanically and a test fails if they disagree, so this page now quotes the same source rather than a hand-copied snapshot. The old floor's history is kept rather than deleted, because it is the argument for the current gate: it needed a margin wider than the largest spec file -- 384 examples in core/encoding/compatible_spec.rb -- and so could not see any regression smaller than that, while sitting 211 below the measured total. A drop of that size could not distinguish one file failing to load from hundreds of specs regressing. That is why the gate is now per file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"the total `BASELINE` sums to on `main`" was missing its relative pronoun. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled
added a commit
to go-embedded-ruby/docs
that referenced
this pull request
Oct 4, 2026
`docs/conformance.md` describes the ruby/spec gate as a frozen scalar floor and tells the reader to raise it with `UPDATE_FLOOR=1`. That mechanism was replaced by a per-file baseline in go-embedded-ruby/ruby#739. The instruction is the part that matters. `UPDATE_FLOOR` does not exist anywhere in that repository -- `git grep UPDATE_FLOOR origin/main` returns nothing -- so a reader following this page runs a variable that silently does nothing. The real one is `UPDATE_BASELINE=1`, which `run.sh` passes to the ratchet as `-update`. Four corrections, all verified against `scripts/conformance/rubyspec/run.sh` on `main` rather than from memory: - the mechanism paragraph: the runner records one row per spec file (`OK<TAB>path<TAB>passing`, or `FILEFAIL<TAB>path`) and a file that regresses is NAMED, rather than summing a total and comparing it to a threshold; - the table row `FLOOR | 22 475`, a dead artefact with a stale number, becomes the row the README uses: "enforced by CI | a per-file baseline, not a single number"; - the "the floor is a gate, not a score" admonition keeps its point -- the guaranteed figure and a measured total are different things and confusing them misreports in both directions -- but says it about the baseline, and gives the current sum (23 481, 2026-10-04) with the note that the README reads it out of the file mechanically under test; - "It prints the measured total and the frozen floor" was describing output `run.sh` no longer produces. The argument for the per-file record is written in rather than asserted, because it is the reason the old gate went: a single threshold needed a margin wider than the largest spec file -- 384 examples in `core/encoding/compatible_spec.rb`, since a file that fails to load loses its whole count at once -- so a drop inside that margin could not distinguish one file failing to load from hundreds of specs regressing. The dated 2026-09-26 sweep table keeps its numbers. A dated measurement going stale is honest; a present-tense description of how CI works today is not, and that distinction is the whole of this change. Same correction as go-embedded-ruby/go-embedded-ruby.github.io#15 made to the landing page. Found by a scan that first reported zero stale counts: this file separates thousands with a space (`22 475`), so a grep for comma-formatted numbers found none. The floor-language search is what caught it. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Phase 8 blurb said the ruby/spec sweep is "gated in CI by a shrink-only floor of 22,475". That floor was replaced by a per-file baseline in go-embedded-ruby/ruby#739, so the sentence described, in the present tense, a mechanism that no longer exists.
That tense is the point. A dated number going stale is honest — the blurb carried its date and its commit, which is exactly right. A dated description of how CI works today is not, because nothing about it tells the reader it has expired.
What changed
3e8e3cc)23,481 is what
scripts/conformance/rubyspec/BASELINEsums to onmaintoday (2,202 of 2,206 spec files produce a result). I computed it from the file rather than copying it from anywhere: the repository README reads the same figure out ofBASELINEmechanically andTestReadmeConformanceCountMatchesTheBaselinefails if the two disagree, so this page now quotes the same source instead of a hand-made snapshot that can only drift.The old floor's history is kept, not deleted
It is the argument for the current gate, so it is worth a reader's time:
The largest single file carries 384 examples (
core/encoding/compatible_spec.rb), and a file that fails to load loses its whole count at once — so a scalar floor had to leave a margin wider than that, and a drop inside the margin could not distinguish one file failing to load from hundreds of specs regressing. That is why the gate is now per file.Checked
hugo.tomlstill parses (9 phases; Phase 8status = "active"unchanged). The rest of the blurb — the "onlylanguage/andcore/are swept, so this is not a percentage of Ruby" caveat, and the note that two of the jumps were fixes to the measurement rather than the interpreter — is untouched; both are still true and both are the kind of caveat worth keeping.Also swept the file for other stale figures while I was in it: the two remaining occurrences of
22,475and "shrink-only" are inside the historical clause above, by design.🤖 Generated with Claude Code