[tannevaled]

core/random/bytes_spec.rb answers 8 or 9 on the same binary. That is documented in four places in this repository already — including cmd/specdelta's own comment, which exists because that file once reported -1 between two engines differing only in a version string.

The 4.0.9 re-record (#882) drew 9 and banked it. The gate failed on the very next pull request (#883, an NDArray change that cannot touch ruby/spec at all), naming a regression that was a coin flip:

passing examples: 23672 (baseline 23673, -1)
-- REGRESSED (1) --
   core/random/bytes_spec.rb: 8 passing, baseline 9 (-1)

That is not a conformance failure — it is a ~50 % flaky gate, on main and on every branch, self-inflicted by recording a maximum as though it were a guarantee.

The fix

A baseline row means "no run may come in below this". For an unstable file the only honest row is its lowest observed value:

• at 8 the gate is deterministic;
• a draw of 9 is reported as an improvement, not a pass;
• a genuine drop to 7 is still caught.

Nothing about the shrink-only property is weakened. The floor is what the gate was always meant to hold; recording 9 asserted a guarantee the interpreter does not give.

BASELINE	core/random/bytes_spec.rb 9 → 8
README.md	23,673 → 23,672, in all three places

The README figures were named by TestReadmeConformanceCountMatchesTheBaseline, not edited by eye — the same mechanism that forced the 4.0.9 update.

Why it happened, and what is still missing

The flake was written down four times and gated zero times, which is the pattern #876 and #877 were built to break — and it still caught me, one layer up: those gates watch whether a re-record loses examples, not whether the number it banks is reproducible.

cmd/specdelta already refuses to build a verdict on a single run (-repeat, and the UNSTABLE marking that keeps such a file out of the regression verdict). cmd/ratchet has no notion of an unstable file at all. That asymmetry is the real gap and is filed separately; this PR stops the bleeding and records why in the conformance README, so the next re-record does not put 9 back.

Refs #799

🤖 Generated with Claude Code