Skip to content

Ruby 4.0.9: the version rbgo reports - #882

Merged
tannevaled merged 3 commits into
mainfrom
ruby-40-parity
Nov 3, 2026
Merged

tannevaled merged 3 commits into
mainfrom
ruby-40-parity

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

rbgo reports RUBY_VERSION 4.0.9, the latest MRI release, and the one
behaviour that could not land without the bump lands with it.

Measured before it was made

Both engines built from one commit minus the version string, each from a
clean distinct revision so the provenance gate added in #874 is satisfied
rather than waived, over core and language together — the whole surface
the move touches, which #875 made expressible:

2206 file(s): control 23528 passing, treatment 23672 passing, +144
45 file(s) moved, 13 regressed, 27 example(s) WENT RED

+144 is not progress and 13 regressed is not 13 defects. Six of the
thirteen lost only examples the corpus stops asking for — each has a CLOSED
guard in the census and no WENT RED line. The clearest case is
core/objectspace/_id2ref_spec.rb, which reads 10 → 0: below 4.0 the file
holds eleven examples and rbgo passes ten; at 4.0.9 that block closes and a
single one opens. The work there is one example, not ten.

The real remaining work of the move is 27 named examples, listed per item
in #799.

The one behaviour that comes with the bump

compiler.go already said so: hiding _1 from a Binding "belongs with the
RUBY_VERSION bump"
. Below 4.0 ruby/spec asserts the opposite
(numbered_parameters_spec.rb:93, ruby_version_is ""..."4.0"), so doing it
alone would have traded four passing examples for four failing ones.

The asymmetry is the whole change

proc { _1 }.parameters                        # [[:opt, :_1]]  -- UNGATED, every version
-> { _1; binding.local_variables }.call("a")  # [:_1] below 4.0, [] from 4.0

numbered_parameters_spec.rb:85 carries no version guard, so blanking
b.params as well would have traded four examples for two. it is blanked
from both, because MRI gives it no name in #parameters either. So:
b.locals for both kinds, b.params for it only.

Refusing by name, before any lookup

MRI tests rb_numparam_id_p at the top of all three methods (proc.c
v4.0.9:549, 602, 649), which has two consequences that are easy to miss and are
both tested:

  • it raises in a binding that has no numbered parameter, and at the top
    level — _1 is not a local being hidden, it is not a local variable name;
  • #local_variable_defined? raises rather than answering false.

And it uses rb_numparam_id_p, not rb_implicit_param_p (proc.c:517),
which also covers it. That narrowing is necessary: it is a legal local
variable name — it = 5 compiles — so widening the refusal would break every
binding holding a variable called it. There are controls for that and for the
_0 / _10 / _ boundary.

Measured on its own, control and treatment both at 4.0.9: +4, 0 regressed, 0
WENT RED
. core/proc is in the measured set deliberately — blanking
b.params would have shown up there as a loss.

What the bump breaks

go test ./... across the repository: four assertions, every one a
deliberate pin of the pre-4.0 behaviour, and nothing else in a 210-second VM
suite. Each is rewritten to the 4.0 behaviour with its comment, not flipped
silently.

BASELINE, recorded where it is enforced

Not recorded on a laptop — this machine reads +9 against CI for no change
in the code. Recorded by the rubyspec-baseline lane (#877) on the pinned
ubuntu-24.04, and downloaded as its artefact. 23,511 → 23,673.

The lane was dispatched without -absorb first, because the #876 gate had
only ever refused a synthetic loss:

::error::REFUSING to re-record: 11 file(s) went backwards -- named above

Both of the lane's asymmetries held on that run: the diffstat step ran, the
artefact step was skipped, so no unchanged BASELINE was offered under the
name "the re-recorded baseline". The second dispatch passed -absorb, and the
baseline now carries the stamp naming all eleven.

⛔ Comparing the sets, not the counts, found a hole in the gate

Local predicted 13 going backwards; CI named 11. Both missing entries matter:

  • language/numbered_parameters_spec.rb is absent because this branch fixes
    it
    — independent confirmation in CI of the +4 measured locally;
  • language/variables_spec.rb is absent because the baseline was stale. CI
    reported 118 passing, baseline 115 (+3) — an improvement — while the file
    has passed 119 since deps: go-ruby-parser v0.14.0 closes the last defect of #799 #871, and the move genuinely costs one example there. A
    real regression reported as a gain.

A per-file floor cannot see a regression smaller than a file's unbanked
improvement. Filed as #881; it is the exact complement of #876. This fresh
measurement clears the accumulated slack as a side effect.

Not changed on a hunch

RbConfig::CONFIG["ruby_version"] may be the API version in MRI (4.0.0 for
every 4.0.x) rather than the full one. Nothing in ruby/spec asserts that key, so
the corpus cannot settle it, and a version bump is not the place to change
behaviour on a guess. Filed as #878.

Refs #799
Refs #878, #881

🤖 Generated with Claude Code

tannevaled and others added 3 commits November 3, 2026 13:55
RUBY_VERSION moves from 3.4.1 to 4.0.9, and the one behaviour that could not
land without it lands with it.

## Why these two are one commit

compiler.go said so already: hiding `_1` from a Binding "belongs with the
RUBY_VERSION bump". Below 4.0 ruby/spec asserts the OPPOSITE --
language/numbered_parameters_spec.rb:93 is `ruby_version_is ""..."4.0"` and
pins `binding.local_variables == [:_1]` -- so doing it alone would have traded
four passing examples for four failing ones.

## The asymmetry is the whole change

`_1` leaves the LOCAL TABLE and stays in #parameters:

    proc { _1 }.parameters                        # [[:opt, :_1]]  -- UNGATED
    -> { _1; binding.local_variables }.call("a")  # [:_1] below 4.0, [] from 4.0

numbered_parameters_spec.rb:85 carries no version guard, so blanking b.params
as well would trade four examples for two. `it` is blanked from BOTH, because
MRI gives it no name in #parameters either. So: b.locals for both kinds,
b.params for `it` only.

## Refusing by NAME, before any lookup

MRI tests rb_numparam_id_p at the TOP of #local_variable_get, #local_variable_set
and #local_variable_defined? (proc.c v4.0.9:549, 602, 649):

    if (rb_numparam_id_p(lid)) {
        rb_name_err_raise("numbered parameter '%1$s' is not a local variable", ...)
    }

Two consequences that are easy to miss and are both tested:

  - it raises in a binding that has NO numbered parameter, and at the top
    level. `_1` is not a local being hidden; it is not a local variable NAME.
  - #local_variable_defined? RAISES rather than answering false. "Not defined"
    is the natural reading and the wrong one.

And it uses rb_numparam_id_p, NOT rb_implicit_param_p (proc.c:517-521), which
also covers `it`. That narrowing is deliberate in MRI and necessary here: `it`
is a LEGAL local variable name -- `it = 5` compiles -- so widening the refusal
would break every binding holding a variable called `it`. There is a control
for it, and for the `_0` / `_10` / `_` boundary.

## What the bump breaks, measured

`go test ./...` over the whole repository: FOUR assertions, every one of them a
deliberate pin of the pre-4.0 behaviour, and nothing else in a 210-second VM
suite.

    internal/compiler  TestImplicitBlockParamsAreHidden    "want the numbered names kept"
    internal/vm        TestImplicitItIsInvisible           "_1 is a binding local"
    internal/vm        TestImplicitParametersAreNotLocalVariables
                         subtest: "while _1 stays visible at the version rbgo reports"
    internal/vm        TestPBRbConfigKeys                  the version string

Each is updated to the 4.0 behaviour with its comment rewritten, not flipped
silently. The third gains a case for the point that `it` and `_1` are refused
by DIFFERENT messages.

The fourth raises a question this commit does not answer: MRI may report the
API version ("4.0.0") rather than the full one for
RbConfig::CONFIG["ruby_version"]. Nothing in ruby/spec asserts that key, so the
corpus cannot settle it. Filed as #878 rather than changed on a hunch.

Refs #799
Refs #878

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recorded by the rubyspec-baseline lane (#877) on the pinned ubuntu-24.04 --
the same runner the ratchet gates on -- and downloaded as its artefact. Not
recorded here: this machine reads +9 against CI for no change in the code.

    23,511 -> 23,673 passing examples

## The refusal was exercised on the real case first

The lane was dispatched WITHOUT -absorb before it was dispatched with it,
because the gate added in #876 had only ever refused a synthetic loss:

    re-record: 2206 files measured, 2206 in the baseline being replaced
    -- REGRESSED (11) --
       core/objectspace/_id2ref_spec.rb: 0 passing, baseline 10 (-10)
       core/set/flatten_merge_spec.rb: 0 passing, baseline 3 (-3)
       ...
    ::error::REFUSING to re-record: 11 file(s) went backwards -- named above

Both of the lane's deliberate asymmetries held on that run: the diffstat step
ran (it carries always()) and the artefact step was SKIPPED (it does not), so
no unchanged BASELINE was offered as "the re-recorded baseline".

The second dispatch passed -absorb, and the baseline now carries the stamp:

    # ⛔ ABSORBED 11 file(s) that went backwards, recorded with -absorb on 2026-11-03:
    #     core/gc/config_spec.rb: 7 passing, baseline 8 (-1)
    #     ...

Six of the eleven are guards CLOSING -- examples 4.0 stops asking for, each
with a `ruby_version_is ""..."4.0"` site in the census -- and not defects.

## ⛔ What comparing the SETS found, which the counts hid

The local sweep predicted 13 files going backwards; CI named 11. Both missing
entries are informative:

  - language/numbered_parameters_spec.rb is absent because THIS BRANCH fixes
    it. Independent confirmation in CI of the +4 measured locally.

  - language/variables_spec.rb is absent because the baseline was STALE. CI
    reported "118 passing, baseline 115 (+3)" -- an improvement -- while the
    file has passed 119 since #871 and the move genuinely costs one example
    there. A real regression reported as a gain.

That is a structural blind spot in the gate: a per-file floor cannot see a
regression smaller than the file's unbanked improvement. Filed as #881. It is
the exact complement of #876, which stops a re-record banking a LOSS; this is
the cost of not banking a WIN -- and it existed because until #877 there was no
correct way to re-record at all.

This fresh measurement clears the accumulated slack as a side effect.

## README

TestReadmeConformanceCountMatchesTheBaseline named all three stale sites and
the new figure, which is how the README was updated rather than by hand. The
dated table is left as the 3.4.1 sweep it is and now says so.

Refs #799
Refs #881

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit bbbed8e into main Nov 3, 2026
50 of 51 checks passed
@tannevaled
tannevaled deleted the ruby-40-parity branch November 3, 2026 19:09
tannevaled added a commit that referenced this pull request Nov 3, 2026
…ot (#885)

`core/random/bytes_spec.rb` answers 8 or 9 on the same binary. That is
documented in FOUR places in this repository, including cmd/specdelta's own
comment -- which exists because that file once reported -1 between two engines
differing only in a version string.

The 4.0.9 re-record (#882) drew 9 and banked it. The gate then failed on the
very next pull request, naming a regression that was a coin flip:

    passing examples: 23672 (baseline 23673, -1)
    -- REGRESSED (1) --
       core/random/bytes_spec.rb: 8 passing, baseline 9 (-1)

That is not a conformance failure. It is a ~50% flaky gate on main and on every
branch, self-inflicted by recording a maximum as though it were a guarantee.

A baseline row means "no run may come in below this". For an unstable file the
only honest row is its LOWEST observed value: at 8 the gate is deterministic, a
9 is reported as an improvement, and a real drop to 7 is still caught. Nothing
about the shrink-only property is weakened -- the floor is what the gate was
always supposed to hold.

  BASELINE  core/random/bytes_spec.rb  9 -> 8
  README    23,673 -> 23,672 in all three places, named by
            TestReadmeConformanceCountMatchesTheBaseline rather than by hand

The conformance README now carries why, so the next re-record does not put it
back. The deeper gap is that `cmd/ratchet` has no notion of an unstable file at
all, while `cmd/specdelta` already refuses to build a verdict on a single run
(-repeat, and the UNSTABLE marking). Filed separately.

Refs #799

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant