Repository navigation
compiler: _1 beside a written parameter list is a SyntaxError - #861
Conversation
The last of the five defects in #799 that this repository can fix — it fails at 3.4.1 and at 4.0.9 alike — and the one that cost nothing at all before it was looked for: ```console $ rbgo -e 'proc { |x| _1 }' # on main $ # no error, no warning, nothing ``` MRI rejects that at parse time. A block cannot have both a written parameter list and a numbered parameter, because the grammar reserves `_1`..`_9` for the numbered-parameter mechanism, so the two would be the same slot under two names. `language/numbered_parameters_spec.rb:49` pins eight shapes in one example, and the pairing is the interesting part: ```ruby -> { eval("-> () { _1 }") }.should.raise(SyntaxError, /ordinary parameter is defined/) -> { eval("-> (x) { _1 }") }.should.raise(SyntaxError, /ordinary parameter is defined/) -> { eval("proc { || _1 }") }.should.raise(SyntaxError, /ordinary parameter is defined/) -> { eval("proc { |x| _1 }") }.should.raise(SyntaxError, /ordinary parameter is defined/) … ``` Each kind of block appears twice, once with an EMPTY written list and once with a parameter in it: writing `||` or `()` at all is what declares the list, not what it contains. A check that looked for a parameter would have passed four of the eight. ## Where it goes, and why there `blockParamsAreImplicit` already tells a written list from one the parser synthesised out of `it` / `_1`..`_9` in the body — the trace being that a written list pads `Defaults` to one entry per parameter and a synthesised one leaves it empty. The builder now records that answer as `writtenParams`, set BEFORE the body is compiled, because the body is where `_1` is met. The refusal itself sits in `compileCall`, beside the `__LINE__` and `block_given?` intrinsics and shaped like them — receiverless, blockless, argumentless, bare name. Reaching that point means the name resolved to no local, so it was about to become a receiverless send; that is exactly the position in which MRI has already decided the program is ill-formed. Two scopes are deliberately NOT covered, because MRI's answer there is a different one and no spec in the gate asks for it: - a `for` body (`forScope`), since `for` opens no Ruby scope; - a method body and the top level, where `_1` stays a runtime NameError. ## Measured, with the control `language/numbered_parameters_spec.rb`, pinned corpus 87b1631, shim at 3.4.1: | binary | result | |---|---| | `main` | `pass=16 fail=1` | | this branch | **`pass=17 fail=0`** | The file is clean. The eight shapes, probed directly: | | `main` | this branch | |---|---|---| | `-> () { _1 }` … `lambda { \|x\| _1 }` (6 forms) | **no error at all** | SyntaxError | | `['a'].map { \|\| _1 }`, `['a'].map { \|x\| _1 }` | NameError, at CALL time | SyntaxError | | `proc { _1 }.call(7)` | `7` | `7` | | `proc { _1 + _2 }.call(3, 4)` | `7` | `7` | | `proc { \|x\| _10 }` | fine | fine | The two rows that said NameError are worth keeping in view: on `main` only the shapes that were actually CALLED failed, and the four that were merely defined raised nothing — so the defect was invisible to anything but a spec that evaluates the source. ## ⛔ Not one row of the new test was run against a reference interpreter The Ruby on this machine is 2.6.10 and numbered parameters arrived in 2.7, so the local oracle cannot answer any of these. The authority is the spec file, which pins all eight verbatim, and MRI's rule that the grammar reserves the names. The test says so in its own comment rather than borrowing the "verified against MRI 4.0.5" line its neighbour in the same file has earned. Tests: `internal/compiler/numbered_param_ordinary_test.go` — the eight shapes, three more written-list kinds (`_9`, a block parameter, a keyword), thirteen controls, and a second test asserting the two refusals stay DISTINGUISHABLE: binding `_1` is "reserved for numbered parameters" and reading it beside an ordinary parameter is "ordinary parameter is defined". They are different sentences in MRI and the specs match them separately, so collapsing them into one message would satisfy neither. Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The red was a known flake, not this change
One note for whoever reads the job data rather than the log: that job reports 🤖 Generated with Claude Code |
The macOS lane went red. Its log was unreadable — `gh api .../jobs/<id>/logs`
answered 403 twice while `gh api rate_limit` reported **4707 of 5000 core calls
remaining**, so that endpoint has its own burst limit and the hourly budget was
never the problem. Diagnosed from the diff instead, and confirmed locally with
the same instrument the lane uses.
Taking the gate off the hot path moved the `'#'` guard from
```go
if !vm.warningEnabled() || strings.HasPrefix(name, "#") { // before
```
to a statement of its own behind the `uninitialized` flag. Before, every global
read with `$VERBOSE` unset executed it; after, it is reachable only when a name
is both never-assigned and starts with `'#'` — which is `bytecode.ErrinfoCell`
("#$!") and nothing else.
⛔ And no Ruby program can reach it through the interpreter: vm.go's getgvar arm
intercepts that name **before** calling `gvarGet`. The only production caller is
AOT-compiled code, where the two tiers emit `vm.gvarGet` without the
interception. So running a `rescue` does not exercise the guard, and the
per-function coverage ratchet correctly noticed that nothing did.
A whitebox test calls `gvarGet(bytecode.ErrinfoCell)` directly, which is the
honest way to cover a path whose only caller is generated code. It sets
`$VERBOSE` to true first — rb_warning's gate — so a missing guard would produce
a line rather than being masked by the gate, and it plants the positive beside
it: the same call with an ordinary never-assigned name **must** warn, or the
silence would prove only that the gate is shut.
`gvarGet` measures 100.0% under `go tool cover -func` from those three tests
alone.
`main` is merged in, since #861 has landed.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…862) * compiler: assigning a constant inside a method body is a SyntaxError ```console $ rbgo -e 'def d; FOO = 1; end' # on main $ # nothing ``` MRI refuses that at parse time with `dynamic constant assignment`, and has for many versions. rbgo refused no form of it. ## It has its own spec, in a file nobody had connected to this `language/constants_spec.rb` fails on `Assignment dynamic assignment | raises SyntaxError` — measured `pass=88 fail=8` → **`pass=89 fail=7`** with this change, a gain this branch was not aiming at. I reached the defect sideways, from #799's fifth item, which I had written down as *"a non-ASCII upcased character must be rejected as a constant identifier"*. That title points at the lexer. The spec behind it is ```ruby -> { eval("def test\n ἍBB = 1\nend") }.should.raise(SyntaxError, /dynamic constant assignment/) ``` so what it needs is this check, plus a lexer that classifies `ἍBB` as a constant at all (go-ruby-parser#57 — rbgo's `defined?(ἍDD)` is nil where MRI says "constant"). Running the simplest version of the example — `def t; FOO_X = 1; end`, three words shorter — is what showed the real defect was ASCII-wide and in the compiler. Corrected on #799. ## The boundary is syntactic, and two measured rows prove it | | MRI | rbgo before | |---|---|---| | `def d; Class.new { FOO = 1 }; end` | refused | — | | `class C; define_method(:m) { FOO = 1 }; end` | **allowed** | — | Both blocks end up running as a method body, so reasoning from "will this run as a method?" gets both backwards. The rule is about an enclosing `def` **keyword**, reached through any number of blocks and stopped by a class or module body — which is exactly what `inDefBody` walks. The other row I expected to go the other way: a **scoped** assignment, `C::FOO = 1`, is refused inside a `def` too. Naming the namespace explicitly settles where the constant would go, and upstream refuses it anyway. ## Measured: eighteen shapes, eighteen agreeing Against `/usr/bin/ruby`. The rule predates that interpreter by many versions, so unlike the numbered-parameter check in #861 this oracle is a valid one here. Refused by both, after the change — plain, through one/two/three blocks, a lambda, `Class.new`, `instance_eval`, scoped, `::FOO`, `||=`, `+=`, masgn in either position, nested masgn, `def self.m`, `class << self` + `def`, `def o.m`, a nested `def`, a `rescue` clause, an `ensure` clause, and a default argument. Allowed by both — the top level, a class body, a module body, a nested class body, a block at the top level, a block in a class body, a `class << self` body, `define_method`, `module_eval`, and, inside a `def`, a local, an ivar, a class variable, a global, and READING a constant. One row deliberately still diverges: `def d; class I; end; end` is `class definition in method body` upstream and still compiles here. Different message, different construct, different check — filed as #860, and the `inDefBody` predicate this adds is what it will use. ## Blast radius No valid Ruby does this, so our own Ruby should be clean — and it is: the prelude, stdlib and gem tests pass, which includes regenerating the frozen prelude. `language/def_spec.rb` is unchanged at `pass=63 fail=8 error=1`. ⛔ The full `internal/vm` suite was NOT run to completion for this branch. The machine is at load 37–76 on 16 cores (around forty peer sessions are building across this fleet), and the run on the previous branch hit `panic: test timed out after 10m0s` at 604s where the same suite took 297s earlier this morning. Above roughly 8 this machine stops measuring, so I am claiming nothing from it either way; the `test` lanes and the ruby/spec ratchet here are the verdict. Tests: `internal/compiler/dynamic_constant_assignment_test.go` — the refusals, the fifteen controls, a planted positive (every control row passes with the check deleted), a same-name/two-scopes pair proving the predicate reads the SCOPE rather than the name, and `inDefBody` pinned directly through sources whose outcome depends only on the walk. Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * aot: the setconst codegen case was testing invalid Ruby The three `arch` lanes failed on this branch, and the log names one case: ``` --- FAIL: TestCompileSupported/setconst codegen_test.go:84: compile "def m\n X = 1\n X\nend\nm": compile error: dynamic constant assignment ``` That fixture is **not valid Ruby**. MRI refuses a constant assignment inside a method body, and this branch makes rbgo refuse it too — so the fixture was exercising the `OpSetConst` arm through a source no Ruby program can contain. ## What it was hiding A method's ISeq can never legally hold `OpSetConst`, because upstream forbids the assignment in every form inside a `def` — including the scoped one, `C::X = 1`. Where the opcode does reach a child ISeq is a **block** whose lexical chain does not cross a `def`, or a class/module body. `methodISeq` takes `Children[0]` whatever kind it is, so a top-level block serves directly: ```go {"setconst", "[1].each { X = 1 }\nX", "block in <main>", "vm.consts["}, ``` `[1].each { X = 1 }` at the top level is accepted by MRI — measured, alongside the eight refusals in the commit below. So the fixture change is not a workaround for the new refusal. The old source could only ever have described a program the language does not permit, and the question of which ISeq kinds legally carry `OpSetConst` was invisible while it stood. ## ⛔ Why CI found this and I did not I ran `./internal/compiler/` and `./internal/vm/` for this branch, plus the prelude/stdlib/gem tests — and never `./internal/aot/`. A compiler change reaches every package that compiles Ruby, so the suite to run was `./...`. `go test ./... -count=1` now passes on this branch: exit 0, no FAIL lines, with the full `internal/vm` suite included (which also clears the only local measurement I could not complete when this PR was opened — the machine had been at load 37–76 and the suite had timed out at 604s). `main` is merged in as well, since it has moved (#856 landed). Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * compiler: inDefBody's trailing return was unreachable The macOS lane failed, and not on a percentage: ``` -- NEWLY BELOW 100% (1) -- internal/compiler/compiler.go inDefBody 83.3% (was fully covered) ##[error]1 function(s) dropped below 100% -- named above ``` Five statements of six. The missing one was the `return false` **after** the loop, and it is not missing a test — it cannot have one. Every builder chain is rooted at `<main>`, `<class:…>`, `<module:…>`, `<singleton class>` or `<binding>`, and none of those is a block, so the walk always leaves through the `!b.isBlock` arm and never by exhausting `b != nil`. So the arm returns through a `break` and the function ends with the single reachable `return false`. `inDefBody` and `refuseDynamicConstAssign` both measure 100.0% under `go tool cover -func`, the same instrument the lane uses. ⛔ Worth noting about this gate, because my own notes had it wrong: the percentage is **reported and not gated** any more — `coverage: 99.9% (reported, NOT gated -- see below)` — and `scripts/coverage/cmd/covratchet` gates per-function against `scripts/coverage/BELOW100.<lane>` instead. So a POSIX lane failing here is no longer the rounding knife-edge it used to be; it names a function, and the two "further below than recorded" rows it also printed (`bootstrap` 99.6% and `registerNetHTTP` 92.3%) are pre-existing and ungated. That is a strictly better gate than the one that had to be argued with. The comment now says why the return cannot be covered, which is cheaper than the next person rediscovering it. Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* vm: take the uninitialized-global gate off the hot path #856 put `vm.globals["$VERBOSE"]` in front of **every** Ruby-level global read, in the interpreter and in both AOT tiers: ```go if !vm.warningEnabled() || strings.HasPrefix(name, "#") { return vm.gvar(name) } ``` The comment I shipped with it said "one map lookup and false in any ordinary run". That was true and beside the point. `getgvar` is a hot instruction, the lookup was unconditional, and a cost on a hot path is not excused by being small — particularly one that buys nothing on the reads that have a value. ## The reorder Every test now sits behind `uninitialized`, the flag `gvarResolve` already returns: ```go v, uninitialized := vm.gvarResolve(name) if !uninitialized { return v } ``` The resolution happens exactly once, as it did before #856 existed, and a read that found a value returns through the same path it always did. `$stdout`, `$LOAD_PATH`, `$~`, `$1` and every assigned user global now pay **nothing** for the warning — not the map lookup, not the `strings.HasPrefix`, not the table lookups. Nothing is deferred that could matter: `uninitialized` can only be true for a name that resolved to no special, no match-data slot and no entry in `vm.globals`, which is both rare and already an anomaly in the program being run. ## ⛔ Why this is a reorder and not a benchmark The machine is at 1-minute load 8.9 and 5-minute load 16 — around twenty-five peer sessions build across this fleet — and above roughly 8 it cannot time anything honestly. So the choice was between a number I could not defend and a structure that makes the number unnecessary. Removing a cost beats measuring it. ## ⛔ And the reorder found dead code, which the coverage gate would have caught Deferring the tests made one of them unreachable: ```go if mriPredefinedGvars[name] || englishAlias[name] != "" { ``` Probed, by walking both tables through `gvarResolve`: **0 of englishAlias's 9 keys** resolve uninitialised, while **9 of the predefined table's 42** do — and those nine are exactly `$,` `$-F` `$-i` `$.` `$;` `$?` `$FILENAME` `$\` `$_`, the names this VM leaves on the ordinary user-global path. So the English clause is gone, and the invariant that makes it unnecessary is written down instead: `englishAlias` lists only names whose target this VM models, so `specialGvar` or the match-data resolver answers every one of them and none can arrive there. An alias added for an unmodelled target would warn — the comment says so, and says what to do about it. ## Measured `cmd/oracle` on the globals probe, this branch against the reference: `stderr 810 vs 810 bytes`, three warnings each, same names and same message text. The only differences are the two the probes' README records as expected — 2.6.10's backtick quote style and its one-line lag in a warning's position. Behaviour is unchanged, which is the point: the warning tests and the predefined-table walk from #856 pass untouched. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * vm: cover the errinfo-cell guard, which only generated code reaches The macOS lane went red. Its log was unreadable — `gh api .../jobs/<id>/logs` answered 403 twice while `gh api rate_limit` reported **4707 of 5000 core calls remaining**, so that endpoint has its own burst limit and the hourly budget was never the problem. Diagnosed from the diff instead, and confirmed locally with the same instrument the lane uses. Taking the gate off the hot path moved the `'#'` guard from ```go if !vm.warningEnabled() || strings.HasPrefix(name, "#") { // before ``` to a statement of its own behind the `uninitialized` flag. Before, every global read with `$VERBOSE` unset executed it; after, it is reachable only when a name is both never-assigned and starts with `'#'` — which is `bytecode.ErrinfoCell` ("#$!") and nothing else. ⛔ And no Ruby program can reach it through the interpreter: vm.go's getgvar arm intercepts that name **before** calling `gvarGet`. The only production caller is AOT-compiled code, where the two tiers emit `vm.gvarGet` without the interception. So running a `rescue` does not exercise the guard, and the per-function coverage ratchet correctly noticed that nothing did. A whitebox test calls `gvarGet(bytecode.ErrinfoCell)` directly, which is the honest way to cover a path whose only caller is generated code. It sets `$VERBOSE` to true first — rb_warning's gate — so a missing guard would produce a line rather than being masked by the gate, and it plants the positive beside it: the same call with an ordinary never-assigned name **must** warn, or the silence would prove only that the gate is shut. `gvarGet` measures 100.0% under `go tool cover -func` from those three tests alone. `main` is merged in, since #861 has landed. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The last of the five defects in #799 that this repository can fix — it fails at
3.4.1 and at 4.0.9 alike — and the one that cost nothing at all before it was
looked for:
MRI rejects that at parse time. A block cannot have both a written parameter
list and a numbered parameter, because the grammar reserves
_1.._9for thenumbered-parameter mechanism, so the two would be the same slot under two names.
language/numbered_parameters_spec.rb:49pins eight shapes in one example, andthe pairing is the interesting part:
Each kind of block appears twice, once with an EMPTY written list and once with
a parameter in it: writing
||or()at all is what declares the list, notwhat it contains. A check that looked for a parameter would have passed four of
the eight.
Where it goes, and why there
blockParamsAreImplicitalready tells a written list from one the parsersynthesised out of
it/_1.._9in the body — the trace being that a writtenlist pads
Defaultsto one entry per parameter and a synthesised one leaves itempty. The builder now records that answer as
writtenParams, set BEFORE thebody is compiled, because the body is where
_1is met.The refusal itself sits in
compileCall, beside the__LINE__andblock_given?intrinsics and shaped like them — receiverless, blockless,argumentless, bare name. Reaching that point means the name resolved to no
local, so it was about to become a receiverless send; that is exactly the
position in which MRI has already decided the program is ill-formed.
Two scopes are deliberately NOT covered, because MRI's answer there is a
different one and no spec in the gate asks for it:
forbody (forScope), sinceforopens no Ruby scope;_1stays a runtime NameError.Measured, with the control
language/numbered_parameters_spec.rb, pinned corpus 87b1631, shim at 3.4.1:mainpass=16 fail=1pass=17 fail=0The file is clean. The eight shapes, probed directly:
main-> () { _1 }…lambda { |x| _1 }(6 forms)['a'].map { || _1 },['a'].map { |x| _1 }proc { _1 }.call(7)77proc { _1 + _2 }.call(3, 4)77proc { |x| _10 }The two rows that said NameError are worth keeping in view: on
mainonly theshapes that were actually CALLED failed, and the four that were merely defined
raised nothing — so the defect was invisible to anything but a spec that
evaluates the source.
⛔ The local
internal/vmrun is VOID, and CI is the checkinternal/compilerpasses in 0.2s andinternal/aotin 0.18s. Theinternal/vmrun beside them did not fail an assertion — it hitpanic: test timed out after 10m0sat 604s, where the same suite took 297s and434s earlier this morning, with the machine at load 37.5 / 74.3 / 76.5 on 16
cores (around forty peer sessions are building across this fleet right now).
Above roughly 8 that machine stops measuring anything, so this is an artifact and
I am claiming neither a regression nor the absence of one from it. The three
testlanes here, on clean runners, are the verdict; if one of them reds, thecause is in this diff and not in the load.
⛔ Not one row of the new test was run against a reference interpreter
The Ruby on this machine is 2.6.10 and numbered parameters arrived in 2.7, so
the local oracle cannot answer any of these. The authority is the spec file,
which pins all eight verbatim, and MRI's rule that the grammar reserves the
names. The test says so in its own comment rather than borrowing the
"verified against MRI 4.0.5" line its neighbour in the same file has earned.
Tests:
internal/compiler/numbered_param_ordinary_test.go— the eight shapes,three more written-list kinds (
_9, a block parameter, a keyword), thirteencontrols, and a second test asserting the two refusals stay DISTINGUISHABLE:
binding
_1is "reserved for numbered parameters" and reading it beside anordinary parameter is "ordinary parameter is defined". They are different
sentences in MRI and the specs match them separately, so collapsing them into
one message would satisfy neither.
Refs #799
🤖 Generated with Claude Code