Skip to content

compiler: _1 beside a written parameter list is a SyntaxError - #861

Merged
tannevaled merged 2 commits into
mainfrom
numbered-param-ordinary
Nov 1, 2026
Merged

tannevaled merged 2 commits into
mainfrom
numbered-param-ordinary

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

The last of the five defects in #799 that this repository can fix — it fails at
3.4.1 and at 4.0.9 alike — and the one that cost nothing at all before it was
looked for:

$ rbgo -e 'proc { |x| _1 }'   # on main
$                             # no error, no warning, nothing

MRI rejects that at parse time. A block cannot have both a written parameter
list and a numbered parameter, because the grammar reserves _1.._9 for the
numbered-parameter mechanism, so the two would be the same slot under two names.

language/numbered_parameters_spec.rb:49 pins eight shapes in one example, and
the pairing is the interesting part:

-> { eval("-> () { _1 }")         }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("-> (x) { _1 }")        }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("proc { || _1 }")       }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("proc { |x| _1 }")      }.should.raise(SyntaxError, /ordinary parameter is defined/)
…

Each kind of block appears twice, once with an EMPTY written list and once with
a parameter in it: writing || or () at all is what declares the list, not
what it contains. A check that looked for a parameter would have passed four of
the eight.

Where it goes, and why there

blockParamsAreImplicit already tells a written list from one the parser
synthesised out of it / _1.._9 in the body — the trace being that a written
list pads Defaults to one entry per parameter and a synthesised one leaves it
empty. The builder now records that answer as writtenParams, set BEFORE the
body is compiled, because the body is where _1 is met.

The refusal itself sits in compileCall, beside the __LINE__ and
block_given? intrinsics and shaped like them — receiverless, blockless,
argumentless, bare name. Reaching that point means the name resolved to no
local, so it was about to become a receiverless send; that is exactly the
position in which MRI has already decided the program is ill-formed.

Two scopes are deliberately NOT covered, because MRI's answer there is a
different one and no spec in the gate asks for it:

  • a for body (forScope), since for opens no Ruby scope;
  • a method body and the top level, where _1 stays a runtime NameError.

Measured, with the control

language/numbered_parameters_spec.rb, pinned corpus 87b1631, shim at 3.4.1:

binary result
main pass=16 fail=1
this branch pass=17 fail=0

The file is clean. The eight shapes, probed directly:

main this branch
-> () { _1 } … lambda { |x| _1 } (6 forms) no error at all SyntaxError
['a'].map { || _1 }, ['a'].map { |x| _1 } NameError, at CALL time SyntaxError
proc { _1 }.call(7) 7 7
proc { _1 + _2 }.call(3, 4) 7 7
proc { |x| _10 } fine fine

The two rows that said NameError are worth keeping in view: on main only the
shapes that were actually CALLED failed, and the four that were merely defined
raised nothing — so the defect was invisible to anything but a spec that
evaluates the source.

⛔ The local internal/vm run is VOID, and CI is the check

internal/compiler passes in 0.2s and internal/aot in 0.18s. The
internal/vm run beside them did not fail an assertion — it hit
panic: test timed out after 10m0s at 604s, where the same suite took 297s and
434s earlier this morning, with the machine at load 37.5 / 74.3 / 76.5 on 16
cores (around forty peer sessions are building across this fleet right now).

Above roughly 8 that machine stops measuring anything, so this is an artifact and
I am claiming neither a regression nor the absence of one from it. The three
test lanes here, on clean runners, are the verdict; if one of them reds, the
cause is in this diff and not in the load.

⛔ Not one row of the new test was run against a reference interpreter

The Ruby on this machine is 2.6.10 and numbered parameters arrived in 2.7, so
the local oracle cannot answer any of these. The authority is the spec file,
which pins all eight verbatim, and MRI's rule that the grammar reserves the
names. The test says so in its own comment rather than borrowing the
"verified against MRI 4.0.5" line its neighbour in the same file has earned.

Tests: internal/compiler/numbered_param_ordinary_test.go — the eight shapes,
three more written-list kinds (_9, a block parameter, a keyword), thirteen
controls, and a second test asserting the two refusals stay DISTINGUISHABLE:
binding _1 is "reserved for numbered parameters" and reading it beside an
ordinary parameter is "ordinary parameter is defined". They are different
sentences in MRI and the specs match them separately, so collapsing them into
one message would satisfy neither.

Refs #799

🤖 Generated with Claude Code

tannevaled and others added 2 commits November 1, 2026 09:48
The last of the five defects in #799 that this repository can fix — it fails at
3.4.1 and at 4.0.9 alike — and the one that cost nothing at all before it was
looked for:

```console
$ rbgo -e 'proc { |x| _1 }'   # on main
$                             # no error, no warning, nothing
```

MRI rejects that at parse time. A block cannot have both a written parameter
list and a numbered parameter, because the grammar reserves `_1`..`_9` for the
numbered-parameter mechanism, so the two would be the same slot under two names.

`language/numbered_parameters_spec.rb:49` pins eight shapes in one example, and
the pairing is the interesting part:

```ruby
-> { eval("-> () { _1 }")         }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("-> (x) { _1 }")        }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("proc { || _1 }")       }.should.raise(SyntaxError, /ordinary parameter is defined/)
-> { eval("proc { |x| _1 }")      }.should.raise(SyntaxError, /ordinary parameter is defined/)
…
```

Each kind of block appears twice, once with an EMPTY written list and once with
a parameter in it: writing `||` or `()` at all is what declares the list, not
what it contains. A check that looked for a parameter would have passed four of
the eight.

## Where it goes, and why there

`blockParamsAreImplicit` already tells a written list from one the parser
synthesised out of `it` / `_1`..`_9` in the body — the trace being that a written
list pads `Defaults` to one entry per parameter and a synthesised one leaves it
empty. The builder now records that answer as `writtenParams`, set BEFORE the
body is compiled, because the body is where `_1` is met.

The refusal itself sits in `compileCall`, beside the `__LINE__` and
`block_given?` intrinsics and shaped like them — receiverless, blockless,
argumentless, bare name. Reaching that point means the name resolved to no
local, so it was about to become a receiverless send; that is exactly the
position in which MRI has already decided the program is ill-formed.

Two scopes are deliberately NOT covered, because MRI's answer there is a
different one and no spec in the gate asks for it:

- a `for` body (`forScope`), since `for` opens no Ruby scope;
- a method body and the top level, where `_1` stays a runtime NameError.

## Measured, with the control

`language/numbered_parameters_spec.rb`, pinned corpus 87b1631, shim at 3.4.1:

| binary | result |
|---|---|
| `main` | `pass=16 fail=1` |
| this branch | **`pass=17 fail=0`** |

The file is clean. The eight shapes, probed directly:

| | `main` | this branch |
|---|---|---|
| `-> () { _1 }` … `lambda { \|x\| _1 }` (6 forms) | **no error at all** | SyntaxError |
| `['a'].map { \|\| _1 }`, `['a'].map { \|x\| _1 }` | NameError, at CALL time | SyntaxError |
| `proc { _1 }.call(7)` | `7` | `7` |
| `proc { _1 + _2 }.call(3, 4)` | `7` | `7` |
| `proc { \|x\| _10 }` | fine | fine |

The two rows that said NameError are worth keeping in view: on `main` only the
shapes that were actually CALLED failed, and the four that were merely defined
raised nothing — so the defect was invisible to anything but a spec that
evaluates the source.

## ⛔ Not one row of the new test was run against a reference interpreter

The Ruby on this machine is 2.6.10 and numbered parameters arrived in 2.7, so
the local oracle cannot answer any of these. The authority is the spec file,
which pins all eight verbatim, and MRI's rule that the grammar reserves the
names. The test says so in its own comment rather than borrowing the
"verified against MRI 4.0.5" line its neighbour in the same file has earned.

Tests: `internal/compiler/numbered_param_ordinary_test.go` — the eight shapes,
three more written-list kinds (`_9`, a block parameter, a keyword), thirteen
controls, and a second test asserting the two refusals stay DISTINGUISHABLE:
binding `_1` is "reserved for numbered parameters" and reading it beside an
ordinary parameter is "ordinary parameter is defined". They are different
sentences in MRI and the specs match them separately, so collapsing them into
one message would satisfy neither.

Refs #799

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@tannevaled

Copy link
Copy Markdown
Contributor Author

The red was a known flake, not this change

test (windows-latest) was the only failing lane; ubuntu and macOS passed, and
go vet ./... and go build ./... passed on windows too. The failing step is
"Test + 100% coverage gate", which since #542 does not enforce coverage on
windows — it prints "coverage ratchet not enforced on windows (POSIX-only paths
unreachable)"
and only runs the -race suite. So the step failing there means a
real test failure, and the log names it:

--- FAIL: TestEtcd (32.06s)
    --- FAIL: TestEtcd/LockAndMaintenance (5.32s)
FAIL	github.com/go-embedded-ruby/ruby/internal/vm	2222.562s

TestEtcd is an embedded-etcd integration test with a port/serve dependency and
a known history of flaking on that lane. Nothing in this branch touches it: the
diff is internal/compiler only — one builder field, one refusal in
compileCall, and a test file.

gh run rerun refuses this run ("cannot be rerun"), so instead the branch now
carries a merge of current main — which re-triggers CI and is worth doing
on its own merits, since main has moved (#856 landed). internal/compiler and
internal/aot pass on the merged tree.

One note for whoever reads the job data rather than the log: that job reports
completed_at 09:34:56 before started_at 09:52:22, which is why
gh pr checks printed a duration of 0. A zero-second lane looks like one that
never started; this one ran for 37 minutes.

🤖 Generated with Claude Code

@tannevaled
tannevaled merged commit 0b3e8cc into main Nov 1, 2026
26 checks passed
@tannevaled
tannevaled deleted the numbered-param-ordinary branch November 1, 2026 11:44
tannevaled added a commit that referenced this pull request Nov 1, 2026
The macOS lane went red. Its log was unreadable — `gh api .../jobs/<id>/logs`
answered 403 twice while `gh api rate_limit` reported **4707 of 5000 core calls
remaining**, so that endpoint has its own burst limit and the hourly budget was
never the problem. Diagnosed from the diff instead, and confirmed locally with
the same instrument the lane uses.

Taking the gate off the hot path moved the `'#'` guard from

```go
if !vm.warningEnabled() || strings.HasPrefix(name, "#") {   // before
```

to a statement of its own behind the `uninitialized` flag. Before, every global
read with `$VERBOSE` unset executed it; after, it is reachable only when a name
is both never-assigned and starts with `'#'` — which is `bytecode.ErrinfoCell`
("#$!") and nothing else.

⛔ And no Ruby program can reach it through the interpreter: vm.go's getgvar arm
intercepts that name **before** calling `gvarGet`. The only production caller is
AOT-compiled code, where the two tiers emit `vm.gvarGet` without the
interception. So running a `rescue` does not exercise the guard, and the
per-function coverage ratchet correctly noticed that nothing did.

A whitebox test calls `gvarGet(bytecode.ErrinfoCell)` directly, which is the
honest way to cover a path whose only caller is generated code. It sets
`$VERBOSE` to true first — rb_warning's gate — so a missing guard would produce
a line rather than being masked by the gate, and it plants the positive beside
it: the same call with an ordinary never-assigned name **must** warn, or the
silence would prove only that the gate is shut.

`gvarGet` measures 100.0% under `go tool cover -func` from those three tests
alone.

`main` is merged in, since #861 has landed.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
tannevaled added a commit that referenced this pull request Nov 2, 2026
…862)

* compiler: assigning a constant inside a method body is a SyntaxError

```console
$ rbgo -e 'def d; FOO = 1; end'   # on main
$                                 # nothing
```

MRI refuses that at parse time with `dynamic constant assignment`, and has for
many versions. rbgo refused no form of it.

## It has its own spec, in a file nobody had connected to this

`language/constants_spec.rb` fails on `Assignment dynamic assignment | raises
SyntaxError` — measured `pass=88 fail=8` → **`pass=89 fail=7`** with this
change, a gain this branch was not aiming at.

I reached the defect sideways, from #799's fifth item, which I had written down
as *"a non-ASCII upcased character must be rejected as a constant identifier"*.
That title points at the lexer. The spec behind it is

```ruby
-> { eval("def test\n  ἍBB = 1\nend") }.should.raise(SyntaxError, /dynamic constant assignment/)
```

so what it needs is this check, plus a lexer that classifies `ἍBB` as a
constant at all (go-ruby-parser#57 — rbgo's `defined?(ἍDD)` is nil where MRI
says "constant"). Running the simplest version of the example —
`def t; FOO_X = 1; end`, three words shorter — is what showed the real defect
was ASCII-wide and in the compiler. Corrected on #799.

## The boundary is syntactic, and two measured rows prove it

| | MRI | rbgo before |
|---|---|---|
| `def d; Class.new { FOO = 1 }; end` | refused | — |
| `class C; define_method(:m) { FOO = 1 }; end` | **allowed** | — |

Both blocks end up running as a method body, so reasoning from "will this run
as a method?" gets both backwards. The rule is about an enclosing `def`
**keyword**, reached through any number of blocks and stopped by a class or
module body — which is exactly what `inDefBody` walks.

The other row I expected to go the other way: a **scoped** assignment,
`C::FOO = 1`, is refused inside a `def` too. Naming the namespace explicitly
settles where the constant would go, and upstream refuses it anyway.

## Measured: eighteen shapes, eighteen agreeing

Against `/usr/bin/ruby`. The rule predates that interpreter by many versions, so
unlike the numbered-parameter check in #861 this oracle is a valid one here.

Refused by both, after the change — plain, through one/two/three blocks, a
lambda, `Class.new`, `instance_eval`, scoped, `::FOO`, `||=`, `+=`, masgn in
either position, nested masgn, `def self.m`, `class << self` + `def`, `def o.m`,
a nested `def`, a `rescue` clause, an `ensure` clause, and a default argument.

Allowed by both — the top level, a class body, a module body, a nested class
body, a block at the top level, a block in a class body, a `class << self`
body, `define_method`, `module_eval`, and, inside a `def`, a local, an ivar, a
class variable, a global, and READING a constant.

One row deliberately still diverges: `def d; class I; end; end` is
`class definition in method body` upstream and still compiles here. Different
message, different construct, different check — filed as #860, and the
`inDefBody` predicate this adds is what it will use.

## Blast radius

No valid Ruby does this, so our own Ruby should be clean — and it is: the
prelude, stdlib and gem tests pass, which includes regenerating the frozen
prelude. `language/def_spec.rb` is unchanged at `pass=63 fail=8 error=1`.

⛔ The full `internal/vm` suite was NOT run to completion for this branch. The
machine is at load 37–76 on 16 cores (around forty peer sessions are building
across this fleet), and the run on the previous branch hit
`panic: test timed out after 10m0s` at 604s where the same suite took 297s
earlier this morning. Above roughly 8 this machine stops measuring, so I am
claiming nothing from it either way; the `test` lanes and the ruby/spec ratchet
here are the verdict.

Tests: `internal/compiler/dynamic_constant_assignment_test.go` — the refusals,
the fifteen controls, a planted positive (every control row passes with the
check deleted), a same-name/two-scopes pair proving the predicate reads the
SCOPE rather than the name, and `inDefBody` pinned directly through sources
whose outcome depends only on the walk.

Refs #799

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* aot: the setconst codegen case was testing invalid Ruby

The three `arch` lanes failed on this branch, and the log names one case:

```
--- FAIL: TestCompileSupported/setconst
    codegen_test.go:84: compile "def m\n  X = 1\n  X\nend\nm":
        compile error: dynamic constant assignment
```

That fixture is **not valid Ruby**. MRI refuses a constant assignment inside a
method body, and this branch makes rbgo refuse it too — so the fixture was
exercising the `OpSetConst` arm through a source no Ruby program can contain.

## What it was hiding

A method's ISeq can never legally hold `OpSetConst`, because upstream forbids
the assignment in every form inside a `def` — including the scoped one,
`C::X = 1`. Where the opcode does reach a child ISeq is a **block** whose
lexical chain does not cross a `def`, or a class/module body. `methodISeq` takes
`Children[0]` whatever kind it is, so a top-level block serves directly:

```go
{"setconst", "[1].each { X = 1 }\nX", "block in <main>", "vm.consts["},
```

`[1].each { X = 1 }` at the top level is accepted by MRI — measured, alongside
the eight refusals in the commit below.

So the fixture change is not a workaround for the new refusal. The old source
could only ever have described a program the language does not permit, and the
question of which ISeq kinds legally carry `OpSetConst` was invisible while it
stood.

## ⛔ Why CI found this and I did not

I ran `./internal/compiler/` and `./internal/vm/` for this branch, plus the
prelude/stdlib/gem tests — and never `./internal/aot/`. A compiler change
reaches every package that compiles Ruby, so the suite to run was `./...`.

`go test ./... -count=1` now passes on this branch: exit 0, no FAIL lines, with
the full `internal/vm` suite included (which also clears the only local
measurement I could not complete when this PR was opened — the machine had been
at load 37–76 and the suite had timed out at 604s).

`main` is merged in as well, since it has moved (#856 landed).

Refs #799

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* compiler: inDefBody's trailing return was unreachable

The macOS lane failed, and not on a percentage:

```
-- NEWLY BELOW 100% (1) --
   internal/compiler/compiler.go inDefBody  83.3% (was fully covered)
##[error]1 function(s) dropped below 100% -- named above
```

Five statements of six. The missing one was the `return false` **after** the
loop, and it is not missing a test — it cannot have one. Every builder chain is
rooted at `<main>`, `<class:…>`, `<module:…>`, `<singleton class>` or
`<binding>`, and none of those is a block, so the walk always leaves through the
`!b.isBlock` arm and never by exhausting `b != nil`.

So the arm returns through a `break` and the function ends with the single
reachable `return false`. `inDefBody` and `refuseDynamicConstAssign` both
measure 100.0% under `go tool cover -func`, the same instrument the lane uses.

⛔ Worth noting about this gate, because my own notes had it wrong: the
percentage is **reported and not gated** any more — `coverage: 99.9% (reported,
NOT gated -- see below)` — and `scripts/coverage/cmd/covratchet` gates
per-function against `scripts/coverage/BELOW100.<lane>` instead. So a POSIX lane
failing here is no longer the rounding knife-edge it used to be; it names a
function, and the two "further below than recorded" rows it also printed
(`bootstrap` 99.6% and `registerNetHTTP` 92.3%) are pre-existing and ungated.
That is a strictly better gate than the one that had to be argued with.

The comment now says why the return cannot be covered, which is cheaper than
the next person rediscovering it.

Refs #799

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
tannevaled added a commit that referenced this pull request Nov 2, 2026
* vm: take the uninitialized-global gate off the hot path

#856 put `vm.globals["$VERBOSE"]` in front of **every** Ruby-level global read,
in the interpreter and in both AOT tiers:

```go
if !vm.warningEnabled() || strings.HasPrefix(name, "#") {
	return vm.gvar(name)
}
```

The comment I shipped with it said "one map lookup and false in any ordinary
run". That was true and beside the point. `getgvar` is a hot instruction, the
lookup was unconditional, and a cost on a hot path is not excused by being
small — particularly one that buys nothing on the reads that have a value.

## The reorder

Every test now sits behind `uninitialized`, the flag `gvarResolve` already
returns:

```go
v, uninitialized := vm.gvarResolve(name)
if !uninitialized {
	return v
}
```

The resolution happens exactly once, as it did before #856 existed, and a read
that found a value returns through the same path it always did. `$stdout`,
`$LOAD_PATH`, `$~`, `$1` and every assigned user global now pay **nothing** for
the warning — not the map lookup, not the `strings.HasPrefix`, not the table
lookups.

Nothing is deferred that could matter: `uninitialized` can only be true for a
name that resolved to no special, no match-data slot and no entry in
`vm.globals`, which is both rare and already an anomaly in the program being
run.

## ⛔ Why this is a reorder and not a benchmark

The machine is at 1-minute load 8.9 and 5-minute load 16 — around twenty-five
peer sessions build across this fleet — and above roughly 8 it cannot time
anything honestly. So the choice was between a number I could not defend and a
structure that makes the number unnecessary. Removing a cost beats measuring it.

## ⛔ And the reorder found dead code, which the coverage gate would have caught

Deferring the tests made one of them unreachable:

```go
if mriPredefinedGvars[name] || englishAlias[name] != "" {
```

Probed, by walking both tables through `gvarResolve`: **0 of englishAlias's 9
keys** resolve uninitialised, while **9 of the predefined table's 42** do — and
those nine are exactly `$,` `$-F` `$-i` `$.` `$;` `$?` `$FILENAME` `$\` `$_`,
the names this VM leaves on the ordinary user-global path.

So the English clause is gone, and the invariant that makes it unnecessary is
written down instead: `englishAlias` lists only names whose target this VM
models, so `specialGvar` or the match-data resolver answers every one of them
and none can arrive there. An alias added for an unmodelled target would warn —
the comment says so, and says what to do about it.

## Measured

`cmd/oracle` on the globals probe, this branch against the reference:
`stderr 810 vs 810 bytes`, three warnings each, same names and same message
text. The only differences are the two the probes' README records as expected —
2.6.10's backtick quote style and its one-line lag in a warning's position.

Behaviour is unchanged, which is the point: the warning tests and the
predefined-table walk from #856 pass untouched.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* vm: cover the errinfo-cell guard, which only generated code reaches

The macOS lane went red. Its log was unreadable — `gh api .../jobs/<id>/logs`
answered 403 twice while `gh api rate_limit` reported **4707 of 5000 core calls
remaining**, so that endpoint has its own burst limit and the hourly budget was
never the problem. Diagnosed from the diff instead, and confirmed locally with
the same instrument the lane uses.

Taking the gate off the hot path moved the `'#'` guard from

```go
if !vm.warningEnabled() || strings.HasPrefix(name, "#") {   // before
```

to a statement of its own behind the `uninitialized` flag. Before, every global
read with `$VERBOSE` unset executed it; after, it is reachable only when a name
is both never-assigned and starts with `'#'` — which is `bytecode.ErrinfoCell`
("#$!") and nothing else.

⛔ And no Ruby program can reach it through the interpreter: vm.go's getgvar arm
intercepts that name **before** calling `gvarGet`. The only production caller is
AOT-compiled code, where the two tiers emit `vm.gvarGet` without the
interception. So running a `rescue` does not exercise the guard, and the
per-function coverage ratchet correctly noticed that nothing did.

A whitebox test calls `gvarGet(bytecode.ErrinfoCell)` directly, which is the
honest way to cover a path whose only caller is generated code. It sets
`$VERBOSE` to true first — rb_warning's gate — so a missing guard would produce
a line rather than being masked by the gate, and it plants the positive beside
it: the same call with an ordinary never-assigned name **must** warn, or the
silence would prove only that the gate is shut.

`gvarGet` measures 100.0% under `go tool cover -func` from those three tests
alone.

`main` is merged in, since #861 has landed.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant