Repository navigation
compiler: assigning a constant inside a method body is a SyntaxError - #862
Conversation
```console $ rbgo -e 'def d; FOO = 1; end' # on main $ # nothing ``` MRI refuses that at parse time with `dynamic constant assignment`, and has for many versions. rbgo refused no form of it. ## It has its own spec, in a file nobody had connected to this `language/constants_spec.rb` fails on `Assignment dynamic assignment | raises SyntaxError` — measured `pass=88 fail=8` → **`pass=89 fail=7`** with this change, a gain this branch was not aiming at. I reached the defect sideways, from #799's fifth item, which I had written down as *"a non-ASCII upcased character must be rejected as a constant identifier"*. That title points at the lexer. The spec behind it is ```ruby -> { eval("def test\n ἍBB = 1\nend") }.should.raise(SyntaxError, /dynamic constant assignment/) ``` so what it needs is this check, plus a lexer that classifies `ἍBB` as a constant at all (go-ruby-parser#57 — rbgo's `defined?(ἍDD)` is nil where MRI says "constant"). Running the simplest version of the example — `def t; FOO_X = 1; end`, three words shorter — is what showed the real defect was ASCII-wide and in the compiler. Corrected on #799. ## The boundary is syntactic, and two measured rows prove it | | MRI | rbgo before | |---|---|---| | `def d; Class.new { FOO = 1 }; end` | refused | — | | `class C; define_method(:m) { FOO = 1 }; end` | **allowed** | — | Both blocks end up running as a method body, so reasoning from "will this run as a method?" gets both backwards. The rule is about an enclosing `def` **keyword**, reached through any number of blocks and stopped by a class or module body — which is exactly what `inDefBody` walks. The other row I expected to go the other way: a **scoped** assignment, `C::FOO = 1`, is refused inside a `def` too. Naming the namespace explicitly settles where the constant would go, and upstream refuses it anyway. ## Measured: eighteen shapes, eighteen agreeing Against `/usr/bin/ruby`. The rule predates that interpreter by many versions, so unlike the numbered-parameter check in #861 this oracle is a valid one here. Refused by both, after the change — plain, through one/two/three blocks, a lambda, `Class.new`, `instance_eval`, scoped, `::FOO`, `||=`, `+=`, masgn in either position, nested masgn, `def self.m`, `class << self` + `def`, `def o.m`, a nested `def`, a `rescue` clause, an `ensure` clause, and a default argument. Allowed by both — the top level, a class body, a module body, a nested class body, a block at the top level, a block in a class body, a `class << self` body, `define_method`, `module_eval`, and, inside a `def`, a local, an ivar, a class variable, a global, and READING a constant. One row deliberately still diverges: `def d; class I; end; end` is `class definition in method body` upstream and still compiles here. Different message, different construct, different check — filed as #860, and the `inDefBody` predicate this adds is what it will use. ## Blast radius No valid Ruby does this, so our own Ruby should be clean — and it is: the prelude, stdlib and gem tests pass, which includes regenerating the frozen prelude. `language/def_spec.rb` is unchanged at `pass=63 fail=8 error=1`. ⛔ The full `internal/vm` suite was NOT run to completion for this branch. The machine is at load 37–76 on 16 cores (around forty peer sessions are building across this fleet), and the run on the previous branch hit `panic: test timed out after 10m0s` at 604s where the same suite took 297s earlier this morning. Above roughly 8 this machine stops measuring, so I am claiming nothing from it either way; the `test` lanes and the ruby/spec ratchet here are the verdict. Tests: `internal/compiler/dynamic_constant_assignment_test.go` — the refusals, the fifteen controls, a planted positive (every control row passes with the check deleted), a same-name/two-scopes pair proving the predicate reads the SCOPE rather than the name, and `inDefBody` pinned directly through sources whose outcome depends only on the walk. Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The three `arch` lanes failed on this branch, and the log names one case:
```
--- FAIL: TestCompileSupported/setconst
codegen_test.go:84: compile "def m\n X = 1\n X\nend\nm":
compile error: dynamic constant assignment
```
That fixture is **not valid Ruby**. MRI refuses a constant assignment inside a
method body, and this branch makes rbgo refuse it too — so the fixture was
exercising the `OpSetConst` arm through a source no Ruby program can contain.
## What it was hiding
A method's ISeq can never legally hold `OpSetConst`, because upstream forbids
the assignment in every form inside a `def` — including the scoped one,
`C::X = 1`. Where the opcode does reach a child ISeq is a **block** whose
lexical chain does not cross a `def`, or a class/module body. `methodISeq` takes
`Children[0]` whatever kind it is, so a top-level block serves directly:
```go
{"setconst", "[1].each { X = 1 }\nX", "block in <main>", "vm.consts["},
```
`[1].each { X = 1 }` at the top level is accepted by MRI — measured, alongside
the eight refusals in the commit below.
So the fixture change is not a workaround for the new refusal. The old source
could only ever have described a program the language does not permit, and the
question of which ISeq kinds legally carry `OpSetConst` was invisible while it
stood.
## ⛔ Why CI found this and I did not
I ran `./internal/compiler/` and `./internal/vm/` for this branch, plus the
prelude/stdlib/gem tests — and never `./internal/aot/`. A compiler change
reaches every package that compiles Ruby, so the suite to run was `./...`.
`go test ./... -count=1` now passes on this branch: exit 0, no FAIL lines, with
the full `internal/vm` suite included (which also clears the only local
measurement I could not complete when this PR was opened — the machine had been
at load 37–76 and the suite had timed out at 604s).
`main` is merged in as well, since it has moved (#856 landed).
Refs #799
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…gnment # Conflicts: # internal/aot/codegen_test.go
The three
|
Closes #57. ```ruby ἍBB = 1 defined?(ἍBB) # MRI: "constant" before: nil Object.const_defined?(:ἍBB) # MRI: true before: false ``` `LookupIdent` decided constant-ness with ```go if c := s[0]; c >= 'A' && c <= 'Z' { ``` which reads a **byte**. For any multibyte character `s[0]` is a UTF-8 lead byte of 0xC2 or more, never in `'A'..'Z'`, so every non-ASCII name was classified as a local variable: the assignment created a local and no constant existed afterwards. ## The rule is not the lexer's parse.y assumes `tCONSTANT` **provisionally** and downgrades it afterwards: ```c if (result == tCONSTANT && is_local_id(ident)) result = tIDENTIFIER; ``` (parse.y:10453 at v4.0.9), so the decision belongs to the ID system. It is made in `rb_sym_constant_char_p` (symbol.c:508-542), and it has **five** branches, not two: | first character | verdict | |---|---| | ASCII | `ISUPPER` | | non-ASCII, `rb_enc_isupper` | constant | | non-ASCII, `rb_enc_islower` | local | | titlecase letter, Unicode encoding | **constant** | | anything else — a caseless letter | **local** | Go's `unicode.IsUpper` is that isupper and `unicode.IsTitle` that titlecase property, so `IsConstStart` is those five lines. ## ⛔ The two branches that make "not lowercase" the wrong shortcut - a **titlecase** letter is a constant although `unicode.IsUpper` reports false for it: upstream tests the Onigmo `titlecaseletter` property explicitly, after isupper and islower have both said no; - a **caseless** letter — CJK, Hebrew, Arabic — starts a **local variable**. So neither clause can be written as "not lower", which is the obvious simplification and is wrong in both directions. Measured against `/usr/bin/ruby` through `Object.const_defined?(name.to_sym)`, which raises `NameError` for a name that is not a constant name — that is `rb_is_const_name` (symbol.c:1287), the same predicate parse.y consults: ``` ASCII upper A -> CONSTANT name Lt DZ digr Dž -> CONSTANT name ASCII lower a -> not a constant name Lt DZE digr Lj -> CONSTANT name ASCII under _ -> not a constant name Lo CJK 日 -> not a constant name Lu GREEK Ἅ -> CONSTANT name Lo HEBREW א -> not a constant name Lu CYRILLIC Д -> CONSTANT name Lo ARABIC ب -> not a constant name Ll GREEK mu μ -> not a constant name Lu TURKISH I İ -> CONSTANT name ``` ## Measured end to end This is one of two halves. ruby/spec's `language/variables_spec.rb` has ```ruby it 'parses a non-ASCII upcased character as a constant identifier' do -> { eval("def test\n ἍBB = 1\nend") }.should.raise(SyntaxError, /dynamic constant assignment/) end ``` which needs this classification **and** a `dynamic constant assignment` refusal in the consumer's compiler (go-embedded-ruby/ruby#862). Neither half passes the example alone. With both — an rbgo built from that branch with a local `replace` onto this one — the example disappears from the failure list, and the file's only remaining failures are the two belonging to another PR there. ## Tests `token/const_start_test.go` walks the measured table character by character, asserting `IsConstStart` and `LookupIdent` agree, plus the degenerate inputs (empty, invalid UTF-8) and a control that the ASCII path and the keyword lookup still work — a version of `IsConstStart` that always answered false would otherwise pass every negative row. `nonascii_constant_test.go` checks the parse-level consequence: `ἍBB = 1` is an `*ast.ConstAssign` and `μ = 1` an `*ast.Assign`, and a non-ASCII constant works in every position an ASCII one does — read, assign, op-assign, scope, scoped name, masgn target, class and module name, superclass, pattern, singleton definee, receiver and rescue class. README: the feature list now states the rule, since "constants" alone did not say which names are ones. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The macOS lane failed, and not on a percentage: ``` -- NEWLY BELOW 100% (1) -- internal/compiler/compiler.go inDefBody 83.3% (was fully covered) ##[error]1 function(s) dropped below 100% -- named above ``` Five statements of six. The missing one was the `return false` **after** the loop, and it is not missing a test — it cannot have one. Every builder chain is rooted at `<main>`, `<class:…>`, `<module:…>`, `<singleton class>` or `<binding>`, and none of those is a block, so the walk always leaves through the `!b.isBlock` arm and never by exhausting `b != nil`. So the arm returns through a `break` and the function ends with the single reachable `return false`. `inDefBody` and `refuseDynamicConstAssign` both measure 100.0% under `go tool cover -func`, the same instrument the lane uses. ⛔ Worth noting about this gate, because my own notes had it wrong: the percentage is **reported and not gated** any more — `coverage: 99.9% (reported, NOT gated -- see below)` — and `scripts/coverage/cmd/covratchet` gates per-function against `scripts/coverage/BELOW100.<lane>` instead. So a POSIX lane failing here is no longer the rounding knife-edge it used to be; it names a function, and the two "further below than recorded" rows it also printed (`bootstrap` 99.6% and `registerNetHTTP` 92.3%) are pre-existing and ungated. That is a strictly better gate than the one that had to be argued with. The comment now says why the return cannot be covered, which is cheaper than the next person rediscovering it. Refs #799 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
compiler: inDefBody's trailing return was unreachable The macOS lane failed, and not on a percentage: Five statements of six. The missing one was the So the arm returns through a ⛔ Worth noting about this gate, because my own notes had it wrong: the The comment now says why the return cannot be covered, which is cheaper than Refs #799 Co-authored-by: Claude Opus 5 noreply@anthropic.com |
…gnment # Conflicts: # internal/compiler/compiler.go
MRI refuses that at parse time with
dynamic constant assignment, and has formany versions. rbgo refused no form of it.
It has its own spec, in a file nobody had connected to this
language/constants_spec.rbfails onAssignment dynamic assignment | raises SyntaxError— measuredpass=88 fail=8→pass=89 fail=7with thischange, a gain this branch was not aiming at.
I reached the defect sideways, from #799's fifth item, which I had written down
as "a non-ASCII upcased character must be rejected as a constant identifier".
That title points at the lexer. The spec behind it is
so what it needs is this check, plus a lexer that classifies
ἍBBas aconstant at all (go-ruby-parser#57 — rbgo's
defined?(ἍDD)is nil where MRIsays "constant"). Running the simplest version of the example —
def t; FOO_X = 1; end, three words shorter — is what showed the real defectwas ASCII-wide and in the compiler. Corrected on #799.
The boundary is syntactic, and two measured rows prove it
def d; Class.new { FOO = 1 }; endclass C; define_method(:m) { FOO = 1 }; endBoth blocks end up running as a method body, so reasoning from "will this run
as a method?" gets both backwards. The rule is about an enclosing
defkeyword, reached through any number of blocks and stopped by a class or
module body — which is exactly what
inDefBodywalks.The other row I expected to go the other way: a scoped assignment,
C::FOO = 1, is refused inside adeftoo. Naming the namespace explicitlysettles where the constant would go, and upstream refuses it anyway.
Measured: eighteen shapes, eighteen agreeing
Against
/usr/bin/ruby. The rule predates that interpreter by many versions, sounlike the numbered-parameter check in #861 this oracle is a valid one here.
Refused by both, after the change — plain, through one/two/three blocks, a
lambda,
Class.new,instance_eval, scoped,::FOO,||=,+=, masgn ineither position, nested masgn,
def self.m,class << self+def,def o.m,a nested
def, arescueclause, anensureclause, and a default argument.Allowed by both — the top level, a class body, a module body, a nested class
body, a block at the top level, a block in a class body, a
class << selfbody,
define_method,module_eval, and, inside adef, a local, an ivar, aclass variable, a global, and READING a constant.
One row deliberately still diverges:
def d; class I; end; endisclass definition in method bodyupstream and still compiles here. Differentmessage, different construct, different check — filed as #860, and the
inDefBodypredicate this adds is what it will use.Blast radius
No valid Ruby does this, so our own Ruby should be clean — and it is: the
prelude, stdlib and gem tests pass, which includes regenerating the frozen
prelude.
language/def_spec.rbis unchanged atpass=63 fail=8 error=1.⛔ The full
internal/vmsuite was NOT run to completion for this branch. Themachine is at load 37–76 on 16 cores (around forty peer sessions are building
across this fleet), and the run on the previous branch hit
panic: test timed out after 10m0sat 604s where the same suite took 297searlier this morning. Above roughly 8 this machine stops measuring, so I am
claiming nothing from it either way; the
testlanes and the ruby/spec ratchethere are the verdict.
Tests:
internal/compiler/dynamic_constant_assignment_test.go— the refusals,the fifteen controls, a planted positive (every control row passes with the
check deleted), a same-name/two-scopes pair proving the predicate reads the
SCOPE rather than the name, and
inDefBodypinned directly through sourceswhose outcome depends only on the walk.
Refs #799
🤖 Generated with Claude Code