From 45eb3b154891d049b83231ac01722e08f0cdb2b1 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 5 May 2026 18:41:32 -0700 Subject: [PATCH 1/3] chore(plan): block A8 on A16 _ENV semantics Triage of events.lua located the failing assertion at line 15, which depends on Lua 5.3 _ENV semantics that this VM doesn't implement: globals live in a flat state.globals map and :get_global / :set_global opcodes bypass any user-set _ENV. The entire events.lua file is gated on _ENV behaviour (the swap is on line 10, before any metamethod tests run), so there is no smaller metamethod fix that lets A8 proceed. Mark A8 as blocked with a Discoveries note, and create new plan A16 to implement _ENV semantics across scope.ex, codegen.ex, executor.ex, and stdlib.ex. A16 unlocks events.lua plus attrib.lua, locals.lua, and errors.lua. Plan: A8, A16 --- .agents/plans/A16-env-semantics.md | 140 +++++++++++++++++++++++++++++ .agents/plans/A8-events-suite.md | 54 ++++++++++- 2 files changed, 192 insertions(+), 2 deletions(-) create mode 100644 .agents/plans/A16-env-semantics.md diff --git a/.agents/plans/A16-env-semantics.md b/.agents/plans/A16-env-semantics.md new file mode 100644 index 00000000..1d444b2d --- /dev/null +++ b/.agents/plans/A16-env-semantics.md @@ -0,0 +1,140 @@ +--- +id: A16 +title: Implement Lua 5.3 _ENV semantics for global access +issue: 186 +pr: null +branch: feat/env-semantics +base: main +status: ready +direction: A +unlocks: + - events.lua + - attrib.lua + - locals.lua + - errors.lua +--- + +## Goal + +Make user-level reassignment of `_ENV` actually redirect subsequent +"global" reads and writes through the named environment table, matching +Lua 5.3 semantics. Today `_ENV` is a one-time alias of `_G` set at +stdlib install; reassigning it has no effect because globals live in a +flat `state.globals` Elixir map and `:get_global` / `:set_global` +opcodes bypass any user-controlled environment. + +## Background + +In Lua 5.3, every "global" name reference is syntactic sugar for +`_ENV.name`, where `_ENV` is an implicit upvalue in every function. A +user can swap their environment with `_ENV = setmetatable({}, ...)` or +`local _ENV = ...` and all subsequent free-name accesses go through +that table (and its metamethods). + +This implementation does not do any of that. Free-name resolution in +`scope.ex` produces a `{:global, name}` tag, codegen emits dedicated +`:get_global` / `:set_global` opcodes, and the executor reads/writes a +flat `state.globals` map that no Lua-level table participates in. +`_G` is a metatable-backed proxy whose `data` map is empty and whose +`__index`/`__newindex` route to `state.globals`. `_ENV` is set once to +that same proxy and never consulted again. + +This blocks `events.lua` immediately (line 10 swaps `_ENV`) and is +needed by other suite files. + +## Out of scope + +- Removing `state.globals` entirely. The host API still needs a way to + set globals from Elixir; we may keep the field but reframe it as the + raw data of the top-level `_ENV` table. +- Changing the `_G` proxy table's user-visible identity. Code that does + `_G == _ENV` at the top level must still hold. +- Performance regressions: the existing `:get_global` / `:set_global` + fast path may be retained when `_ENV` is provably the unmodified + globals table, or replaced in a follow-up perf plan. + +## Success criteria + +- [ ] `mix test` passes (no regressions; should remain at or above + current count). +- [ ] New unit tests in `test/lua/vm/env_semantics_test.exs`: + - Reassigning `_ENV` to a fresh table redirects subsequent global + writes (does not touch original `_G`). + - Reading a free variable after `_ENV` swap consults the new + environment's `__index` metamethod. + - `local _ENV = ...` inside a function redirects only that function's + free-name access. + - Setting a key to `nil` in `_ENV`-with-`__index` chain falls through + to the chained table on next read. + - Top-level `_G == _ENV` still holds before any user reassignment. +- [ ] `events.lua` progresses past line 19 (the `_ENV`-dependent block). + Suite count: events.lua should flip to passing or surface a new, + different failure. +- [ ] No regression in `test/language/global_test.exs`. + +## Implementation notes + +Expected files to touch: + +- `lib/lua/compiler/scope.ex` — at lines 308-328 (and 217-227 for + `FuncDecl`), the `:not_found` branch should resolve as if the source + had been `_ENV.name`. Either: + - Rewrite at scope-resolution time to a structured form + `{:env_field, name}` and let codegen handle it; or + - Resolve `_ENV` itself by the normal upvalue/local/global rules and + bake that into the AST node. +- `lib/lua/compiler/codegen.ex` — at the four `{:global, name}` handling + sites (627-633, 721-725, 929-933, 1290-1296), emit: + - Lookup `_ENV` (as upvalue or local), then + - `:get_field` / `:set_field` on it. + - These ops already invoke `__index` / `__newindex` correctly. +- `lib/lua/vm/state.ex` and `lib/lua/vm/executor.ex` — decide on the + globals storage strategy. Options: + 1. Keep `state.globals` as the storage and make the top-level + `_ENV`/`_G` table's `data` map *be* `state.globals` (or keep them + in sync). Pro: minimal host-API churn. Con: ongoing sync risk. + 2. Move globals into the `_G` table's `data` map proper, reframe + `state.globals` as a thin pointer to it, update `set_global/3` to + write into that table. +- `lib/lua/vm/stdlib.ex` — at `install_global_g/1` (lines 67-115): + - Stop using a metatable proxy to fake the globals table; instead + make the storage live in the table's `data`. + - Ensure `_ENV` is installed as an upvalue/local on the main chunk's + function, not just a global named `"_ENV"`. +- Main-chunk execution path — wherever `Lua.VM.execute` runs the top + chunk, bind the `_ENV` upvalue to the globals table. + +The precise design (scope option 1 vs 2 above) should be chosen during +implementation; whichever path is simpler to keep +`test/language/global_test.exs` passing should win. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 +mix test test/lua/vm/env_semantics_test.exs +mix test test/language/global_test.exs +``` + +## Risks + +- Substantial change touching scope resolution, codegen for every free + variable, and global storage. Risk of accidentally breaking unrelated + tests is high — keep a tight verify loop. +- Host API: external callers using `Lua.set_global/3` / + `Lua.get_global/2` must continue to work. Ensure those go through the + new globals storage. +- Performance: replacing direct `:get_global`/`:set_global` with field + access on `_ENV` is one extra indirection per global access. If a + measurable regression appears, retain the fast-path opcodes for the + case where `_ENV` is provably the unmodified globals table. +- Closures: nested functions inherit `_ENV` like any other upvalue. + Closure capture must thread `_ENV` correctly even across multiple + levels. + +## Discoveries + +(populated during implementation) diff --git a/.agents/plans/A8-events-suite.md b/.agents/plans/A8-events-suite.md index 415031ac..5f6775a8 100644 --- a/.agents/plans/A8-events-suite.md +++ b/.agents/plans/A8-events-suite.md @@ -5,7 +5,7 @@ issue: 169 pr: null branch: fix/events-suite base: main -status: ready +status: blocked direction: A unlocks: - events.lua @@ -65,4 +65,54 @@ mix test test/lua/vm/metatable_test.exs ## Discoveries -(populated during implementation) +### Triage outcome: blocked on `_ENV` semantics, not metamethods + +Triage of `events.lua` (per `triage-suite-failure` skill) located the +first failing assertion at line 15: + +```lua +assert(X == 30 and _G.X == 20) +``` + +Reduced to a 5-line repro: + +```lua +X = 20 +_ENV = setmetatable({}, {__index=_G}) +X = X + 10 +-- expected: _G.X == 20 (write went to new _ENV); actual: write goes to _G +``` + +Investigation showed the failure is **not** a metamethod bug. It is a +gap in Lua 5.3 `_ENV` semantics: + +- Globals live in a flat `state.globals` Elixir map + (`lib/lua/vm/state.ex:8`), not in any Lua table. +- The compiler emits dedicated `:get_global` / `:set_global` opcodes + that hit `state.globals` directly + (`lib/lua/vm/executor.ex:264-278`), bypassing any user-set `_ENV`. +- `_G` is a metatable proxy whose `__index`/`__newindex` route to + `state.globals` (`lib/lua/vm/stdlib.ex:67-115`). Its `data` map is + empty, which is why `rawget(_G, "X")` returns `nil` even after + `X = 20`. +- `_ENV` is a one-time alias of `_G` set at stdlib install + (`lib/lua/vm/stdlib.ex:105-106`). Reassigning it has no effect on + subsequent global access. + +Because `events.lua` swaps `_ENV` on line 10 before any metamethod +tests run, the entire file is gated on this. There is no smaller +metamethod-only fix that would let events.lua pass. + +### Follow-up + +- New plan: `.agents/plans/A16-env-semantics.md` (status: ready) — + implements proper `_ENV` semantics across `scope.ex`, `codegen.ex`, + `executor.ex`, and `stdlib.ex`. Unlocks events.lua plus attrib.lua, + locals.lua, errors.lua (also `_ENV`-dependent). +- Regression tests documenting the gap: + `test/lua/vm/env_semantics_test.exs` (currently `@tag :skip`, + un-skipped by A16). + +A8 is set to **blocked** on A16. Once A16 lands, A8 can be reopened to +triage any remaining metamethod-specific failures in events.lua (if +any surface beyond the `_ENV` block). From b07543b47eae53f1208ed7733c14939310daa26e Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 5 May 2026 18:41:38 -0700 Subject: [PATCH 2/3] test(vm): document Lua 5.3 _ENV semantics gap Add regression tests pinning the contract that _ENV reassignment should redirect global access. All four behavioural tests are tagged :skip until plan A16 implements proper _ENV semantics; un-skipping them is a success criterion of A16. One non-skipped test pins the top-level _G == _ENV identity that A16 must preserve. Discovered while triaging events.lua (plan A8). Suite count is unchanged; the 4 skipped tests document a known gap rather than a new failure. Plan: A8, A16 --- test/lua/vm/env_semantics_test.exs | 100 +++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) create mode 100644 test/lua/vm/env_semantics_test.exs diff --git a/test/lua/vm/env_semantics_test.exs b/test/lua/vm/env_semantics_test.exs new file mode 100644 index 00000000..f8ec6f97 --- /dev/null +++ b/test/lua/vm/env_semantics_test.exs @@ -0,0 +1,100 @@ +defmodule Lua.VM.EnvSemanticsTest do + use ExUnit.Case, async: true + + # Regression tests for Lua 5.3 _ENV semantics. + # + # In Lua 5.3, every "global" name reference is syntactic sugar for + # `_ENV.name`, where `_ENV` is an implicit upvalue in every function. A + # user may swap their environment with `_ENV = setmetatable({}, ...)` or + # `local _ENV = ...`, and all subsequent free-name accesses go through + # that table (and its metamethods). + # + # This implementation does not currently honour that. Free names are + # resolved as `{:global, name}` at compile time and the VM reads/writes a + # flat `state.globals` map directly via `:get_global` / `:set_global` + # opcodes that bypass any user-controlled `_ENV`. `_G` is a metatable + # proxy whose `__index`/`__newindex` route to `state.globals`; `_ENV` is + # a one-time alias of `_G` set at stdlib install. + # + # These tests are tagged `:skip`. They must pass once Plan A16 + # (.agents/plans/A16-env-semantics.md) is implemented. Removing the + # skip tags is a success criterion of A16. + # + # Discovered while triaging suite test events.lua (Plan A8). The first + # failing assertion in events.lua is line 15: + # assert(X == 30 and _G.X == 20) + # which depends on this behaviour. + + setup do + %{lua: Lua.new(sandboxed: [])} + end + + describe "_ENV reassignment redirects global access" do + @tag :skip + test "global write after _ENV swap goes to new env, not _G", %{lua: lua} do + code = """ + X = 20 + _ENV = setmetatable({}, {__index=_G}) + X = X + 10 + return _G.X, _ENV.X + """ + + # _G.X must remain 20 (write went to new _ENV), _ENV.X is 30 + assert {[20, 30], _} = Lua.eval!(lua, code) + end + + @tag :skip + test "setting key to nil in new _ENV falls through __index", %{lua: lua} do + code = """ + B = 30 + _ENV = setmetatable({}, {__index=_G}) + B = false + local v1 = B + B = nil + local v2 = B + return v1, v2 + """ + + # After B = false, _ENV.B = false (no fallthrough). + # After B = nil, _ENV has no B key, so __index falls through to _G.B = 30. + assert {[false, 30], _} = Lua.eval!(lua, code) + end + + @tag :skip + test "free name read consults new _ENV's __index", %{lua: lua} do + code = """ + _ENV = setmetatable({}, {__index = function(_, k) return "from-meta-" .. k end}) + return foo + """ + + assert {["from-meta-foo"], _} = Lua.eval!(lua, code) + end + end + + describe "local _ENV scoping" do + @tag :skip + test "local _ENV inside a function redirects only that function", %{lua: lua} do + code = """ + X = 1 + local function f() + local _ENV = setmetatable({}, {__index=_G}) + X = 99 + return X + end + local inside = f() + return inside, X + """ + + # Inside f, the local _ENV captures X = 99. Outer X stays 1. + assert {[99, 1], _} = Lua.eval!(lua, code) + end + end + + describe "_G / _ENV identity at top level" do + test "_G == _ENV before any user reassignment", %{lua: lua} do + # This already passes — included to pin the contract that the A16 + # implementation must not break. + assert {[true], _} = Lua.eval!(lua, "return _G == _ENV") + end + end +end From e57f7d50918207bd77399fae1f969f4054fe9f87 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Wed, 6 May 2026 03:58:11 -0700 Subject: [PATCH 3/3] chore(A8): record triage PR #187 Plan: A8 --- .agents/plans/A8-events-suite.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/plans/A8-events-suite.md b/.agents/plans/A8-events-suite.md index 5f6775a8..b821ac4d 100644 --- a/.agents/plans/A8-events-suite.md +++ b/.agents/plans/A8-events-suite.md @@ -2,7 +2,7 @@ id: A8 title: Fix events.lua metamethod assertion issue: 169 -pr: null +pr: 187 branch: fix/events-suite base: main status: blocked