From 31aaf9971663b6a7e8d50767dfbf2113516ef499 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Sun, 31 May 2026 07:01:29 -0700 Subject: [PATCH 1/3] chore(A47): start plan --- .agents/plans/A47-open-upvalue-block-close.md | 219 ++++++++++++++++++ 1 file changed, 219 insertions(+) create mode 100644 .agents/plans/A47-open-upvalue-block-close.md diff --git a/.agents/plans/A47-open-upvalue-block-close.md b/.agents/plans/A47-open-upvalue-block-close.md new file mode 100644 index 00000000..eff86473 --- /dev/null +++ b/.agents/plans/A47-open-upvalue-block-close.md @@ -0,0 +1,219 @@ +--- +id: A47 +title: Close open upvalues at if/while/for/repeat block exit +issue: 276 +pr: null +branch: fix/open-upvalue-block-close +base: main +status: in-progress +direction: A +unlocks: + - calls.lua +--- + +# A47 — Close open upvalues at if/while/for/repeat block exit + +## Goal + +Fix a live VM bug where `state.open_upvalues` (keyed by register) is +never closed when an `if`/`while`/`for`/`repeat` block ends. When a +later sibling block reuses the same register for a new captured local, +the closure handler reuses the stale cell ref, so reads of the new local +(directly via `get_open_upvalue` or through any inner closure that +captures it) return the *previous* block's value. + +`do…end` was already fixed in PR #286 by emitting a +`:close_upvalues` instruction at block exit and sweeping +`open_upvalues` in the executor. The remaining block-scoping statements +(`if`, `while`, `for`-numeric, `for`-in, `repeat`) do not close their +body locals' cells, so the bug still surfaces — confirmed by reproducing +it with two sibling `if` blocks, which crashes with +`attempt to index a number value (upvalue 'a')`. This is the downstream +blocker for `calls.lua:65–69`, surfaced by the FuncDecl head-resolution +fix in PR #274. + +The fix extends the existing block-close mechanism to every block-scoping +statement, removing only the entries that went out of scope and never +touching `state.upvalue_cells` (existing closures hold valid cell refs +and resolve through it). + +## Out of scope + +- Option 2 in the issue (full per-block open-range tracking in scope + analysis à la PUC-Lua `OP_CLOSE`). We take option 1: emit a close at + block-scope end parametrised by the block's register watermark, reusing + the `do`-block machinery already on `main`. +- Any change to `state.upvalue_cells` semantics or to the closure + capture handler. +- The other `calls.lua` skip ranges (24–36, 135–137, 217–218, 219–401); + those have separate, unrelated causes already documented in + `test/lua53_skips.exs`. +- Rewriting the threshold-based sweep into an explicit register-set + parameter. The existing `close_open_upvalues_at_or_above/2` sweeps + entries at-or-above the block's pre-entry `next_register` watermark, + which is exactly the set of registers the block's locals occupied; the + registers below the watermark belong to enclosing scopes and are left + intact. This already satisfies the issue's "remove ONLY the block's + entries, never touch `upvalue_cells`" requirement. + +## Success criteria + +- [ ] The two-sibling-`do`-block repro from issue #276 passes (already + green on `main` via PR #286 — guard against regression). +- [ ] The two-sibling-`if`-block repro passes (currently crashes with + `attempt to index a number value (upvalue 'a')`). +- [ ] Sibling `while`, `for`-numeric, `for`-in, and `repeat` blocks that + declare a captured local on the same register no longer leak a + stale cell. +- [ ] A loop whose body captures a local and is re-entered across + iterations still observes correct per-iteration values (cells + persist within an iteration, close on iteration boundary and on + loop exit). +- [ ] `calls.lua:65..69` removed from `test/lua53_skips.exs`; the + `calls.lua` range list narrows accordingly (the next downstream + blocker, if any, is documented in the same file). +- [ ] Regression test added to `test/lua/vm/upvalue_test.exs`: two + sibling `do` blocks declaring captured locals on the same register, + plus the `if`/loop variants exercising the same register reuse. +- [ ] `mix test` passes; no regressions in `test/lua/vm/upvalue_test.exs` + or closure-style tests. +- [ ] `mix test --only lua53` passes with no suite regressions; + `calls.lua` passes further than before. + +## Implementation notes + +The close machinery already exists and is shared by both executors — +the work is extending emission to the non-`do` block types. + +**`lib/lua/compiler/scope.ex`** — `Statement.Do` resolution +(`resolve_statement/2`, ~line 359) stashes the pre-block register +watermark under `{:do_close_threshold, do_stmt}`. Add the same watermark +stash for the block-scoping statements that currently lack it: +- `Statement.If` (~line 180) — per `then_block`, each elseif block, and + the `else_block`. Note `if` blocks resolve through + `with_block_scope/2`; capture the watermark for each sub-block so + codegen can emit a close at the end of each branch body. +- `Statement.While` (~line 202) and `Statement.Repeat` (~line 207). +- `Statement.ForNum` (~line 220) and `Statement.ForIn` (~line 316). + These already save `saved_next_register`; stash a per-statement close + threshold keyed on the statement struct. + +Use distinct `var_map` keys per block type (mirroring +`{:do_close_threshold, do_stmt}`) so codegen can fetch them. + +**`lib/lua/compiler/codegen.ex`** — mirror the `Statement.Do` +generator (~line 733, which appends `Instruction.close_upvalues(threshold)` +to the block body): +- `Statement.If` (~line 453) and `gen_elseifs_and_else`/`gen_block` + callers (~line 853, 861): append a close to the end of each branch's + body instructions before they are wrapped in the `:test` instruction. +- `Statement.While` (~line 472) and `Statement.Repeat` (~line 485): + append the close to `body_instructions` so the body closes its locals + on each iteration boundary AND on loop exit (the close runs at the tail + of the body, before the loop re-test). +- `Statement.ForNum` (~line 499) and `Statement.ForIn` (~line 551): + append the close to `body_instructions`. + +For loops, the existing per-iteration `close_open_upvalues_at_or_above` +calls in the continuation handlers +(`lib/lua/vm/executor.ex:574,603`) key on the loop-variable register and +already sweep body-local cells at the iteration boundary; appending a +body-tail close handles the within-iteration block scope and the +final-iteration exit. Confirm with the loop repro that cells still +persist correctly within an iteration (a closure created mid-body and +called later in the same iteration must see the live value). + +**`lib/lua/vm/executor.ex`** — no new handler needed. The +`:close_upvalues` instruction handler (~line 708) and +`close_open_upvalues_at_or_above/2` (~line 2867, with its +`map_size == 0` fast path) already exist and do exactly the required +work: `Map.reject` entries at-or-above the threshold from +`open_upvalues`, leaving `upvalue_cells` untouched. Verify the emitted +thresholds land in this handler unchanged. + +**`lib/lua/vm/dispatcher.ex`** — the PC-dispatch executor is on the hot +path: `Executor` routes function calls through `Dispatcher.execute/4` +(`executor.ex:146,971`), so a closure created in a block may run under +the dispatcher. It already handles `@op_close_upvalues` (op 52, +~line 958) via `Executor.dispatcher_close_open_upvalues_at_or_above/2` +and has per-iteration loop closes (~line 830, 878). Confirm the new +`close_upvalues` instructions encode and decode correctly through +`lib/lua/compiler/bytecode.ex` (op 52 already wired at +`bytecode.ex:349`) and run under the dispatcher; add coverage if the +sibling-block repro routes through it. No new opcode is required. + +**`test/lua/vm/upvalue_test.exs`** — add a `describe` block covering +register reuse across sibling blocks. Include: two sibling `do` blocks +(guards the PR #286 fix), two sibling `if` blocks (the primary repro for +this issue), and `while`/`for`/`repeat` variants where both blocks +declare a captured local landing on the same register. Add at least one +loop case asserting per-iteration values are correct (cells do not leak +across iterations and a closure created in iteration N sees iteration +N's value). Do not reference the plan id in any moduledoc or comment; +describe the §3.4.10 contract being pinned. + +**`test/lua53_skips.exs`** — remove the `calls.lua` `lines: 65..69` +skip entry (the `:executor` / issue 276 one). Re-run `calls.lua` to +confirm where it next stops; if a new downstream blocker appears, +document it as a fresh skip entry with its own one-line reason. + +## Verification + +Run before any code change to snapshot the suite state, and again after: + +``` +mix format +mix compile --warnings-as-errors +mix test test/lua/vm/upvalue_test.exs +mix test +mix test --only lua53 +``` + +Manual repro (must crash before the fix, pass after): + +``` +do + local res = 1 + local function fact (n) + if n == 0 then return res else return n * fact(n - 1) end + end + assert(fact(5) == 120) +end +do + local a = {x = 100} + local function read_a() return a.x end + assert(read_a() == 100, "expected 100, got stale-cell value") +end +``` + +and the `if`/`while`/`for`/`repeat` variants of the same two-block +shape. Capture the `mix test --only lua53` pass-count delta and confirm +`calls.lua` advances past line 69 with no other suite file regressing. + +## Risks + +- **Closing cells at block end is a subtle change to executor + invariants.** Existing closures hold cell refs and resolve through + `state.upvalue_cells`, which must NOT be touched — only the + `open_upvalues` index map. The reused `close_open_upvalues_at_or_above/2` + already respects this; verify no new code path mutates + `upvalue_cells`. +- **Loops re-enter the block body across iterations.** Cells must + persist within an iteration (a closure created mid-body and called + later in the same iteration sees the live value) and close only at the + iteration boundary and loop exit. The body-tail close plus the existing + per-iteration sweep must not close a cell that the current iteration's + later statements still read. The loop regression test guards this. +- **`if` branches share register slots.** Both the `then` and `else` + branches reuse the same register watermark; closing at the end of each + branch body must not strand a cell that an enclosing scope still owns. + The threshold is the pre-branch `next_register`, so only branch-local + registers are swept. +- **Two executors must stay consistent.** `Executor` and `Dispatcher` + both run block code; an asymmetric fix would leave the bug live on one + path. Both already route `:close_upvalues` / op 52, so the risk is + emission coverage, not handler divergence — exercise both via the + closure-call repro. +- **Over-closing could regress unrelated upvalue tests.** The full + `upvalue_test.exs` and closure-style suites must stay green; run them + explicitly before the broader `mix test`. From 5ff1580484e6184712f0ce786a6723e86444e13c Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Sun, 31 May 2026 07:06:50 -0700 Subject: [PATCH 2/3] fix(vm): close open upvalues at if/while/for/repeat block exit state.open_upvalues is keyed by register and was only closed at do-block exit. When an if/while/for/repeat block ended and a later sibling block reused the same register for a new captured local, the closure handler reused the stale cell ref, so reads of the new local returned the previous block's value (crashing as "attempt to index a number value"). Extend the existing do-block close mechanism to every block-scoping statement: scope analysis stashes the pre-block next_register watermark and codegen appends a :close_upvalues at each branch/body tail. Loop bodies re-run the close on every iteration boundary and on loop exit, so cells persist within an iteration and close only at its tail. The executor sweep removes only open_upvalues entries at-or-above the watermark and never touches upvalue_cells. Plan: A47 Closes #276 --- lib/lua/compiler/bytecode.ex | 1 + lib/lua/compiler/codegen.ex | 33 ++- lib/lua/compiler/scope.ex | 31 ++- .../compiler/max_registers_invariant_test.exs | 5 + test/lua/vm/upvalue_test.exs | 196 ++++++++++++++++++ test/lua53_skips.exs | 7 - 6 files changed, 260 insertions(+), 13 deletions(-) diff --git a/lib/lua/compiler/bytecode.ex b/lib/lua/compiler/bytecode.ex index bf04b4fb..bbdcac53 100644 --- a/lib/lua/compiler/bytecode.ex +++ b/lib/lua/compiler/bytecode.ex @@ -427,4 +427,5 @@ defmodule Lua.Compiler.Bytecode do def op_while_loop, do: @op_while_loop def op_repeat_loop, do: @op_repeat_loop def op_generic_for, do: @op_generic_for + def op_close_upvalues, do: @op_close_upvalues end diff --git a/lib/lua/compiler/codegen.ex b/lib/lua/compiler/codegen.ex index 4454aa83..cd1ed6b7 100644 --- a/lib/lua/compiler/codegen.ex +++ b/lib/lua/compiler/codegen.ex @@ -459,6 +459,7 @@ defmodule Lua.Compiler.Codegen do # Generate code for the then block {then_instructions, ctx} = gen_block(then_block, ctx) + then_instructions = append_block_close(then_instructions, then_block, ctx) # Generate code for elseifs and else by building nested if-else {else_instructions, ctx} = gen_elseifs_and_else(elseifs, else_block, ctx) @@ -475,6 +476,7 @@ defmodule Lua.Compiler.Codegen do # Generate code for the body {body_instructions, ctx} = gen_block(body, ctx) + body_instructions = append_block_close(body_instructions, body, ctx) # Create while loop instruction loop_instruction = Instruction.while_loop(condition_instructions, cond_reg, body_instructions) @@ -489,6 +491,12 @@ defmodule Lua.Compiler.Codegen do # Generate code for the condition {condition_instructions, cond_reg, ctx} = gen_expr(condition, ctx) + # The repeat-until condition is part of the body's scope (Lua 5.3 §3.3.4), + # so it may read body locals; close the body's open-upvalue cells only + # after the condition has run — i.e. at the tail of the condition body, + # which executes on every iteration boundary and at loop exit. + condition_instructions = append_block_close(condition_instructions, body, ctx) + # Create repeat loop instruction loop_instruction = Instruction.repeat_loop(body_instructions, condition_instructions, cond_reg) @@ -536,6 +544,7 @@ defmodule Lua.Compiler.Codegen do # Generate body {body_instructions, ctx} = gen_block(body, ctx) + body_instructions = append_block_close(body_instructions, body, ctx) # Create numeric for instruction # The VM will handle: copying base to loop_var_reg, incrementing, checking limit @@ -618,6 +627,7 @@ defmodule Lua.Compiler.Codegen do # Generate body {body_instructions, ctx} = gen_block(body, ctx) + body_instructions = append_block_close(body_instructions, body, ctx) # Emit generic_for instruction with var_regs as a list of register indices loop_instruction = Instruction.generic_for(base, var_regs, body_instructions) @@ -761,6 +771,25 @@ defmodule Lua.Compiler.Codegen do # Stub for other statements defp gen_statement(_stmt, ctx), do: {[], ctx} + # Append a `:close_upvalues` to a control-flow block's instruction list so + # any open-upvalue cell over a register the block's locals occupied is + # detached when the block exits (Lua 5.3 §3.4.10). The threshold is the + # pre-block `next_register` watermark stashed by scope analysis under + # `{:block_close_threshold, block}`; registers below it belong to + # enclosing scopes and are left intact. Emitting unconditionally is cheap — + # the executor's close helper short-circuits when `open_upvalues` is empty. + # + # If/elseif/else branches, while/repeat bodies, and for bodies all key on + # the block AST node. Loop bodies re-run this close on every iteration + # boundary and on loop exit; cells therefore persist within an iteration + # and close only at its tail, which is exactly the §3.4.10 contract. + defp append_block_close(instructions, block, ctx) do + case Map.fetch(ctx.scope.var_map, {:block_close_threshold, block}) do + {:ok, threshold} -> instructions ++ [Instruction.close_upvalues(threshold)] + :error -> instructions + end + end + # Emit instructions that load the head value of a multi-name FuncDecl # (`function a.b.c(...)`) into a register, using the var_map entry # populated by scope analysis. Mirrors the four cases of `gen_expr` on @@ -850,7 +879,8 @@ defmodule Lua.Compiler.Codegen do defp gen_elseifs_and_else([], else_block, ctx) do # No elseifs, just else block - gen_block(else_block, ctx) + {else_instructions, ctx} = gen_block(else_block, ctx) + {append_block_close(else_instructions, else_block, ctx), ctx} end defp gen_elseifs_and_else([{elseif_cond, elseif_block} | rest_elseifs], else_block, ctx) do @@ -859,6 +889,7 @@ defmodule Lua.Compiler.Codegen do # Generate body for this elseif {then_instructions, ctx} = gen_block(elseif_block, ctx) + then_instructions = append_block_close(then_instructions, elseif_block, ctx) # Generate remaining elseifs and else {else_instructions, ctx} = gen_elseifs_and_else(rest_elseifs, else_block, ctx) diff --git a/lib/lua/compiler/scope.ex b/lib/lua/compiler/scope.ex index dba9df36..3badd9ef 100644 --- a/lib/lua/compiler/scope.ex +++ b/lib/lua/compiler/scope.ex @@ -121,10 +121,18 @@ defmodule Lua.Compiler.Scope do # Per Lua 5.3 §3.3.4, each control-flow block (then/else, while/repeat # body, for body) is its own lexical scope: locals declared inside it # must not leak past the block's end. - defp with_block_scope(state, fun) do + # Resolve `fun` in a fresh block scope, restoring the enclosing scope's + # locals and register watermark afterward, and stash the pre-block register + # watermark under `key` so codegen can emit a `:close_upvalues` at the + # block's exit (Lua 5.3 §3.4.10). Any open-upvalue cell over a register the + # block's locals occupied must be detached when the block ends, so a later + # sibling block reusing the same slot does not read or write through the + # stale cell. + defp with_block_scope(state, key, fun) do saved_locals = state.locals saved_next_register = state.next_register + state = %{state | var_map: Map.put(state.var_map, key, saved_next_register)} state = fun.(state) %{state | locals: saved_locals, next_register: saved_next_register} @@ -184,16 +192,16 @@ defmodule Lua.Compiler.Scope do # Per Lua 5.3 §3.3.4, each branch of an `if` is its own block, so locals # declared inside it do not leak past `end`. state = resolve_expr(condition, state) - state = with_block_scope(state, &resolve_block(then_block, &1)) + state = with_block_scope(state, {:block_close_threshold, then_block}, &resolve_block(then_block, &1)) state = Enum.reduce(elseifs, state, fn {elseif_cond, elseif_block}, state -> state = resolve_expr(elseif_cond, state) - with_block_scope(state, &resolve_block(elseif_block, &1)) + with_block_scope(state, {:block_close_threshold, elseif_block}, &resolve_block(elseif_block, &1)) end) if else_block do - with_block_scope(state, &resolve_block(else_block, &1)) + with_block_scope(state, {:block_close_threshold, else_block}, &resolve_block(else_block, &1)) else state end @@ -201,7 +209,7 @@ defmodule Lua.Compiler.Scope do defp resolve_statement(%Statement.While{condition: condition, body: body}, state) do state = resolve_expr(condition, state) - with_block_scope(state, &resolve_block(body, &1)) + with_block_scope(state, {:block_close_threshold, body}, &resolve_block(body, &1)) end defp resolve_statement(%Statement.Repeat{body: body, condition: condition}, state) do @@ -211,6 +219,7 @@ defmodule Lua.Compiler.Scope do saved_locals = state.locals saved_next_register = state.next_register + state = %{state | var_map: Map.put(state.var_map, {:block_close_threshold, body}, saved_next_register)} state = resolve_block(body, state) state = resolve_expr(condition, state) @@ -234,6 +243,13 @@ defmodule Lua.Compiler.Scope do state = %{state | var_map: Map.put(state.var_map, {:for_num_var_reg, for_stmt}, loop_var_reg)} + # Stash the post-loop-variable watermark so codegen can close body-local + # cells at the tail of each iteration. The loop variable's own cells are + # swept by the per-iteration close in the executor's continuation handler + # (keyed on `loop_var_reg`); the body-tail close handles inner-block + # locals declared above this watermark. + state = %{state | var_map: Map.put(state.var_map, {:block_close_threshold, body}, state.next_register)} + func_scope = state.functions[state.current_function] func_scope = %{func_scope | max_register: max(func_scope.max_register, state.next_register)} state = %{state | functions: Map.put(state.functions, state.current_function, func_scope)} @@ -328,6 +344,11 @@ defmodule Lua.Compiler.Scope do state = %{state | var_map: Map.put(state.var_map, {:for_in_var_regs, for_stmt}, Enum.reverse(var_regs))} + # Stash the post-loop-variable watermark so codegen can close body-local + # cells at the tail of each iteration. The loop variables' own cells are + # swept by the per-iteration close in the executor's continuation handler. + state = %{state | var_map: Map.put(state.var_map, {:block_close_threshold, body}, state.next_register)} + func_scope = state.functions[state.current_function] func_scope = %{func_scope | max_register: max(func_scope.max_register, state.next_register)} state = %{state | functions: Map.put(state.functions, state.current_function, func_scope)} diff --git a/test/lua/compiler/max_registers_invariant_test.exs b/test/lua/compiler/max_registers_invariant_test.exs index aad1bfe9..3e655761 100644 --- a/test/lua/compiler/max_registers_invariant_test.exs +++ b/test/lua/compiler/max_registers_invariant_test.exs @@ -84,6 +84,11 @@ defmodule Lua.Compiler.MaxRegistersInvariantTest do op == Bytecode.op_while_loop() -> :while_loop op == Bytecode.op_repeat_loop() -> :repeat_loop op == Bytecode.op_generic_for() -> :generic_for + # close_upvalues carries a register *watermark* (the pre-block + # next_register), not a register operand it reads or writes. The + # watermark can equal max_registers (one past the last live slot), so + # it must not count toward the max-register bound. + op == Bytecode.op_close_upvalues() -> [] true -> raise "register_positions/1 is missing a case for opcode #{inspect(op)}" end end diff --git a/test/lua/vm/upvalue_test.exs b/test/lua/vm/upvalue_test.exs index a224879d..f9825a59 100644 --- a/test/lua/vm/upvalue_test.exs +++ b/test/lua/vm/upvalue_test.exs @@ -7,6 +7,13 @@ defmodule Lua.VM.UpvalueTest do alias Lua.VM.State alias Lua.VM.Stdlib + defp run_lua(code) do + assert {:ok, ast} = Parser.parse(code) + assert {:ok, proto} = Compiler.compile(ast, source: "test.lua") + state = Stdlib.install(State.new()) + VM.execute(proto, state) + end + # Regression test for plan A15. # # Bug: when a local function L was followed by a sibling/descendant closure @@ -169,4 +176,193 @@ defmodule Lua.VM.UpvalueTest do assert {:ok, [42, 42], _state} = VM.execute(proto, state) end end + + # Lua 5.3 §3.4.10: when a block ends, any open-upvalue cell over a register + # the block's locals occupied must be detached so a later sibling block + # reusing the same register slot binds a fresh cell. Without the block-exit + # close, the second block's captured local resolves through the first + # block's stale cell — e.g. an integer where a table is expected, crashing + # with `attempt to index a number value`. + describe "open upvalues close at block exit so sibling blocks do not share cells" do + test "two sibling do blocks reusing the same register for a captured local" do + code = """ + do + local res = 1 + local function fact(n) + if n == 0 then return res else return n * fact(n - 1) end + end + assert(fact(5) == 120) + end + + do + local a = {x = 100} + local function read_a() return a.x end + assert(read_a() == 100, "expected 100, got stale-cell value") + end + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "two sibling if blocks reusing the same register for a captured local" do + code = """ + if true then + local res = 1 + local function fact(n) + if n == 0 then return res else return n * fact(n - 1) end + end + assert(fact(5) == 120) + end + + if true then + local a = {x = 100} + local function read_a() return a.x end + assert(read_a() == 100, "expected 100, got stale-cell value") + end + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "then and else branches reusing the same register for a captured local" do + code = """ + local function pick(flag) + if flag then + local a = {x = 11} + local function get() return a.x end + return get() + else + local b = {y = 22} + local function get() return b.y end + return get() + end + end + + return pick(true), pick(false) + """ + + assert {:ok, [11, 22], _state} = run_lua(code) + end + + test "two sibling while blocks reusing the same register for a captured local" do + code = """ + local n = 0 + while n < 1 do + n = n + 1 + local res = 1 + local function f() return res end + assert(f() == 1) + end + + while n < 2 do + n = n + 1 + local a = {x = 5} + local function g() return a.x end + assert(g() == 5, "stale cell leaked from the previous while block") + end + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "two sibling repeat blocks reusing the same register for a captured local" do + code = """ + local n = 0 + repeat + n = n + 1 + local res = 1 + local function f() return res end + assert(f() == 1) + until n >= 1 + + repeat + n = n + 1 + local a = {x = 7} + local function g() return a.x end + assert(g() == 7, "stale cell leaked from the previous repeat block") + until n >= 2 + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "two sibling numeric for blocks reusing the same register for a captured local" do + code = """ + for _ = 1, 1 do + local res = 1 + local function f() return res end + assert(f() == 1) + end + + for _ = 1, 1 do + local a = {x = 9} + local function g() return a.x end + assert(g() == 9, "stale cell leaked from the previous for block") + end + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "two sibling generic for blocks reusing the same register for a captured local" do + code = """ + for _ in ipairs({1}) do + local res = 1 + local function f() return res end + assert(f() == 1) + end + + for _ in ipairs({1}) do + local a = {x = 13} + local function g() return a.x end + assert(g() == 13, "stale cell leaked from the previous for-in block") + end + + return "ok" + """ + + assert {:ok, ["ok"], _state} = run_lua(code) + end + + test "numeric for: each iteration's closure captures that iteration's value" do + # Cells must persist within an iteration and close on the iteration + # boundary — a closure created in iteration N must observe iteration N's + # value, not leak the next iteration's value. + code = """ + local fns = {} + for i = 1, 3 do + local v = i * 10 + fns[i] = function() return v end + end + return fns[1](), fns[2](), fns[3]() + """ + + assert {:ok, [10, 20, 30], _state} = run_lua(code) + end + + test "loop closure created mid-iteration sees the live value later in the same iteration" do + code = """ + local total = 0 + for i = 1, 3 do + local v = i + local function add() total = total + v end + v = v * 100 + add() + end + return total + """ + + assert {:ok, [600], _state} = run_lua(code) + end + end end diff --git a/test/lua53_skips.exs b/test/lua53_skips.exs index e2662e5f..4c0ec44b 100644 --- a/test/lua53_skips.exs +++ b/test/lua53_skips.exs @@ -59,13 +59,6 @@ reason: "print does not call user-overridden _ENV.tostring at each invocation", issue: nil }, - %{ - lines: 65..69, - category: :executor, - reason: - "stale upvalue cell across do blocks: register reused for a new local still resolves through the previous block's open_upvalues entry", - issue: 276 - }, %{ lines: 135..137, category: :parser, From b084887d4b9ad3d00c135249edf36895278de5e2 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Sun, 31 May 2026 07:10:28 -0700 Subject: [PATCH 3/3] chore(A47): mark plan as review --- .agents/plans/A47-open-upvalue-block-close.md | 53 ++++++++++++++++++- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/.agents/plans/A47-open-upvalue-block-close.md b/.agents/plans/A47-open-upvalue-block-close.md index eff86473..9c6eba6a 100644 --- a/.agents/plans/A47-open-upvalue-block-close.md +++ b/.agents/plans/A47-open-upvalue-block-close.md @@ -2,10 +2,10 @@ id: A47 title: Close open upvalues at if/while/for/repeat block exit issue: 276 -pr: null +pr: 303 branch: fix/open-upvalue-block-close base: main -status: in-progress +status: review direction: A unlocks: - calls.lua @@ -217,3 +217,52 @@ shape. Capture the `mix test --only lua53` pass-count delta and confirm - **Over-closing could regress unrelated upvalue tests.** The full `upvalue_test.exs` and closure-style suites must stay green; run them explicitly before the broader `mix test`. + +## Discoveries + +- `lib/lua/vm/executor.ex` and `lib/lua/vm/dispatcher.ex` needed **no + changes**: the `:close_upvalues` handler, the at-or-above sweep, and + the op-52 bytecode round-trip were all already in place from PR #286. + The entire fix lives in scope analysis (stash the watermark) and + codegen (emit the close at each block's tail). +- Extending `close_upvalues` emission to `if`/`while`/`for`/`repeat` + bodies surfaced a gap in + `test/lua/compiler/max_registers_invariant_test.exs`: its + `register_positions/1` raises on any opcode it does not enumerate, and + op 52 (`close_upvalues`) had no case because no fixture previously + generated one outside a `do` block. Added the case (the threshold is a + register *watermark*, not a register operand, so it contributes no + register and maps to `[]`) and a public `Bytecode.op_close_upvalues/0` + accessor to mirror the other opcode accessors the test relies on. + These two files (`test/lua/compiler/max_registers_invariant_test.exs`, + `lib/lua/compiler/bytecode.ex`) were not in the original scope list but + are pure consumer-side wiring required to keep the suite green; no + behaviour change. + +## What changed + +PR: #303 + +Files touched: +- `lib/lua/compiler/scope.ex` — stash the pre-block `next_register` + watermark under `{:block_close_threshold, block}` for `if` + (then/elseif/else), `while`, `repeat`, `for`-numeric and `for`-in + bodies. Added a keyed `with_block_scope/3`; the old `with_block_scope/2` + was inlined away. +- `lib/lua/compiler/codegen.ex` — new `append_block_close/3` helper; + emits a `:close_upvalues` at the tail of each `if` branch, while/for + body, and (after the condition) the repeat body. +- `lib/lua/compiler/bytecode.ex` — public `op_close_upvalues/0` accessor. +- `test/lua/compiler/max_registers_invariant_test.exs` — handle op 52. +- `test/lua/vm/upvalue_test.exs` — new describe block with 9 regression + tests (sibling do/if/then-else/while/repeat/for-num/for-in + two + per-iteration loop captures). +- `test/lua53_skips.exs` — removed the `calls.lua` 65..69 skip. + +Suite delta: `mix test` 2101 passed / 19 skipped (no failures); +`mix test --only lua53` 17 passed / 12 skipped (unchanged), with +`calls.lua` now executing lines 65–69. No follow-up issues opened. + +The executor and dispatcher were left untouched — the close handler, +the at-or-above sweep, and the op-52 round-trip were already present +from PR #286, as the plan anticipated.