diff --git a/.agents/plans/B5c-v2-dispatcher-closures.md b/.agents/plans/B5c-v2-dispatcher-closures.md index b3bb1d47..901211c3 100644 --- a/.agents/plans/B5c-v2-dispatcher-closures.md +++ b/.agents/plans/B5c-v2-dispatcher-closures.md @@ -1,11 +1,11 @@ --- id: B5c-v2 title: Dispatcher closures, varargs, and multi-return — every opcode covered -issue: null +issue: 272 pr: null branch: perf/dispatcher-closures base: main -status: blocked +status: in_progress direction: B unlocks: - 100% opcode coverage in the dispatcher (no more fallbacks) @@ -15,7 +15,7 @@ parent: B5-dispatcher-and-bytecode ## Blocked on -- B5a-v2 (foundation), B5b-v2 (tables). +- B5a-v2 (foundation, PR #237, merged), B5b-v2 (tables, PR #275, merged). ## Goal @@ -26,8 +26,17 @@ Cover the remaining opcodes in `Lua.VM.Dispatcher`: - `:get_open_upvalue`, `:set_open_upvalue` — open-cell access for captures of mutable locals. - `:vararg`, `:return_vararg`. -- `:return` with count > 1. -- `:generic_for`, `:self`, `:tail_call`. +- `:return` with count > 1, including `{:multi_return, _}` shape. +- `:call` with variable result counts (`-1`, `-2`, `n > 1`) and + `{:multi, _}` arg counts. +- `:generic_for`, `:while_loop`, `:repeat_loop`, `:break`. +- `:self`. +- `:concatenate`. + +`:tail_call` is verified dead in the current codegen (only +`Lua.Compiler.Instruction.tail_call/3` exists as an unused +constructor); tail-position calls compile to `:call` with +`result_count = -1`, which falls out of the multi-return work above. After this PR no prototype falls back to the list-of-tuples interpreter for opcode-coverage reasons. @@ -38,12 +47,317 @@ interpreter for opcode-coverage reasons. from tail-recursive dispatch. - Optimised closure capture (escape analysis, capture-by-value promotion). Defer. +- Error position fidelity in the dispatcher → B5d-v2. All new + bridges and call-site failures pass `line: 0`. +- Mutable register storage / `setelement/3` elimination → its own + plan if perf-justified after this lands. +- Mutable table storage. `Table.put/3` allocation churn is the + ceiling for table workloads on the BEAM; not addressed here. ## Success criteria -(To be detailed when this plan unblocks. Mirrors original B5d -success criteria.) +- [ ] `Lua.Compiler.Bytecode` accepts every opcode listed above; the + catch-all `:fallback` clause now matches only opcodes whose + lowering is genuinely out of scope (`:goto`, `:label`, + `:bitwise_*`, `:set_global` — see Discoveries). +- [ ] `Lua.VM.Dispatcher` has one `case` branch per new opcode plus + new continuation markers (`:cps_while_test`, + `:cps_while_body`, `:cps_repeat_body`, `:cps_repeat_cond`, + `:cps_generic_for`, `:loop_exit`) handled in + `finish_body/6`. +- [ ] Frame tuple gains a `{:multi, dest, count}` variant so + `:call`-multi and `:return`-multi share one frame shape. +- [ ] `Lua.VM.Executor` exposes new `dispatcher_*` bridges where the + dispatcher needs metamethod / stdlib coupling: + `dispatcher_index_method_target/5` for `:self`, + `dispatcher_call_value/5` for `:generic_for` step, + `dispatcher_concat/5` for `:concatenate` slow path. +- [ ] `:break` inside `:numeric_for` no longer forces fallback (the + B5b-v2 `contains_break?` guard is removed). +- [ ] `mix compile --warnings-as-errors` passes. +- [ ] `mix test` passes, no regressions vs. main. +- [ ] `mix test --only lua53` passes. +- [ ] `test/lua/vm/leak_regression_test.exs` still passes. +- [ ] `closures`, `oop`, `string_ops` benchmark inner functions + compile to `:compiled_closure` end-to-end (no fallback chain). +- [ ] No workload regresses by >10% vs. interpreter on the + `dispatcher_vs_interpreter`-style A/B harness. +- [ ] `closures.exs` beats the interpreter by ≥1.5x. + +## Implementation notes + +### New opcode tags (slots 37-47) + +Reserve a contiguous block in both `Lua.Compiler.Bytecode` and +`Lua.VM.Dispatcher`: + +``` +@op_closure 37 +@op_set_upvalue 38 +@op_get_open_upvalue 39 +@op_set_open_upvalue 40 +@op_vararg 41 +@op_return_vararg 42 +@op_return_multi 43 # :return with count > 1 +@op_call_multi 44 # :call with result_count ∉ {0, 1} +@op_self 45 +@op_concatenate 46 +@op_break 47 +@op_while_loop 48 +@op_repeat_loop 49 +@op_generic_for 50 +``` + +`:tail_call` reserves no slot — never emitted by codegen. + +### Frame tuple variant + +Today `base` in the frame tuple is either an integer or `:discard`. +Add a third shape `{:multi, dest, count}` for multi-return: + +- `count == -1` — forward all results to the caller's caller; used + by `result_count = -1` (tail-position calls). +- `count == -2` — expand all results into consecutive regs starting + at `dest`, set `state.multi_return_count`; used by table + constructors with trailing call (`{1, 2, f()}`) and similar. +- `count > 1` — write exactly `count` results, pad with nil. + +`return_one/3` becomes `return_results/3` covering the three frame +variants. Single-result fast path stays. + +### Multi-return contract + +Multi-return is `state.multi_return_count` threading, identical to +the interpreter. `:vararg`-count-0, multi-return `:call`, and +`:return` with `count < 0` all read/write `state.multi_return_count` +on the same contract as `Lua.VM.Executor`. + +Arg-count resolution for `:call_multi`: + +``` +{:multi, fixed_count} -> fixed_count + state.multi_return_count +n when n > 0 -> n +n when n < 0 -> -(n + 1) + state.multi_return_count +0 -> 0 +``` + +Same as `executor.ex:832`. + +### Loop CPS markers + +`:while_loop`, `:repeat_loop`, `:generic_for` each push one of two +markers onto `cont` (one when entering condition/body, the second +when re-entering after the body) and an additional `:loop_exit` +marker beneath them. `:break` scans `cont` for the nearest +`:loop_exit` marker and resumes there. The interpreter's +`find_loop_exit/1` (executor.ex:1891) ports verbatim. + +After the port, the B5b-v2 `contains_break?` guard at +`bytecode.ex:243` can be removed — `:break` inside `:numeric_for` +is no longer special. + +### `:self` bridge + +`:self` resolves `obj[method_name]` via `index_value/6`, the same +helper that backs `:get_field`'s slow path. Wrap as +`dispatcher_index_method_target/5` and return both `{func, state}`. + +### `:concatenate` + +Mirror executor.ex:1210-1241: + +1. Both binary → `left <> right`. +2. Either binary-or-number on both sides → `concat_coerce/3` on each + then `<>`. +3. Else → bridge through `dispatcher_concat/5` which wraps + `try_binary_metamethod("__concat", ...)`. + +Per-pair binary `:concatenate` opcodes (no n-ary chains in the +codegen; `a..b..c` lowers to two sequential `:concatenate` ops). + +### Compiled vs. interpreted closure values + +`:closure` builds `{:compiled_closure, proto, upvalues}` when +`nested_proto.bytecode != nil`, else `{:lua_closure, ...}`. Decision +flows through the tag — no parent-level metadata. Mirrors the +existing executor path verbatim. + +### Files + +- `lib/lua/compiler/bytecode.ex` — 14 new opcode constants, 14 new + `encode/1` clauses, tag accessor methods. +- `lib/lua/vm/dispatcher.ex` — 14 new dispatch branches, 6 new + `finish_body/6` continuation clauses, `find_loop_exit/1` + port, frame-multi return path, closure-cell allocation helper. +- `lib/lua/vm/executor.ex` — 3 new `dispatcher_*` bridges: + `dispatcher_index_method_target`, + `dispatcher_call_value`, + `dispatcher_concat`. The existing public `call_function/3` and + `dispatcher_close_open_upvalues_at_or_above/2` cover the rest. +- `test/lua/compiler/bytecode_test.exs` — flip fallback assertions + for the now-covered opcodes; add new fallback tests for genuinely + unsupported shapes (`:goto`, `:bitwise_*`, `:set_global`). +- `test/lua/vm/dispatcher_test.exs` — per-opcode goldens for every + new dispatch branch plus benchmark-shape goldens for closures, + oop, and string_ops. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 +mix test test/lua/vm/dispatcher_test.exs +mix test test/lua/compiler/bytecode_test.exs +mix test test/lua/vm/leak_regression_test.exs + +# Closures gap check — primary perf gate. +MIX_ENV=benchmark mix run benchmarks/closures.exs + +# Smoke other workloads (no regression > 10%). +MIX_ENV=benchmark mix run benchmarks/oop.exs +MIX_ENV=benchmark mix run benchmarks/string_ops.exs +MIX_ENV=benchmark mix run benchmarks/table_ops.exs +MIX_ENV=benchmark mix run benchmarks/fibonacci.exs +``` + +## Risks + +- **Multi-return frame plumbing touches every call site.** Every + `:call_*` and `:return_*` branch updates. Mitigation: keep the + single-result and zero-result fast paths; only the new + `:call_multi` / `:return_multi` / `:return_vararg` branches use + the `{:multi, _, _}` frame slot. +- **Closure-tag cascade between compiled parent and interpreted + child.** When `proto.bytecode != nil` but a nested closure's + `:closure` opcode encounters a child with `bytecode == nil`, the + dispatcher must emit `:lua_closure`, not `:compiled_closure`. + Symmetric on the interpreter side. Tested directly. +- **`:break` marker scanning.** `find_loop_exit/1` must skip + non-loop continuations (`{code, pc}` from `:test`, + `:cps_for` from `:numeric_for`) and stop at the first + `:loop_exit`. Port the interpreter's matchspec verbatim. +- **Vararg state leakage.** `state.multi_return_count` must reset + after a single-return call following a multi-return call. + Mirroring the interpreter's behaviour: `continue_after_call`'s + count-1 branch never touches `multi_return_count`, but the next + multi-return read picks up the new value. Tested with + back-to-back multi/single calls. ## Discoveries -(Empty until implementation.) +### `:tail_call` is dead code + +The frontmatter listed `:tail_call` as an opcode to cover. After +grepping the entire codebase, the only reference is the unused +constructor helper `Lua.Compiler.Instruction.tail_call/3`. Codegen +never emits this opcode — tail-position calls compile to `:call` with +`result_count = -1`, which forwards through the multi-return machinery. +Once `:call_multi` was wired, `:tail_call` was implicitly covered. No +dedicated opcode added. + +### `:return_vararg` and multi-return `:return` are different data sources + +The original plan conflated `:return_vararg` (returns `proto.varargs` +verbatim — used for `return ...`) with `:return base count<0` / +`{:multi_return, fixed}` (collects from regs, including a fixed prefix +plus `state.multi_return_count` trailing). They share the unwind path +(`return_multi/3`) but read from different sources, so the encoder +emits two distinct opcodes: + +- `@op_return_proto_varargs` (42) — single-tag opcode, reads `proto.varargs`. +- `@op_return_collect` (43) — `{tag, base, fixed}`, reads regs. + +### Falling-off-the-end ≠ explicit `:return _ 0` + +Codegen emits *no* return instruction when a function falls off its +last statement (e.g., a function whose only body is `if cond then +return … end`). The interpreter handles this at executor.ex:506 by +returning the empty list `[]`. The dispatcher's `finish_body([])` +initially called `return_one(nil, …)` which wraps to `[nil]` — that +broke the `range(0, 255)` test where the empty-tail base case +otherwise pads a spurious nil into the multi-return chain. Fix: +`finish_body([])` now calls `return_multi([], …)`. Explicit +`:return _ 0` still returns `[nil]` via `:return_zero` (matching the +interpreter's `count == 0 -> [nil]` branch). + +### `:call_one`/`:call_zero` had to grow vararg setup + +Previously the comment on `:call_one` said "Vararg bodies are out of +scope for the bytecode encoder, so a `{:compiled_closure, ...}` is, by +construction, never a vararg function." After `:vararg` lowering +landed, that invariant no longer holds — a compiled callee can be +vararg. Both fast-path call opcodes now call `setup_vararg_proto/4` +to collect varargs from the caller's regs before the dispatch. The +helper short-circuits at one tuple-field read for the non-vararg +hot path (fib, every recursive call). + +### Frame's `dest` slot generalises to four shapes + +The B5b-v2 frame had `dest` as `integer` (single-result) or +`:discard` (zero-result). B5c-v2 adds `{:multi, base, count}` for the +multi-return shapes (-1 forward, -2 expand, n > 1 fixed multi). The +`:call_multi` opcode picks the slot based on `result_count`: +`result_count == 0` reuses `:discard`, `== 1` reuses `integer base`, +everything else uses the tagged tuple. `return_one/3` (fast path) +and `return_multi/3` (list path) both pattern-match on all four +shapes; the fib hot path stays on the integer-base branch. + +### Register tuple needs lazy growth on multi-return + +The dispatcher's `init_regs/2` sizes the register tuple exactly to +`max_registers`, with no `+16` safety buffer like the interpreter. +Multi-return expansion (`{a, b, c} = f()` where `f` returns 256 +values) can outrun that bound. Added `grow_regs/2` and threaded it +through `write_varargs/4`, `write_results/3`, `write_results_n/4`, +and `pad_nils/3`. Growth is rare and bounded by the call's actual +result count; the fib hot path (single-value returns, integer-base +frames) never triggers it. + +### `dispatcher_call_function` bridge for name_hint fidelity + +The plain `Executor.call_function/3` doesn't take a `name_hint` arg. +When the dispatcher's `:call_*` routes a nil or non-callable through +it, the resulting `TypeError` drops the `(upvalue 'x')`-style +suffix. Added a new bridge `Executor.dispatcher_call_function/5` +that mirrors `:call`'s nil / non-function / __call branches inline, +so error wording stays in lockstep with the interpreter +(`Lua.ErrorMessagesTest` pins this directly). + +### `call_stack` push/pop on dispatcher → dispatcher and dispatcher → lua_closure calls + +The pre-B5c-v2 dispatcher didn't update `state.call_stack` around +compiled-closure calls. With closures, OOP, and vararg now +dispatcher-routed, many error-context tests started failing +(`error.call_stack` empty when they expected ≥2 entries). +Push/pop now happens at every dispatcher `:call_*` site for both +`:compiled_closure` (push at call site, pop in `return_one/3` / +`return_multi/3`) and `:lua_closure` (synchronous push around the +`Executor.call_function/3` invocation). `:native_func` and `__call` +metamethod paths don't push, matching the interpreter's behaviour +at executor.ex:880-882. + +### Perf gates + +- **Hard floor (≤10% regression on any workload):** Met. +- **Soft target (`closures.exs` ≥1.5x interpreter):** Brushed at + 1.22x. Profile attribution: closure-construction allocation, + upvalue cell allocation, and generic-for iterator dispatch + dominate. Further closure-benchmark wins require post-B5 work + (mutable register storage, mutable upvalue cells, escape + analysis). +- **`dispatcher_vs_interpreter` fib(25):** 1.15x (vs B5a-v2's 1.17x + baseline). The ~2% loss is the call_stack push/pop that B5c-v2 + added to match the interpreter's error-context invariants. +- **OOP A/B:** 1.07x faster than interpreter. The `:self` lookup, + table construction, and concatenation dominate over dispatch + wins. + +### Coverage outcome + +`closures.exs`, `oop.exs`, `string_ops.exs` all compile end-to-end +(no fallback in any sub-prototype). The encoder's catch-all +`:fallback` clause now only matches the opcodes called out as out +of scope: `:goto` / `:label` (label resolution), `:set_global` +(codegen vestige), and `:bitwise_*` (their own follow-up plan). diff --git a/lib/lua/compiler/bytecode.ex b/lib/lua/compiler/bytecode.ex index be7afd1d..8bebcac3 100644 --- a/lib/lua/compiler/bytecode.ex +++ b/lib/lua/compiler/bytecode.ex @@ -67,6 +67,31 @@ defmodule Lua.Compiler.Bytecode do @op_length 35 @op_numeric_for 36 + # B5c-v2: closures, upvalues, varargs, multi-return, loops, self, concat. + # After this block lands, the only opcodes the encoder still rejects are + # the data-shape concerns explicitly out of scope (`:goto` / `:label` for + # label-resolution semantics, `:set_global` for codegen vestige, and the + # bitwise family — none of which are blockers for the closures / OOP / + # string_ops benchmarks). + @op_closure 37 + @op_set_upvalue 38 + @op_get_open_upvalue 39 + @op_set_open_upvalue 40 + @op_vararg 41 + # `:return_vararg` (returns `proto.varargs`) and the `:return base count<0` / + # `{:multi_return, fixed}` form (collects from regs) are different data + # sources, so they get distinct tags. Both unwind through `return_multi/3`. + @op_return_proto_varargs 42 + @op_return_collect 43 + @op_return_multi 44 + @op_call_multi 45 + @op_self 46 + @op_concatenate 47 + @op_break 48 + @op_while_loop 49 + @op_repeat_loop 50 + @op_generic_for 51 + @doc """ Compile a prototype, populating its `bytecode` field on success. @@ -176,16 +201,18 @@ defmodule Lua.Compiler.Bytecode do end end - # `:call` shapes the dispatcher covers: + # `:call` shapes: # - # - `result_count == 1`: the expression-call hot path (every - # recursive call in fib/factorial, every `f(x)` used as an rvalue). - # - `result_count == 0`: the statement-call form, e.g. - # `table.sort(t)`. Results are discarded. + # - `result_count == 1` with fixed positive arg_count → `:call_one`, + # the expression-call hot path (every recursive call in fib / + # factorial, every `f(x)` used as an rvalue). + # - `result_count == 0` with fixed positive arg_count → `:call_zero`, + # the statement-call form (e.g. `table.sort(t)`). + # - Anything else (multi-return result, negative result count, + # `{:multi, _}` arg shape, negative arg count) → `:call_multi`. + # This is the B5c-v2 catch-all for the multi-return machinery. # - # Multi-return (`{:multi, _}`), negative result counts, and tail-call - # marker shapes all stay on the interpreter via the catchall. - # `name_hint` is preserved on both shapes for error attribution. + # `name_hint` is preserved on every shape for error attribution. defp encode({:call, base, arg_count, 1, name_hint}) when is_integer(arg_count) and arg_count >= 0 do {:ok, {@op_call_one, base, arg_count, name_hint}} end @@ -194,12 +221,40 @@ defmodule Lua.Compiler.Bytecode do {:ok, {@op_call_zero, base, arg_count, name_hint}} end - # `:return` shapes: single-value is the hot path (every recursive return - # in fib/factorial), zero-value falls through to the interpreter's "no - # explicit return = nil" handling. + defp encode({:call, base, arg_count, result_count, name_hint}) do + {:ok, {@op_call_multi, base, arg_count, result_count, name_hint}} + end + + # `:return` shapes: + # + # - `count == 1` is the hot path (every recursive return in fib/factorial). + # Stays a dedicated opcode so the dispatcher can avoid the result-list + # allocation on the bottom of the call stack. + # - `count == 0` is the codegen-emitted "no explicit return" form; + # handled by `:return_zero`. + # - `count > 1` is the explicit multi-return form (`return a, b, c`). + # - `count < 0` is `-1` for forwarding all results to caller's caller, + # `-2` for expansion. Both share the multi-return collection path. + # - `{:multi_return, fixed}` is the codegen sentinel for "fixed plus + # `state.multi_return_count` trailing values"; equivalent to the + # negative-count form but encoded explicitly. defp encode({:return, base, 1}), do: {:ok, {@op_return_one, base}} defp encode({:return, _base, 0}), do: {:ok, {@op_return_zero}} + defp encode({:return, base, count}) when is_integer(count) and count > 1 do + {:ok, {@op_return_multi, base, count}} + end + + defp encode({:return, base, count}) when is_integer(count) and count < 0 do + # Negative counts: `-(count + 1)` is the fixed prefix; the rest comes + # from `state.multi_return_count` at dispatch time. + {:ok, {@op_return_collect, base, -(count + 1)}} + end + + defp encode({:return, base, {:multi_return, fixed_count}}) do + {:ok, {@op_return_collect, base, fixed_count}} + end + # ── Table opcodes ────────────────────────────────────────────────────── # # The dispatcher inlines the tref + integer/binary-key fast path for @@ -235,47 +290,82 @@ defmodule Lua.Compiler.Bytecode do defp encode({:length, dest, source}), do: {:ok, {@op_length, dest, source}} - # `:numeric_for` carries a nested instruction list for the loop body. - # The body encodes recursively; any uncovered opcode (or a `:break` - # atom — `break`'s loop_exit machinery is its own follow-up) collapses - # the whole loop to interpretation. + # `:numeric_for`, `:while_loop`, `:repeat_loop`, `:generic_for` each + # carry nested instruction lists for the loop body (and, for while / + # repeat, the condition). Bodies encode recursively. The B5b-v2 guard + # against `:break` inside `:numeric_for` is gone — `:break` is a + # first-class opcode now and the dispatcher's `find_loop_exit/1` + # handles the unwind. + defp encode({:numeric_for, base, loop_var, body}) do - if contains_break?(body) do - :fallback - else - case encode_list(body, []) do - {:ok, body_enc} -> - {:ok, {@op_numeric_for, base, loop_var, List.to_tuple(body_enc)}} - - :fallback -> - :fallback - end + case encode_list(body, []) do + {:ok, body_enc} -> + {:ok, {@op_numeric_for, base, loop_var, List.to_tuple(body_enc)}} + + :fallback -> + :fallback + end + end + + defp encode({:while_loop, cond_body, test_reg, loop_body}) do + with {:ok, cond_enc} <- encode_list(cond_body, []), + {:ok, body_enc} <- encode_list(loop_body, []) do + {:ok, {@op_while_loop, test_reg, List.to_tuple(cond_enc), List.to_tuple(body_enc)}} + end + end + + defp encode({:repeat_loop, loop_body, cond_body, test_reg}) do + with {:ok, body_enc} <- encode_list(loop_body, []), + {:ok, cond_enc} <- encode_list(cond_body, []) do + {:ok, {@op_repeat_loop, test_reg, List.to_tuple(body_enc), List.to_tuple(cond_enc)}} end end - # Anything else — `:closure`, `:concatenate`, while/repeat/generic-for - # loops, multi-return calls, vararg, etc. — is out of scope for v2. + # `:generic_for` carries a list of variable register indices that + # receive successive iterator results plus the body. The variable list + # is small (typically one or two regs) so we encode it as a tuple for + # the dispatcher to walk with `:erlang.element/2`. + defp encode({:generic_for, base, var_regs, body}) when is_list(var_regs) do + case encode_list(body, []) do + {:ok, body_enc} -> + {:ok, {@op_generic_for, base, List.to_tuple(var_regs), List.to_tuple(body_enc)}} + + :fallback -> + :fallback + end + end + + # ── Closures, upvalues, varargs, self, concat, break ──────────────────── + + defp encode({:closure, dest, proto_index}), do: {:ok, {@op_closure, dest, proto_index}} + + defp encode({:set_upvalue, index, source}), do: {:ok, {@op_set_upvalue, index, source}} + + defp encode({:get_open_upvalue, dest, reg}), do: {:ok, {@op_get_open_upvalue, dest, reg}} + + defp encode({:set_open_upvalue, reg, source}), do: {:ok, {@op_set_open_upvalue, reg, source}} + + defp encode({:vararg, base, count}), do: {:ok, {@op_vararg, base, count}} + + defp encode({:return_vararg}), do: {:ok, {@op_return_proto_varargs}} + + defp encode({:self, base, obj_reg, method_name, name_hint}), + do: {:ok, {@op_self, base, obj_reg, method_name, name_hint}} + + defp encode({:concatenate, dest, a, b}), do: {:ok, {@op_concatenate, dest, a, b}} + + defp encode(:break), do: {:ok, {@op_break}} + + # Anything else — `:goto` / `:label` (label-resolution semantics not + # ported to the dispatcher), `:set_global` (codegen vestige; modern + # codegen uses `:set_field` on `_ENV` instead), and the bitwise family + # (out of scope for B5c-v2; their own follow-up) — stays on the + # interpreter. # # `:source_line` is stripped upstream in `encode_list/2`, so it never # reaches this clause table. defp encode(_other), do: :fallback - # ── Helpers ───────────────────────────────────────────────────────────── - - # `:break` inside a `:numeric_for` body forces the enclosing loop to - # fall back. The interpreter's loop_exit continuation walk relies on - # the broader `cont` shape; reproducing it inside the dispatcher is a - # B5c-v2 concern, not a table-coverage one. - defp contains_break?(body) when is_list(body) do - Enum.any?(body, fn - :break -> true - {:test, _reg, then_body, else_body} -> contains_break?(then_body) or contains_break?(else_body) - _ -> false - end) - end - - defp contains_break?(_), do: false - # ── Opcode tag accessors ──────────────────────────────────────────────── # # Exposed for `Lua.VM.Dispatcher` (which mirrors these as its own @@ -319,4 +409,19 @@ defmodule Lua.Compiler.Bytecode do def op_set_list, do: @op_set_list def op_length, do: @op_length def op_numeric_for, do: @op_numeric_for + def op_closure, do: @op_closure + def op_set_upvalue, do: @op_set_upvalue + def op_get_open_upvalue, do: @op_get_open_upvalue + def op_set_open_upvalue, do: @op_set_open_upvalue + def op_vararg, do: @op_vararg + def op_return_proto_varargs, do: @op_return_proto_varargs + def op_return_collect, do: @op_return_collect + def op_return_multi, do: @op_return_multi + def op_call_multi, do: @op_call_multi + def op_self, do: @op_self + def op_concatenate, do: @op_concatenate + def op_break, do: @op_break + def op_while_loop, do: @op_while_loop + def op_repeat_loop, do: @op_repeat_loop + def op_generic_for, do: @op_generic_for end diff --git a/lib/lua/vm/dispatcher.ex b/lib/lua/vm/dispatcher.ex index 66dd150f..52fc804c 100644 --- a/lib/lua/vm/dispatcher.ex +++ b/lib/lua/vm/dispatcher.ex @@ -22,6 +22,7 @@ defmodule Lua.VM.Dispatcher do alias Lua.Compiler.Prototype alias Lua.VM.Executor + alias Lua.VM.InternalError alias Lua.VM.Numeric alias Lua.VM.State alias Lua.VM.Table @@ -80,6 +81,24 @@ defmodule Lua.VM.Dispatcher do @op_length 35 @op_numeric_for 36 + # B5c-v2 additions. The contiguous 37..50 block keeps the BEAM's + # case-jump table dense across the dispatcher's outer match. + @op_closure 37 + @op_set_upvalue 38 + @op_get_open_upvalue 39 + @op_set_open_upvalue 40 + @op_vararg 41 + @op_return_proto_varargs 42 + @op_return_collect 43 + @op_return_multi 44 + @op_call_multi 45 + @op_self 46 + @op_concatenate 47 + @op_break 48 + @op_while_loop 49 + @op_repeat_loop 50 + @op_generic_for 51 + @doc """ Execute a compiled prototype against `args` and `state`. """ @@ -533,17 +552,22 @@ defmodule Lua.VM.Dispatcher do # "throw the return value away"; `return_one/3` skips its # setelement write when it sees it. - {@op_call_zero, base, arg_count, _name_hint} -> + {@op_call_zero, base, arg_count, name_hint} -> func_value = :erlang.element(base + 1, regs) case func_value do {:compiled_closure, callee_proto, callee_upvalues} -> callee_regs = init_callee_regs(callee_proto, regs, base + 1, arg_count) + # B5c-v2: compiled callees may now be vararg functions. The + # `is_vararg` check in `setup_vararg_proto/4` short-circuits for + # the common non-vararg case at one tuple-field read. + callee_proto = setup_vararg_proto(callee_proto, regs, base + 1, arg_count) frame = {code, pc + 1, regs, upvalues, proto, cont, :discard, state.open_upvalues} - state = %{state | open_upvalues: %{}} + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack], open_upvalues: %{}} dispatch( callee_proto.bytecode, @@ -556,22 +580,30 @@ defmodule Lua.VM.Dispatcher do [frame | frames] ) + {:lua_closure, _, _} = closure -> + args = collect_args(regs, base + 1, arg_count) + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack]} + {_results, state} = Executor.call_function(closure, args, state) + state = %{state | call_stack: tl(state.call_stack)} + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + _ -> args = collect_args(regs, base + 1, arg_count) - {_results, state} = Executor.call_function(func_value, args, state) + + {_results, state} = + Executor.dispatcher_call_function(func_value, args, state, proto, name_hint) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) end - {@op_call_one, base, arg_count, _name_hint} -> + {@op_call_one, base, arg_count, name_hint} -> func_value = :erlang.element(base + 1, regs) case func_value do - # Vararg bodies are out of scope for the bytecode encoder - # (`:vararg` / `:return_vararg` fall through to `:fallback`), - # so a `{:compiled_closure, ...}` is, by construction, never a - # vararg function. No varargs collection needed here. {:compiled_closure, callee_proto, callee_upvalues} -> callee_regs = init_callee_regs(callee_proto, regs, base + 1, arg_count) + callee_proto = setup_vararg_proto(callee_proto, regs, base + 1, arg_count) # Frame is a tuple, not a map: pattern-matching a tuple in # `return_one/3` skips Map.fetch! lookups and lets the BEAM @@ -579,7 +611,8 @@ defmodule Lua.VM.Dispatcher do frame = {code, pc + 1, regs, upvalues, proto, cont, base, state.open_upvalues} - state = %{state | open_upvalues: %{}} + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack], open_upvalues: %{}} dispatch( callee_proto.bytecode, @@ -592,9 +625,27 @@ defmodule Lua.VM.Dispatcher do [frame | frames] ) + {:lua_closure, _, _} = closure -> + args = collect_args(regs, base + 1, arg_count) + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack]} + {results, state} = Executor.call_function(closure, args, state) + state = %{state | call_stack: tl(state.call_stack)} + + first = + case results do + [v | _] -> v + [] -> nil + end + + regs = :erlang.setelement(base + 1, regs, first) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + _ -> args = collect_args(regs, base + 1, arg_count) - {results, state} = Executor.call_function(func_value, args, state) + + {results, state} = + Executor.dispatcher_call_function(func_value, args, state, proto, name_hint) first = case results do @@ -754,8 +805,266 @@ defmodule Lua.VM.Dispatcher do regs = :erlang.setelement(loop_var + 1, regs, counter) state = Executor.dispatcher_close_open_upvalues_at_or_above(state, loop_var) marker = {:cps_for, base, loop_var, body_bc, code, pc + 1} - dispatch(body_bc, 1, regs, upvalues, proto, state, [marker | cont], frames) + loop_exit = {:loop_exit, code, pc + 1} + dispatch(body_bc, 1, regs, upvalues, proto, state, [marker, loop_exit | cont], frames) + else + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + end + + # ── while_loop / repeat_loop / generic_for ───────────────────── + # + # All three loop forms use a pair of `cont` markers: a CPS marker on + # top, then a `:loop_exit` underneath. The CPS marker drives normal + # body / condition handoff in `finish_body/6`; the loop_exit anchors + # `:break`'s unwind via `find_loop_exit/1`. + + {@op_while_loop, test_reg, cond_bc, body_bc} -> + cps = {:cps_while_test, test_reg, cond_bc, body_bc, code, pc + 1} + loop_exit = {:loop_exit, code, pc + 1} + dispatch(cond_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | cont], frames) + + {@op_repeat_loop, test_reg, body_bc, cond_bc} -> + cps = {:cps_repeat_body, test_reg, body_bc, cond_bc, code, pc + 1} + loop_exit = {:loop_exit, code, pc + 1} + dispatch(body_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | cont], frames) + + {@op_generic_for, base, var_regs, body_bc} -> + # Iterator call follows the same shape as the executor: + # results = iter_func(invariant_state, control). If the first + # result is nil the loop terminates; otherwise control gets the + # first result, var_regs[i] each get results[i]. + iter_func = :erlang.element(base + 1, regs) + invariant_state = :erlang.element(base + 2, regs) + control = :erlang.element(base + 3, regs) + + {results, state} = + Executor.dispatcher_call_value(iter_func, [invariant_state, control], proto, state) + + case results do + [nil | _] -> + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + [] -> + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + [first | _] -> + regs = :erlang.setelement(base + 3, regs, first) + regs = assign_iter_results(regs, var_regs, results, 0) + first_var_reg = :erlang.element(1, var_regs) + state = Executor.dispatcher_close_open_upvalues_at_or_above(state, first_var_reg) + marker = {:cps_generic_for, base, var_regs, body_bc, code, pc + 1} + loop_exit = {:loop_exit, code, pc + 1} + dispatch(body_bc, 1, regs, upvalues, proto, state, [marker, loop_exit | cont], frames) + end + + # ── break ───────────────────────────────────────────────────── + # + # `:break` scans `cont` for the nearest `:loop_exit` marker and + # dispatches to its post-loop target. Everything above the + # loop_exit (CPS markers, `:test` resume points) is dropped. + + {@op_break} -> + {exit_code, exit_pc, rest_cont} = find_loop_exit(cont) + dispatch(exit_code, exit_pc, regs, upvalues, proto, state, rest_cont, frames) + + # ── Closure construction ────────────────────────────────────── + # + # Walks `nested_proto.upvalue_descriptors`, allocating a new cell + # for any `:parent_local` capture that doesn't already have one + # open on this register, or reusing the existing cell so multiple + # closures capturing the same local share a single mutation point. + # `:parent_upvalue` descriptors just forward our own cell ref. + # The closure tag flips between `:lua_closure` and `:compiled_closure` + # based on the child prototype's bytecode availability — the + # decision flows through the closure value, not back through the + # parent prototype. + + {@op_closure, dest, proto_index} -> + nested_proto = Enum.at(proto.prototypes, proto_index) + {cells, state} = build_upvalues(nested_proto.upvalue_descriptors, regs, upvalues, state, []) + upvalues_tuple = List.to_tuple(:lists.reverse(cells)) + + closure = + case nested_proto.bytecode do + nil -> {:lua_closure, nested_proto, upvalues_tuple} + _ -> {:compiled_closure, nested_proto, upvalues_tuple} + end + + regs = :erlang.setelement(dest + 1, regs, closure) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + # ── Upvalue access ──────────────────────────────────────────── + # + # `:set_upvalue` writes through a closed cell (the upvalue tuple + # entry is the ref itself). `:get_open_upvalue` and + # `:set_open_upvalue` operate on locals captured by some other + # closure: if a cell exists, it owns the value; if not, the + # register itself is the source of truth and a `:set_open_upvalue` + # is a no-op (codegen always emits a `:move` first). + + {@op_set_upvalue, index, source} -> + cell_ref = :erlang.element(index + 1, upvalues) + value = :erlang.element(source + 1, regs) + state = %{state | upvalue_cells: Map.put(state.upvalue_cells, cell_ref, value)} + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + {@op_get_open_upvalue, dest, reg} -> + value = + case Map.get(state.open_upvalues, reg) do + nil -> :erlang.element(reg + 1, regs) + cell_ref -> Map.get(state.upvalue_cells, cell_ref) + end + + regs = :erlang.setelement(dest + 1, regs, value) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + {@op_set_open_upvalue, reg, source} -> + state = + case Map.get(state.open_upvalues, reg) do + nil -> + state + + cell_ref -> + value = :erlang.element(source + 1, regs) + %{state | upvalue_cells: Map.put(state.upvalue_cells, cell_ref, value)} + end + + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + # ── Vararg ──────────────────────────────────────────────────── + # + # `:vararg` writes the prototype's varargs into a contiguous range. + # `count == 0` is the "consume all" form: every vararg lands in + # regs[base..] and `state.multi_return_count` is set so the next + # multi-return-aware opcode sees the right length. `count > 0` + # writes exactly that many slots, padding with nil — no + # multi_return_count change. + + {@op_vararg, base, 0} -> + varargs = proto.varargs + {regs, n} = write_varargs(regs, base, varargs, 0) + state = %{state | multi_return_count: n} + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + {@op_vararg, base, count} -> + regs = write_varargs_n(regs, base, proto.varargs, count) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + # ── Multi-return returns ────────────────────────────────────── + + {@op_return_proto_varargs} -> + return_multi(proto.varargs, state, frames) + + {@op_return_collect, base, fixed} -> + total = fixed + state.multi_return_count + results = collect_args(regs, base, total) + return_multi(results, state, frames) + + {@op_return_multi, base, count} -> + results = collect_args(regs, base, count) + return_multi(results, state, frames) + + # ── Multi-return calls ──────────────────────────────────────── + # + # `arg_count` may be a fixed integer, a negative integer (mirror + # of `:return` count<0), or `{:multi, fixed}`. All three fold in + # `state.multi_return_count` exactly as the executor does + # (executor.ex:830). `result_count` discriminates how results + # come back: + # + # -1 → forward to current function's caller (tail-call return). + # -2 → expand into consecutive regs at base, set multi_return_count. + # n>1 → place first n results into regs starting at base. + + {@op_call_multi, base, arg_count, result_count, name_hint} -> + func_value = :erlang.element(base + 1, regs) + + total_args = + case arg_count do + {:multi, fixed} -> fixed + state.multi_return_count + n when is_integer(n) and n > 0 -> n + n when is_integer(n) and n < 0 -> -(n + 1) + state.multi_return_count + 0 -> 0 + end + + case func_value do + {:compiled_closure, callee_proto, callee_upvalues} -> + callee_regs = init_callee_regs(callee_proto, regs, base + 1, total_args) + callee_proto = setup_vararg_proto(callee_proto, regs, base + 1, total_args) + # Reuse the fast-path frame shapes when result_count is 0 + # (discard) or 1 (single integer base). Only the genuine + # multi-return shapes (-1, -2, n > 1) need the tagged + # `{:multi, _, _}` dest. + dest = + case result_count do + 0 -> :discard + 1 -> base + _ -> {:multi, base, result_count} + end + + frame = {code, pc + 1, regs, upvalues, proto, cont, dest, state.open_upvalues} + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack], open_upvalues: %{}} + + dispatch( + callee_proto.bytecode, + 1, + callee_regs, + callee_upvalues, + callee_proto, + state, + [], + [frame | frames] + ) + + {:lua_closure, _, _} = closure -> + args = collect_args(regs, base + 1, total_args) + call_info = Executor.dispatcher_call_info(proto, name_hint, 0) + state = %{state | call_stack: [call_info | state.call_stack]} + {results, state} = Executor.call_function(closure, args, state) + state = %{state | call_stack: tl(state.call_stack)} + apply_multi_call_result(result_count, base, results, code, pc + 1, regs, upvalues, proto, state, cont, frames) + + _ -> + args = collect_args(regs, base + 1, total_args) + + {results, state} = + Executor.dispatcher_call_function(func_value, args, state, proto, name_hint) + + apply_multi_call_result(result_count, base, results, code, pc + 1, regs, upvalues, proto, state, cont, frames) + end + + # ── self ────────────────────────────────────────────────────── + # + # `obj:method(args)` lowers to `:self` followed by `:call`. The + # `:self` opcode writes the resolved method into regs[base] and + # the object into regs[base+1]; the subsequent `:call` reads them + # as func+first-arg. Resolution goes through `index_value/6` so + # `__index` metamethods (the table-OOP idiom) work. + + {@op_self, base, obj_reg, method_name, name_hint} -> + obj = :erlang.element(obj_reg + 1, regs) + {func, state} = Executor.dispatcher_index_method_target(obj, method_name, state, proto, name_hint) + regs = :erlang.setelement(base + 2, regs, obj) + regs = :erlang.setelement(base + 1, regs, func) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) + + # ── Concatenation ───────────────────────────────────────────── + # + # Fast paths inline binary-binary and number/binary mixed forms; + # anything else (table with `__concat`, etc.) bridges to the + # interpreter for metamethod fidelity. + + {@op_concatenate, dest, a, b} -> + left = :erlang.element(a + 1, regs) + right = :erlang.element(b + 1, regs) + + if is_binary(left) and is_binary(right) do + regs = :erlang.setelement(dest + 1, regs, left <> right) + dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) else + {result, state} = Executor.dispatcher_concat(left, right, state, proto) + regs = :erlang.setelement(dest + 1, regs, result) dispatch(code, pc + 1, regs, upvalues, proto, state, cont, frames) end end @@ -768,17 +1077,18 @@ defmodule Lua.VM.Dispatcher do end # `:numeric_for` body ran to completion. Increment the counter, re-test, - # and either restart the body (keeping the marker for the next pass) or - # pop back to the post-loop PC. A `step` of zero infinite-loops here, - # matching the interpreter's behaviour at `do_execute([{:numeric_for, …}])`; - # neither path implements PUC-Lua's "for step is zero" runtime check. - # Fixing that is a separate concern across both executors. + # and either restart the body (re-pushing both the cps_for marker and + # the loop_exit anchor) or pop both off and dispatch at the post-loop + # PC. A `step` of zero infinite-loops here, matching the interpreter's + # behaviour at `do_execute([{:numeric_for, …}])`; neither path implements + # PUC-Lua's "for step is zero" runtime check. Fixing that is a separate + # concern across both executors. defp finish_body( regs, upvalues, proto, state, - [{:cps_for, base, loop_var, body_bc, outer_code, outer_pc} = marker | rest_cont], + [{:cps_for, base, loop_var, body_bc, outer_code, outer_pc} = marker, {:loop_exit, _, _} = loop_exit | rest_cont], frames ) do counter = :erlang.element(base + 1, regs) @@ -791,16 +1101,141 @@ defmodule Lua.VM.Dispatcher do if should_continue do regs = :erlang.setelement(loop_var + 1, regs, new_counter) state = Executor.dispatcher_close_open_upvalues_at_or_above(state, loop_var) - dispatch(body_bc, 1, regs, upvalues, proto, state, [marker | rest_cont], frames) + dispatch(body_bc, 1, regs, upvalues, proto, state, [marker, loop_exit | rest_cont], frames) else dispatch(outer_code, outer_pc, regs, upvalues, proto, state, rest_cont, frames) end end - # Body exhausted with no continuation: prototype ran off the end. - # Lua spec: missing return yields nil. + # `:while_loop`: cond body just finished. Read test_reg; enter body (push + # `:cps_while_body`) or exit. + defp finish_body( + regs, + upvalues, + proto, + state, + [ + {:cps_while_test, test_reg, cond_bc, body_bc, outer_code, outer_pc}, + {:loop_exit, _, _} = loop_exit | rest_cont + ], + frames + ) do + case :erlang.element(test_reg + 1, regs) do + v when v === nil or v === false -> + dispatch(outer_code, outer_pc, regs, upvalues, proto, state, rest_cont, frames) + + _ -> + cps = {:cps_while_body, test_reg, cond_bc, body_bc, outer_code, outer_pc} + dispatch(body_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | rest_cont], frames) + end + end + + # `:while_loop`: body just finished. Restart the condition by pushing + # `:cps_while_test` and dispatching into cond_bc. + defp finish_body( + regs, + upvalues, + proto, + state, + [ + {:cps_while_body, test_reg, cond_bc, body_bc, outer_code, outer_pc}, + {:loop_exit, _, _} = loop_exit | rest_cont + ], + frames + ) do + cps = {:cps_while_test, test_reg, cond_bc, body_bc, outer_code, outer_pc} + dispatch(cond_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | rest_cont], frames) + end + + # `:repeat_loop`: body just finished. Run the condition next. + defp finish_body( + regs, + upvalues, + proto, + state, + [ + {:cps_repeat_body, test_reg, body_bc, cond_bc, outer_code, outer_pc}, + {:loop_exit, _, _} = loop_exit | rest_cont + ], + frames + ) do + cps = {:cps_repeat_cond, test_reg, body_bc, cond_bc, outer_code, outer_pc} + dispatch(cond_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | rest_cont], frames) + end + + # `:repeat_loop`: condition just finished. test_reg truthy = exit (Lua's + # `repeat ... until cond` exits when cond is true). Otherwise loop. + defp finish_body( + regs, + upvalues, + proto, + state, + [ + {:cps_repeat_cond, test_reg, body_bc, cond_bc, outer_code, outer_pc}, + {:loop_exit, _, _} = loop_exit | rest_cont + ], + frames + ) do + case :erlang.element(test_reg + 1, regs) do + v when v === nil or v === false -> + cps = {:cps_repeat_body, test_reg, body_bc, cond_bc, outer_code, outer_pc} + dispatch(body_bc, 1, regs, upvalues, proto, state, [cps, loop_exit | rest_cont], frames) + + _ -> + dispatch(outer_code, outer_pc, regs, upvalues, proto, state, rest_cont, frames) + end + end + + # `:generic_for`: body finished. Call iterator again, re-check nil. + defp finish_body( + regs, + upvalues, + proto, + state, + [ + {:cps_generic_for, base, var_regs, body_bc, outer_code, outer_pc} = marker, + {:loop_exit, _, _} = loop_exit | rest_cont + ], + frames + ) do + iter_func = :erlang.element(base + 1, regs) + invariant_state = :erlang.element(base + 2, regs) + control = :erlang.element(base + 3, regs) + + {results, state} = + Executor.dispatcher_call_value(iter_func, [invariant_state, control], proto, state) + + case results do + [nil | _] -> + dispatch(outer_code, outer_pc, regs, upvalues, proto, state, rest_cont, frames) + + [] -> + dispatch(outer_code, outer_pc, regs, upvalues, proto, state, rest_cont, frames) + + [first | _] -> + regs = :erlang.setelement(base + 3, regs, first) + regs = assign_iter_results(regs, var_regs, results, 0) + first_var_reg = :erlang.element(1, var_regs) + state = Executor.dispatcher_close_open_upvalues_at_or_above(state, first_var_reg) + dispatch(body_bc, 1, regs, upvalues, proto, state, [marker, loop_exit | rest_cont], frames) + end + end + + # Fell off the end of a loop body normally (no CPS marker fired — the + # body ran past its last instruction with the loop_exit still on top). + # Drop the loop_exit and let the next iteration of finish_body see the + # cont below it. + defp finish_body(regs, upvalues, proto, state, [{:loop_exit, _, _} | rest_cont], frames) do + finish_body(regs, upvalues, proto, state, rest_cont, frames) + end + + # Body exhausted with no continuation: prototype ran off the end. Lua + # 5.3 §3.3.4 / the interpreter at executor.ex:506 both return *no* + # values when control falls off the end, not a single `nil` — the + # caller's `result_count` decides how that's projected (nil for a + # single-value site, empty slot for a multi-return one). defp finish_body(_regs, _upvalues, _proto, state, [], frames) do - return_one(nil, state, frames) + return_multi([], state, frames) end # ── Return propagation through frames ─────────────────────────────────── @@ -808,23 +1243,89 @@ defmodule Lua.VM.Dispatcher do # The bottom-of-stack return shape (`{results_list, state}`) matches # the contract `Executor.call_function/3` expects, so the dispatcher's # top-level `execute/3,4` callers see exactly what they would from - # the interpreter. Mid-stack returns skip the list wrapping entirely. + # the interpreter. Mid-stack returns from `:return_one` skip the list + # wrapping entirely so the fib hot path never allocates a cons. + # + # The frame's `dest` slot discriminates how the caller wants results: + # + # integer N → single result lands in regs[N]. `:call_one`. + # :discard → result ignored. `:call_zero`. + # {:multi, B, -1} → forward all results to caller's caller. + # {:multi, B, -2} → expand all into regs[B..], set multi_return_count. + # {:multi, B, n>1} → write n results into regs[B..], pad nil. defp return_one(value, state, []) do {[value], state} end defp return_one(value, state, [frame | rest_frames]) do - {code, pc, regs, upvalues, proto, cont, base, saved_open} = frame + {code, pc, regs, upvalues, proto, cont, dest, saved_open} = frame + # Every dispatcher frame corresponds to a Lua-level call that pushed a + # call_stack entry. Pop it on the way out — the interpreter's + # `do_frame_return/6` does the same at executor.ex:1767. + state = %{state | open_upvalues: saved_open, call_stack: tl(state.call_stack)} + + case dest do + :discard -> + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + n when is_integer(n) -> + regs = :erlang.setelement(n + 1, regs, value) + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + {:multi, _, -1} -> + return_one(value, state, rest_frames) + + {:multi, base, -2} -> + regs = :erlang.setelement(base + 1, regs, value) + state = %{state | multi_return_count: 1} + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + {:multi, base, n} when is_integer(n) and n > 1 -> + regs = :erlang.setelement(base + 1, regs, value) + regs = pad_nils(regs, base + 1, n - 1) + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + end + end - regs = - case base do - :discard -> regs - b -> :erlang.setelement(b + 1, regs, value) - end + # List-return path for `:return_multi`, `:return_collect`, + # `:return_proto_varargs`, and the non-compiled-callee branch of + # `:call_multi`. Mirrors `return_one/3`'s frame-variant handling. - state = %{state | open_upvalues: saved_open} - dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + defp return_multi(results, state, []) do + {results, state} + end + + defp return_multi(results, state, [frame | rest_frames]) do + {code, pc, regs, upvalues, proto, cont, dest, saved_open} = frame + state = %{state | open_upvalues: saved_open, call_stack: tl(state.call_stack)} + + case dest do + :discard -> + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + n when is_integer(n) -> + v = + case results do + [head | _] -> head + [] -> nil + end + + regs = :erlang.setelement(n + 1, regs, v) + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + {:multi, _, -1} -> + return_multi(results, state, rest_frames) + + {:multi, base, -2} -> + regs = write_results(regs, base, results) + state = %{state | multi_return_count: length(results)} + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + + {:multi, base, n} when is_integer(n) and n > 1 -> + regs = write_results_n(regs, base, results, n) + dispatch(code, pc, regs, upvalues, proto, state, cont, rest_frames) + end end # ── Helpers ───────────────────────────────────────────────────────────── @@ -876,4 +1377,212 @@ defmodule Lua.VM.Dispatcher do value = :erlang.element(start + i + 1, regs) set_list_into_table(Table.put(table, offset + i + 1, value), regs, start, count - 1, offset, i + 1) end + + # ── B5c-v2 helpers ────────────────────────────────────────────────────── + + # `:break` unwind. Drops everything above the nearest `:loop_exit` + # marker and returns its post-loop dispatch target plus whatever + # `cont` was below the marker. + defp find_loop_exit([{:loop_exit, code, pc} | rest_cont]), do: {code, pc, rest_cont} + defp find_loop_exit([_other | rest_cont]), do: find_loop_exit(rest_cont) + + defp find_loop_exit([]), do: raise(InternalError, value: "break outside loop") + + # Vararg setup at the call boundary. Mirrors the executor's per-call + # behaviour: when calling a vararg function, regs[param_count..total_args) + # become the varargs list carried on `%{proto | varargs: ...}`. + defp setup_vararg_proto(callee_proto, src_regs, src_off, total_args) do + if callee_proto.is_vararg do + param_count = callee_proto.param_count + vararg_count = max(total_args - param_count, 0) + varargs = collect_args(src_regs, src_off + param_count, vararg_count) + %{callee_proto | varargs: varargs} + else + callee_proto + end + end + + # Closure upvalue capture. `:parent_local` allocates (or reuses) an + # open-upvalue cell so multiple closures over the same register share + # mutation. `:parent_upvalue` forwards our own cell ref to the child. + defp build_upvalues([], _regs, _upvalues, state, acc), do: {acc, state} + + defp build_upvalues([{:parent_local, reg, _name} | rest], regs, upvalues, state, acc) do + {cell_ref, state} = + case Map.get(state.open_upvalues, reg) do + nil -> + new_ref = make_ref() + value = :erlang.element(reg + 1, regs) + + new_state = %{ + state + | upvalue_cells: Map.put(state.upvalue_cells, new_ref, value), + open_upvalues: Map.put(state.open_upvalues, reg, new_ref) + } + + {new_ref, new_state} + + existing_cell -> + {existing_cell, state} + end + + build_upvalues(rest, regs, upvalues, state, [cell_ref | acc]) + end + + defp build_upvalues([{:parent_upvalue, index, _name} | rest], regs, upvalues, state, acc) do + cell_ref = :erlang.element(index + 1, upvalues) + build_upvalues(rest, regs, upvalues, state, [cell_ref | acc]) + end + + # `:vararg` count=0 form: write every vararg into regs[base..] + # and return the count for `state.multi_return_count` update. + # Grows the regs tuple if base + length exceeds current size — the + # codegen sizes `max_registers` for the syntactic call site, but a + # vararg expansion can outrun that statically reserved range + # (e.g. `f(...)` where the variadic call site has more args than + # the caller's other locals require). + defp write_varargs(regs, base, varargs, _n) do + n = length(varargs) + regs = grow_regs(regs, base + n) + write_varargs_loop(regs, base, varargs, 0) + end + + defp write_varargs_loop(regs, _base, [], n), do: {regs, n} + + defp write_varargs_loop(regs, base, [v | rest], n) do + write_varargs_loop(:erlang.setelement(base + n + 1, regs, v), base, rest, n + 1) + end + + # `:vararg` count>0 form: write the first `n` varargs, pad with nil. + defp write_varargs_n(regs, base, vs, n) do + regs = grow_regs(regs, base + n) + write_varargs_n_loop(regs, base, vs, n) + end + + defp write_varargs_n_loop(regs, _base, _vs, 0), do: regs + + defp write_varargs_n_loop(regs, base, [], n) do + write_varargs_n_loop(:erlang.setelement(base + 1, regs, nil), base + 1, [], n - 1) + end + + defp write_varargs_n_loop(regs, base, [v | rest], n) do + write_varargs_n_loop(:erlang.setelement(base + 1, regs, v), base + 1, rest, n - 1) + end + + # `:generic_for` result distribution: each var_reg in the tuple gets + # the next value from `results`. Walks `var_regs` by tuple index and + # consumes `results` head-by-head so each step is O(1); slots past the + # end of `results` receive `nil`. + defp assign_iter_results(regs, var_regs, _results, i) when i >= tuple_size(var_regs), do: regs + + defp assign_iter_results(regs, var_regs, [], i) do + reg = :erlang.element(i + 1, var_regs) + assign_iter_results(:erlang.setelement(reg + 1, regs, nil), var_regs, [], i + 1) + end + + defp assign_iter_results(regs, var_regs, [v | rest], i) do + reg = :erlang.element(i + 1, var_regs) + assign_iter_results(:erlang.setelement(reg + 1, regs, v), var_regs, rest, i + 1) + end + + # Multi-return result writers used by `:call_multi` and `return_multi/3`. + # Both grow the caller's regs tuple if the result range exceeds the + # statically reserved size — mirrors the interpreter's + # `ensure_regs_capacity/2` at the post-call site (executor.ex:1873). + + defp write_results(regs, base, results) do + regs = grow_regs(regs, base + length(results)) + write_results_loop(regs, base, results) + end + + defp write_results_loop(regs, _base, []), do: regs + + defp write_results_loop(regs, base, [v | rest]) do + write_results_loop(:erlang.setelement(base + 1, regs, v), base + 1, rest) + end + + # Bounded version: writes at most `n` values, padding missing slots with nil. + defp write_results_n(regs, base, list, n) do + regs = grow_regs(regs, base + n) + write_results_n_loop(regs, base, list, n) + end + + defp write_results_n_loop(regs, _base, _list, 0), do: regs + + defp write_results_n_loop(regs, base, [], n) do + write_results_n_loop(:erlang.setelement(base + 1, regs, nil), base + 1, [], n - 1) + end + + defp write_results_n_loop(regs, base, [v | rest], n) do + write_results_n_loop(:erlang.setelement(base + 1, regs, v), base + 1, rest, n - 1) + end + + # Pad `n` register slots starting at `start` with nil. Used when a + # multi-result frame receives a single value via `return_one/3`. + defp pad_nils(regs, start, n) do + regs = grow_regs(regs, start + n) + pad_nils_loop(regs, start, n) + end + + defp pad_nils_loop(regs, _start, 0), do: regs + + defp pad_nils_loop(regs, start, n) do + pad_nils_loop(:erlang.setelement(start + 1, regs, nil), start + 1, n - 1) + end + + # Place results from a non-compiled-callee multi-call back into the + # caller's regs. The compiled-callee path goes through the frame stack + # and unwinds via `return_multi/3`; this helper is for the synchronous + # post-call shape (native, __call metamethod, lua_closure via + # call_function). Mirrors `Executor.continue_after_call/11` shape. + defp apply_multi_call_result(0, _base, _results, code, pc, regs, upvalues, proto, state, cont, frames) do + dispatch(code, pc, regs, upvalues, proto, state, cont, frames) + end + + defp apply_multi_call_result(1, base, results, code, pc, regs, upvalues, proto, state, cont, frames) do + first = + case results do + [v | _] -> v + [] -> nil + end + + regs = :erlang.setelement(base + 1, regs, first) + dispatch(code, pc, regs, upvalues, proto, state, cont, frames) + end + + defp apply_multi_call_result(-1, _base, results, _code, _pc, _regs, _upvalues, _proto, state, _cont, frames) do + return_multi(results, state, frames) + end + + defp apply_multi_call_result(-2, base, results, code, pc, regs, upvalues, proto, state, cont, frames) do + regs = write_results(regs, base, results) + state = %{state | multi_return_count: length(results)} + dispatch(code, pc, regs, upvalues, proto, state, cont, frames) + end + + defp apply_multi_call_result(n, base, results, code, pc, regs, upvalues, proto, state, cont, frames) + when is_integer(n) and n > 1 do + regs = write_results_n(regs, base, results, n) + dispatch(code, pc, regs, upvalues, proto, state, cont, frames) + end + + # Lazy regs-tuple growth. Used at the points where multi-return + # expansion or vararg writes can exceed the statically reserved size. + # `Tuple.insert_at/3` is O(n) per insert; this is fine because growth + # is rare and bounded by the call's actual result count. + defp grow_regs(regs, needed) do + current = tuple_size(regs) + + if needed > current do + grow_tuple(regs, needed - current) + else + regs + end + end + + defp grow_tuple(tuple, 0), do: tuple + + defp grow_tuple(tuple, n) do + grow_tuple(Tuple.insert_at(tuple, tuple_size(tuple), nil), n - 1) + end end diff --git a/lib/lua/vm/executor.ex b/lib/lua/vm/executor.ex index 6b4e27b1..de20b805 100644 --- a/lib/lua/vm/executor.ex +++ b/lib/lua/vm/executor.ex @@ -382,6 +382,111 @@ defmodule Lua.VM.Executor do close_open_upvalues_at_or_above(state, threshold) end + # ── Dispatcher bridges: B5c-v2 ────────────────────────────────────────── + # + # `:self` method resolution. Wraps `index_value/6` so __index metamethod + # dispatch matches the interpreter clause-for-clause. Line attribution is + # `0` for the same reason as the other bridges — error positions are + # B5d-v2. + @doc false + @spec dispatcher_index_method_target(term(), term(), State.t(), term(), term()) :: + {term(), State.t()} + def dispatcher_index_method_target(obj, method_name, state, proto, name_hint) do + index_value(obj, method_name, state, 0, proto.source, name_hint) + end + + # `:generic_for` step: invoke the iterator function. The iterator can be + # any callable value (Lua closure, compiled closure, native function, + # value with `__call`), so we route through `call_value/5` which handles + # the whole shape via the interpreter's call machinery. + @doc false + @spec dispatcher_call_value(term(), [term()], term(), State.t()) :: + {[term()], State.t()} + def dispatcher_call_value(callable, args, proto, state) do + call_value(callable, args, proto, state, 0) + end + + # `:concatenate` slow path. Mirrors the interpreter's three-way fallback: + # both-binary → `<>`, both binary-or-number → `concat_coerce/3 . <>`, + # otherwise `__concat` metamethod via `try_binary_metamethod/5`. + # The dispatcher inlines the binary-binary fast path itself, but defers + # the metatable case and the coerce path here so the type-error wording + # stays in sync with the interpreter. + @doc false + @spec dispatcher_concat(term(), term(), State.t(), term()) :: + {binary(), State.t()} + def dispatcher_concat(left, right, state, proto) do + src = proto.source + + try_binary_metamethod("__concat", left, right, state, fn -> + concat_coerce(left, 0, src) <> concat_coerce(right, 0, src) + end) + end + + # `:call_*` out-of-mode bridge with name_hint-aware error wording. + # The plain `call_function/3` path doesn't take a hint, so calling a + # nil/non-function with `(upvalue 'x')`-style attribution would drop + # the suffix. This wraps the same dispatch as `call_function/3` for + # the callable shapes and inlines the type-error paths so the + # dispatcher's error messages match the interpreter's `:call` opcode. + @doc false + @spec dispatcher_call_function(term(), [term()], State.t(), term(), term()) :: + {[term()], State.t()} + def dispatcher_call_function(nil, _args, state, proto, name_hint) do + raise TypeError, + value: "attempt to call a nil value" <> format_target_hint(name_hint), + source: proto.source, + call_stack: state.call_stack, + line: 0, + error_kind: :call_nil, + value_type: nil + end + + def dispatcher_call_function({:lua_closure, _, _} = closure, args, state, _proto, _name_hint), + do: call_function(closure, args, state) + + def dispatcher_call_function({:compiled_closure, _, _} = closure, args, state, _proto, _name_hint), + do: call_function(closure, args, state) + + def dispatcher_call_function({:native_func, _} = nf, args, state, _proto, _name_hint), + do: call_function(nf, args, state) + + def dispatcher_call_function(other, args, state, proto, name_hint) do + case get_metatable(other, state) do + nil -> + raise TypeError, + value: "attempt to call a #{Value.type_name(other)} value" <> format_target_hint(name_hint), + source: proto.source, + call_stack: state.call_stack, + line: 0, + error_kind: :call_non_function, + value_type: value_type(other) + + {:tref, mt_id} -> + mt = Map.fetch!(state.tables, mt_id) + + case Map.get(mt.data, "__call") do + nil -> + raise TypeError, + value: "attempt to call a #{Value.type_name(other)} value" <> format_target_hint(name_hint), + source: proto.source, + call_stack: state.call_stack, + line: 0, + error_kind: :call_non_function, + value_type: value_type(other) + + call_mm -> + call_function(call_mm, [other | args], state) + end + end + end + + @doc false + @spec dispatcher_call_info(term(), term(), non_neg_integer()) :: map() + def dispatcher_call_info(proto, name_hint, line) do + %{source: proto.source, line: line, name: hint_name(name_hint)} + end + # ── Break ────────────────────────────────────────────────────────────────── defp do_execute([:break | _rest], regs, upvalues, proto, state, cont, frames, line) do diff --git a/test/lua/compiler/bytecode_test.exs b/test/lua/compiler/bytecode_test.exs index 92dd83db..633704b0 100644 --- a/test/lua/compiler/bytecode_test.exs +++ b/test/lua/compiler/bytecode_test.exs @@ -54,19 +54,16 @@ defmodule Lua.Compiler.BytecodeTest do end end - describe "fallback on unsupported opcodes" do - test ":closure causes the enclosing prototype to fall back" do - # The chunk emits `:closure` to materialize `f`, so the chunk - # itself falls back. The nested `f` body still compiles. + describe "supported opcodes (B5c-v2)" do + test ":closure compiles to bytecode" do + # B5c-v2: chunks that build closures now compile end-to-end. proto = compile!("function f() return 1 end") - assert proto.bytecode == nil + assert is_tuple(proto.bytecode) [fn_proto] = proto.prototypes assert is_tuple(fn_proto.bytecode) end - test ":generic_for causes fallback" do - # `for k, v in pairs(t)` emits `:generic_for`, which sits outside - # the dispatcher's loop coverage (only numeric-for is wired up). + test ":generic_for compiles" do proto = compile!(""" function iter(t) @@ -77,18 +74,18 @@ defmodule Lua.Compiler.BytecodeTest do """) [fn_proto] = proto.prototypes - assert fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) end - test ":concatenate causes fallback" do + test ":concatenate compiles" do proto = compile!("function f(a, b) return a .. b end") [fn_proto] = proto.prototypes - assert fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) end - test "multi-return call causes fallback" do + test "multi-return call compiles" do # `return f(x)` compiles as a tail-call-style multi-return (-1), - # which is outside dispatcher coverage. + # which routes through `:call_multi` in B5c-v2. proto = compile!(""" function caller() @@ -97,12 +94,10 @@ defmodule Lua.Compiler.BytecodeTest do """) [caller_proto] = proto.prototypes - assert caller_proto.bytecode == nil + assert is_tuple(caller_proto.bytecode) end - test "while-loops cause fallback" do - # While/repeat/generic-for stay on the interpreter; only - # numeric-for is covered by the dispatcher in B5b-v2. + test "while-loop compiles" do proto = compile!(""" function f(n) @@ -113,13 +108,10 @@ defmodule Lua.Compiler.BytecodeTest do """) [fn_proto] = proto.prototypes - assert fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) end - test ":break inside numeric_for causes fallback" do - # `:break` requires loop_exit continuation walking which the - # dispatcher doesn't model. The whole enclosing numeric-for - # collapses to interpretation when the body contains a break. + test ":break inside numeric_for compiles" do proto = compile!(""" function f(n) @@ -131,18 +123,41 @@ defmodule Lua.Compiler.BytecodeTest do """) [fn_proto] = proto.prototypes - assert fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) end - test ":vararg opcode causes fallback" do - # Using `...` as an expression emits a `:vararg` opcode, which is - # out of scope. A vararg signature alone (without using `...`) - # doesn't emit anything special and is fine. + test ":vararg opcode compiles" do proto = compile!("function f(...) local first = ... return first end") [fn_proto] = proto.prototypes - assert fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) + end + + test ":self method-call compiles" do + proto = + compile!(""" + function obj_method(obj) return obj:method(1, 2) end + """) + + [fn_proto] = proto.prototypes + assert is_tuple(fn_proto.bytecode) end + test "repeat loop compiles" do + proto = + compile!(""" + function f(n) + local i = 0 + repeat i = i + 1 until i >= n + return i + end + """) + + [fn_proto] = proto.prototypes + assert is_tuple(fn_proto.bytecode) + end + end + + describe "fallback on unsupported opcodes" do test ":set_list with count == 0 falls back (interpreter's multi-return sentinel)" do # Current codegen never emits `{:set_list, _, _, 0, _}` from a # literal constructor — the interpreter treats `count == 0` as @@ -179,14 +194,12 @@ defmodule Lua.Compiler.BytecodeTest do describe "cascade independence" do test "child prototype compiles even when sibling falls back" do - # `pure` is pure arithmetic (covered). `impure` returns the - # result of a function call with all results forwarded — a - # multi-return shape (`:call` with `result_count = -1` plus - # `:return_vararg`) that stays outside dispatcher coverage. + # `pure` is pure arithmetic (covered). `impure` uses bitwise AND + # which is not yet in dispatcher coverage (its own follow-up plan). proto = compile!(""" function pure(a, b) return a + b end - function impure(t) return next(t) end + function impure(a, b) return a & b end """) [pure_proto, impure_proto] = proto.prototypes @@ -195,12 +208,12 @@ defmodule Lua.Compiler.BytecodeTest do end test "deeply-nested function compiles even when its parent falls back" do - # The outer `make` builds a table (fallback), but the inner adder - # is a pure-arithmetic single-result function (compiles). + # The outer `make` uses bitwise AND (fallback), but the inner + # adder is a pure-arithmetic single-result function (compiles). proto = compile!(""" function make() - local fns = {} + local m = 1 & 0 local function add(a, b) return a + b end return add end @@ -215,12 +228,12 @@ defmodule Lua.Compiler.BytecodeTest do end describe "edge cases" do - test "an empty function body falls back gracefully (return 0 args)" do + test "an empty function body compiles" do # Empty body codegen emits `{:return, 0, 0}` which is the - # zero-result form. Currently encoded as `@op_return_zero`. + # zero-result form. Encoded as `@op_return_zero`. proto = compile!("function f() end") [fn_proto] = proto.prototypes - assert is_tuple(fn_proto.bytecode) or fn_proto.bytecode == nil + assert is_tuple(fn_proto.bytecode) end test "source_line opcodes are stripped from the encoding" do @@ -243,8 +256,10 @@ defmodule Lua.Compiler.BytecodeTest do end test "fallback returns a Prototype with bytecode: nil, never an error" do - # The encoder must not crash on any well-formed prototype. - proto = compile!("function f() return coroutine.yield() end") + # The encoder must not crash on any well-formed prototype. Bitwise + # operations stay on the interpreter (out of scope for B5c-v2), + # so use one to exercise the fallback path. + proto = compile!("function f(a, b) return a | b end") [fn_proto] = proto.prototypes assert %Prototype{} = fn_proto assert fn_proto.bytecode == nil diff --git a/test/lua/compiler/max_registers_invariant_test.exs b/test/lua/compiler/max_registers_invariant_test.exs index 1e0a82a0..aad1bfe9 100644 --- a/test/lua/compiler/max_registers_invariant_test.exs +++ b/test/lua/compiler/max_registers_invariant_test.exs @@ -55,9 +55,35 @@ defmodule Lua.Compiler.MaxRegistersInvariantTest do op == Bytecode.op_not_equal() -> [1, 2, 3] op == Bytecode.op_not() -> [1, 2] op == Bytecode.op_test() -> [1] + op == Bytecode.op_call_zero() -> [1] op == Bytecode.op_call_one() -> [1] op == Bytecode.op_return_one() -> [1] op == Bytecode.op_return_zero() -> [] + # Table opcodes (B5b-v2). + op == Bytecode.op_new_table() -> [1] + op == Bytecode.op_get_table() -> [1, 2, 3] + op == Bytecode.op_set_table() -> [1, 2, 3] + op == Bytecode.op_set_field() -> [1, 3] + op == Bytecode.op_set_list() -> [1, 2] + op == Bytecode.op_length() -> [1, 2] + op == Bytecode.op_numeric_for() -> :numeric_for + # B5c-v2 additions. Bytecode tags for closures, upvalues, varargs, + # multi-return, loops, self, concat, break. + op == Bytecode.op_closure() -> [1] + op == Bytecode.op_set_upvalue() -> [2] + op == Bytecode.op_get_open_upvalue() -> [1, 2] + op == Bytecode.op_set_open_upvalue() -> [1, 2] + op == Bytecode.op_vararg() -> :vararg + op == Bytecode.op_return_proto_varargs() -> [] + op == Bytecode.op_return_collect() -> [1] + op == Bytecode.op_return_multi() -> :return_multi + op == Bytecode.op_call_multi() -> [1] + op == Bytecode.op_self() -> [1, 2] + op == Bytecode.op_concatenate() -> [1, 2, 3] + op == Bytecode.op_break() -> [] + op == Bytecode.op_while_loop() -> :while_loop + op == Bytecode.op_repeat_loop() -> :repeat_loop + op == Bytecode.op_generic_for() -> :generic_for true -> raise "register_positions/1 is missing a case for opcode #{inspect(op)}" end end @@ -78,6 +104,51 @@ defmodule Lua.Compiler.MaxRegistersInvariantTest do count = :erlang.element(3, instr) dest + count - 1 + :numeric_for -> + # {tag, base, loop_var, body_bc}: reads base..base+2, writes loop_var, + # recurses into body_bc. + base = :erlang.element(2, instr) + loop_var = :erlang.element(3, instr) + body_bc = :erlang.element(4, instr) + Enum.max([base + 2, loop_var, max_register_used(body_bc)]) + + :vararg -> + # {tag, base, count}: writes base..base+count-1 when count>0. + # When count==0 the runtime writes all varargs starting at base, + # but we can only bound by the syntactic count operand. Codegen + # is responsible for sizing max_registers correctly here. + base = :erlang.element(2, instr) + count = :erlang.element(3, instr) + if count == 0, do: base, else: base + count - 1 + + :return_multi -> + # {tag, base, count}: reads base..base+count-1. + base = :erlang.element(2, instr) + count = :erlang.element(3, instr) + base + count - 1 + + :while_loop -> + # {tag, test_reg, cond_bc, body_bc}. + test_reg = :erlang.element(2, instr) + cond_bc = :erlang.element(3, instr) + body_bc = :erlang.element(4, instr) + Enum.max([test_reg, max_register_used(cond_bc), max_register_used(body_bc)]) + + :repeat_loop -> + # {tag, test_reg, body_bc, cond_bc}. + test_reg = :erlang.element(2, instr) + body_bc = :erlang.element(3, instr) + cond_bc = :erlang.element(4, instr) + Enum.max([test_reg, max_register_used(body_bc), max_register_used(cond_bc)]) + + :generic_for -> + # {tag, base, var_regs_tuple, body_bc}. + base = :erlang.element(2, instr) + var_regs_tuple = :erlang.element(3, instr) + body_bc = :erlang.element(4, instr) + var_max = Enum.reduce(Tuple.to_list(var_regs_tuple), -1, &max/2) + Enum.max([base + 2, var_max, max_register_used(body_bc)]) + positions when is_list(positions) -> direct = Enum.reduce(positions, -1, fn pos, acc -> max(acc, :erlang.element(pos + 1, instr)) end) diff --git a/test/lua/vm/dispatcher_test.exs b/test/lua/vm/dispatcher_test.exs index 03e00f7c..99fad540 100644 --- a/test/lua/vm/dispatcher_test.exs +++ b/test/lua/vm/dispatcher_test.exs @@ -345,18 +345,19 @@ defmodule Lua.VM.DispatcherTest do end describe "interop with interpreter" do - test "compiled callee returns to interpreted caller correctly" do - # The outer chunk's `:call` (multi-return into `return`) is not - # bytecode-compilable, so it stays on the interpreter. The inner - # `add` function compiles. This exercises the interpreter → - # dispatcher → interpreter round-trip. + test "compiled callee + compiled outer chunk round-trip" do + # After B5c-v2 the chunk's `:closure` + multi-return `:call` + tail + # `:return_collect` all compile, so the whole program is dispatcher- + # routed. This still exercises the dispatcher → dispatcher call + # path; pinning correctness here guards against frame plumbing + # regressions. {proto, results} = run!(""" function add(a, b) return a + b end return add(2, 3) """) - assert proto.bytecode == nil, "outer chunk should fall back" + assert proto.bytecode != nil, "outer chunk should compile after B5c-v2" assert first_sub(proto).bytecode != nil, "inner fn should compile" assert results == [5] end @@ -370,13 +371,221 @@ defmodule Lua.VM.DispatcherTest do return (x + y) """) - # The __add closure compiles; the chunk falls back due to - # setmetatable/table-construction opcodes outside coverage. _ = proto assert results == [30] end end + # ── B5c-v2 opcodes ──────────────────────────────────────────────────── + + describe "closures + upvalues" do + test ":closure with upvalue capture (parent_local)" do + {proto, results} = + run!(""" + function make_adder(x) + return function(y) return x + y end + end + local f = make_adder(10) + return f(5) + """) + + [make_adder | _] = proto.prototypes + [inner | _] = make_adder.prototypes + assert is_tuple(inner.bytecode) + assert results == [15] + end + + test ":set_upvalue mutates a captured local" do + # The counter pattern from the closures benchmark. + {_proto, results} = + run!(""" + function make_counter() + local count = 0 + return function() count = count + 1 return count end + end + local c = make_counter() + c() + c() + c() + return c() + """) + + assert results == [4] + end + + test "deeply-nested closure cascade" do + {_proto, results} = + run!(""" + function outer(a) + return function(b) + return function(c) return a + b + c end + end + end + return outer(1)(2)(3) + """) + + assert results == [6] + end + + test "multiple closures sharing one open upvalue" do + {_proto, results} = + run!(""" + function make_pair() + local x = 0 + local function get() return x end + local function set(v) x = v end + set(42) + return get() + end + return make_pair() + """) + + assert results == [42] + end + end + + describe "vararg + multi-return" do + test ":vararg expansion" do + {_proto, results} = + run!(""" + function f(...) + return select('#', ...) + end + return f(1, 2, 3, 4) + """) + + assert results == [4] + end + + test ":return_collect (return ... )" do + {_proto, results} = + run!(""" + function f(...) + return ... + end + return f(10, 20, 30) + """) + + assert results == [10, 20, 30] + end + + test "multi-return :call with -2 (table-construction-like expansion)" do + {_proto, results} = + run!(""" + function pair() return 1, 2 end + function trio() local a, b = pair() return a, b, 3 end + return trio() + """) + + assert results == [1, 2, 3] + end + + test "fixed-count assignment from multi-return takes first N" do + {_proto, results} = + run!(""" + function three() return 1, 2, 3 end + local a, b = three() + return a + b + """) + + assert results == [3] + end + end + + describe "self + concatenate (OOP)" do + test ":self method call via __index metatable" do + {_proto, results} = + run!(""" + local Greeter = {} + Greeter.__index = Greeter + function Greeter.new(name) + local self = setmetatable({}, Greeter) + self.name = name + return self + end + function Greeter:hello() return "hi " .. self.name end + local g = Greeter.new("world") + return g:hello() + """) + + assert results == ["hi world"] + end + + test ":concatenate chains binaries and numbers" do + {_proto, results} = + run!(""" + function f(a, b) return a .. " " .. b end + return f("hello", 42) + """) + + assert results == ["hello 42"] + end + end + + describe "loops + break" do + test ":while_loop with break" do + {_proto, results} = + run!(""" + function f(n) + local i = 0 + while true do + i = i + 1 + if i >= n then break end + end + return i + end + return f(7) + """) + + assert results == [7] + end + + test ":repeat_loop terminates on truthy condition" do + {_proto, results} = + run!(""" + function f(n) + local i = 0 + repeat i = i + 1 until i >= n + return i + end + return f(5) + """) + + assert results == [5] + end + + test ":generic_for with pairs" do + {_proto, results} = + run!(""" + function f(t) + local sum = 0 + for _, v in pairs(t) do sum = sum + v end + return sum + end + return f({1, 2, 3, 4}) + """) + + assert results == [10] + end + + test ":break inside :numeric_for now compiles and runs" do + {_proto, results} = + run!(""" + function f(n) + local last = 0 + for i = 1, n do + if i > 5 then break end + last = i + end + return last + end + return f(100) + """) + + assert results == [5] + end + end + describe "upvalues" do test ":get_upvalue reads a captured local through a compiled closure" do # Exercises the dispatcher's :get_upvalue path end-to-end. The