From 1365016bc4f8d6b633338392a06b3fd0b281cc39 Mon Sep 17 00:00:00 2001 From: dave Date: Thu, 21 May 2026 14:07:12 -0700 Subject: [PATCH] chore(plans): scope B4-B8 perf follow-ups after PR #223 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five plan files for the next round of executor performance work. PR #223 closed the cheap-fastpath gap (Luerl 1.5-2x → 0.95-1.35x); these plans address the remaining structural overheads: - B4 — Flat instruction stream + PC dispatch. Replace list-of-tuples instruction representation with a tuple + PC integer. Eliminates per-opcode cons traversal in `do_execute` (currently 43.6% of fib(22) self-time). - B5 — Compile prototypes to Erlang functions. Generate Erlang abstract forms from prototypes; pass through `compile:forms/2` and load as modules. Pushes opcode selection from runtime into compile time. Blocked on B4. - B6 — Direct table refs. Pass the resolved `%Lua.VM.Table{}` through helpers instead of re-fetching from `state.tables` on every step. Combined `Map.get/2 + /3` is ~6.4% of fib(22). - B7 — Array+hash split for table storage. Contiguous integer keys move to a tuple part; everything else stays in the hash. O(1) `#t`, O(1) `t[#t+1] = x`, lower memory ratio. Supersedes A10b. - B8 — Inline `Numeric.to_signed_int64/1` fast path. Guard-clause on in-range integers; one of the smaller, safer wins. Each plan grounds its stretch targets in the post-#223 profile numbers and follows the existing `.agents/plans/` schema. B4, B6, B7, B8 are `status: ready`; B5 is `status: blocked` on B4. --- .agents/plans/B4-flat-instruction-stream.md | 235 ++++++++++++++++ .../plans/B5-compile-prototypes-to-erlang.md | 254 ++++++++++++++++++ .agents/plans/B6-direct-table-refs.md | 212 +++++++++++++++ .agents/plans/B7-table-array-hash-split.md | 253 +++++++++++++++++ .agents/plans/B8-inline-numeric-narrowing.md | 158 +++++++++++ 5 files changed, 1112 insertions(+) create mode 100644 .agents/plans/B4-flat-instruction-stream.md create mode 100644 .agents/plans/B5-compile-prototypes-to-erlang.md create mode 100644 .agents/plans/B6-direct-table-refs.md create mode 100644 .agents/plans/B7-table-array-hash-split.md create mode 100644 .agents/plans/B8-inline-numeric-narrowing.md diff --git a/.agents/plans/B4-flat-instruction-stream.md b/.agents/plans/B4-flat-instruction-stream.md new file mode 100644 index 00000000..3539b64b --- /dev/null +++ b/.agents/plans/B4-flat-instruction-stream.md @@ -0,0 +1,235 @@ +--- +id: B4 +title: Flat instruction stream + PC dispatch (replace list-of-tuples) +issue: null +pr: null +branch: perf/flat-instruction-stream +base: main +status: ready +direction: B +unlocks: + - B5 (Erlang-function compilation builds on flat layout) + - measurable reduction in `do_execute/8` self-time +--- + +## Goal + +Replace the current `instructions :: [tuple()]` representation with a +fixed-size tuple of opcodes plus an integer program counter. The +`do_execute/8` dispatch loop becomes `dispatch(pc, instrs, ...)` with +`elem(instrs, pc)` selection instead of list head-matching. + +This eliminates the per-opcode list cons traversal cost. In the current +fib(22) profile `do_execute/8` accounts for **43.6%** of total time, and +a meaningful share of that is the implicit `[head | rest]` pattern +destructuring on every dispatch. A PC + tuple layout removes those +cons-cell reads. + +## Why now + +The fast-path work in PR #223 (numeric arith, comparison, string +concat, get_field) closed several arithmetic and table-access gaps. +The remaining dispatch cost is structural — it lives in the shape of +the instruction stream, not in any single opcode handler. Until that +shape changes, further per-opcode fast paths give diminishing returns. + +## Out of scope + +- Compiling instruction streams into Erlang functions. That is B5 and + builds on whatever flat layout this plan lands. +- Changing the opcode set or the tuple shape of individual instructions. +- Performance work on `do_frame_return/6` or `copy_args_to_regs/5`. + Those are separate plans. +- Changing the codegen output format beyond what's needed to emit a + tuple + jump table instead of a list. Peephole optimizations are B2. + +## Success criteria + +- [ ] `Lua.Compiler.Prototype.instructions` is a tuple (or a struct + wrapping a tuple plus a label-to-pc map), not a list. +- [ ] `Lua.VM.Executor.do_execute/N` takes a program counter and indexes + into the instruction tuple via `elem/2`. +- [ ] Control flow (`:goto`, `:label`, `:while_loop`, `:repeat_loop`, + `:numeric_for`, `:generic_for`, `:break`) resolves to integer PCs + at compile time. No `find_label/2` linear scan at runtime. +- [ ] `mix test` passes (≥ 1273 tests; current is 1692 + 51 properties + + 55 doctests). +- [ ] `mix test --only lua53` does not regress against the current + pass count. +- [ ] Benchee microbenchmarks improve. Stretch target: **20%+ reduction + in fib(25) median**. Floor: no workload regresses by more than + 2% (within noise). +- [ ] Profile after merge: `do_execute/N` self-time drops below 35% + on fib(22) (currently 43.6%). + +## Implementation notes + +### Current layout (for reference) + +```elixir +# Lua.Compiler.Prototype +defstruct [ + :instructions, # list of tuples: [{:add, 0, 1, 2}, {:return, 0, 1}, ...] + :max_registers, + :param_count, + ... +] + +# Lua.VM.Executor dispatch +defp do_execute([{:add, dest, a, b} | rest], regs, ...) do + ... + do_execute(rest, new_regs, ...) +end +``` + +### Proposed layout + +```elixir +defstruct [ + :instructions, # tuple of tuples: {{:add, 0, 1, 2}, {:return, 0, 1}, ...} + :labels, # %{label_name => pc_integer} for resolving :goto + :max_registers, + :param_count, + ... +] + +# Dispatch via PC +defp do_execute(pc, instrs, regs, ...) when pc < tuple_size(instrs) do + case elem(instrs, pc) do + {:add, dest, a, b} -> ...; do_execute(pc + 1, instrs, ...) + ... + end +end +defp do_execute(_pc, _instrs, regs, ..., frames, ...) do + # Off the end of instruction stream — same as the current `[]` case + ... +end +``` + +Two viable shapes for the dispatch: + +1. **Single `case` in a single function head** — closest to a bytecode + interpreter. Easy to read. BEAM compiles a `case` on a tagged tuple + into a jump table. +2. **Multi-head pattern match on `elem(instrs, pc)`** — same source + as today's head-matching, but the head is the result of `elem/2`. + May compile to less efficient code than a `case` because the BEAM + re-tests the result type per head; needs measurement. + +Recommend (1) for the initial implementation. If profiling shows the +`case` is the bottleneck, B5 supersedes this anyway. + +### Control flow + +The current `goto`/`label` implementation uses `find_label/2` at +runtime — a linear scan of the remaining instruction list. The new +layout resolves labels to PCs at codegen time: + +```elixir +# Codegen pass: walk instructions once, build label index +@spec resolve_labels([tuple()]) :: {tuple(), %{atom() => non_neg_integer()}} +def resolve_labels(instructions) do + {labels, _} = + instructions + |> Enum.with_index() + |> Enum.reduce({%{}, 0}, fn + {{:label, name}, idx}, {labels, _} -> {Map.put(labels, name, idx), idx} + _, acc -> acc + end) + + {List.to_tuple(instructions), labels} +end +``` + +`:goto label` becomes `{:goto, pc}` (resolved at codegen). `:label name` +stays in the stream as a no-op (executor skips it) so the PCs line up +with codegen's labeling. + +### Loop CPS continuations + +The current loop opcodes (`:while_loop`, `:numeric_for`, `:generic_for`, +`:repeat_loop`) carry inline `body` and `cond_body` instruction lists. +Under the flat layout, these become PC ranges or labels. + +Two options: + +1. **Keep inline lists** but lift them at codegen time into separate + PC ranges in the same flat tuple, with header instructions like + `{:numeric_for, base, loop_var, body_pc, exit_pc}`. The CPS frame + stack still threads through the same way — it just stores PCs + instead of instruction lists. + +2. **Compile each loop body to a sub-prototype.** Simpler conceptually + but adds a frame on every iteration (overhead, defeats the purpose). + +Recommend (1). The CPS frames currently store `{:cps_numeric_for, base, +loop_var, body, rest, cont}`. Under the flat layout: `{:cps_numeric_for, +base, loop_var, body_pc, after_loop_pc}`. + +### Compatibility + +This is a breaking change to `Lua.Compiler.Prototype` and the executor's +internal protocol. Public API (`Lua.eval!/1,2,3`, `Lua.load_chunk!/2`, +`Lua.VM.execute/2`) stays unchanged. + +`Lua.Chunk` wraps `%Prototype{}` — that struct's field types change, so +any saved/loaded chunks on disk would be invalid. We don't ship chunk +persistence and the dialyzer specs already mark `Prototype` internal. + +### Files + +- `lib/lua/compiler/prototype.ex` — add `labels` field, change + `instructions` type to `tuple()`. +- `lib/lua/compiler/codegen.ex` — emit a tuple instead of a list, + resolve labels to PCs in a final pass, lift loop bodies into PC + ranges. +- `lib/lua/vm/executor.ex` — rewrite `do_execute/8` (currently takes + an instruction list) to take `(pc, instrs, ...)`. Remove + `find_label/2` and `find_loop_exit/1`. +- `lib/lua/vm.ex` — update entry point to pass PC=0 and the instructions + tuple. +- `test/lua/compiler/integration_test.exs` — instruction-shape + assertions need updates. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 + +# Microbenchmarks — capture before/after numbers in the PR body. +mix run benchmarks/fibonacci.exs +mix run benchmarks/oop.exs +mix run benchmarks/table_ops.exs + +# Profile to confirm do_execute self-time dropped. +mix profile.tprof -e 'lua = Lua.new(); {_, lua} = Lua.eval!(lua, "function fib(n) if n < 2 then return n end return fib(n-1) + fib(n-2) end"); Lua.eval!(lua, "return fib(22)")' +``` + +## Risks + +- The BEAM's pattern-match compiler may not produce a clean jump table + for the unified `case elem(instrs, pc) do` form. If the post-merge + profile shows no improvement (or worse, a regression), the structural + change isn't paying for itself and B5 (Erlang functions) is the + better lever. +- Loop body lifting is error-prone. Nested loops, `break` jumping out + of nested loops, `goto` jumping into/out of loop bodies all need to + resolve to the right PC. The existing CPS frame stack handles this + semantically; the change is purely representational. Extensive + test coverage exists (`for`, `while`, `repeat`, `break`, `goto` are + hammered in the suite) — lean on it. +- Tuple-based instructions are non-resizable. Codegen must emit the + full instruction stream before constructing the tuple; no streaming + emit. The codegen already builds the full list before returning, so + this is mostly a `List.to_tuple/1` at the end. +- One-time codegen cost goes up (tuple construction is O(n)). For + long-lived chunks this is amortized; for one-shot `Lua.eval!` of + short scripts it could be measurable. Stretch goal: codegen time + does not regress by more than 5% on `Lua.eval!("return 1+1")`. + +## Discoveries + +(populated during implementation) diff --git a/.agents/plans/B5-compile-prototypes-to-erlang.md b/.agents/plans/B5-compile-prototypes-to-erlang.md new file mode 100644 index 00000000..3de78007 --- /dev/null +++ b/.agents/plans/B5-compile-prototypes-to-erlang.md @@ -0,0 +1,254 @@ +--- +id: B5 +title: Compile Lua prototypes to Erlang functions (executor JIT) +issue: null +pr: null +branch: perf/compile-to-erlang +base: main +status: blocked +direction: B +unlocks: + - sub-Luerl latency on tight numeric/call workloads + - "perf parity with Luerl, ±10%" 1.0 commitment headroom +--- + +## Blocked on + +- B4 — the flat instruction stream is the natural intermediate + representation to translate into Erlang. Trying to JIT directly from + the list-of-tuples shape would mix two structural changes in one PR. + +## Goal + +For each compiled `Prototype`, generate an Erlang function +(`:erlang.compile` / `:dynamic_compile` / a module-per-script) whose +body is the prototype's instruction stream rendered as Erlang code. +Dispatch becomes a regular Erlang call instead of an interpreter loop; +the BEAM's JIT (BEAMASM on OTP 25+) then natively optimizes the hot +path. + +This is the architectural lever Luerl uses (`luerl_comp_*` compiles to +Erlang abstract code and `compile:forms/2`s it). It's the single +biggest gap between an interpreter loop and a "Lua VM" with serious +throughput. + +## Why now + +After B4, the interpreter dispatch loop is a `case elem(instrs, pc)`. +Even with the BEAM's jump-table optimization, every opcode pays for +the case scrutiny, the tuple destructure, and the implicit `pc + 1` +recursion. At the limit, the cost floor is set by the dispatch shape +itself, not by anything inside any individual handler. + +A function-per-prototype layout pushes opcode selection from runtime +into compile time: `:add` becomes a literal `R0 = R1 + R2` in the +generated Erlang. The BEAM's static analyzer can then inline, fold, +and register-allocate within each prototype. + +The current Lua/Luerl gap on fib(25) is ~1.13x. PUC-Lua is ~50x faster +still. Closing meaningfully on PUC-Lua needs codegen-to-Erlang; staying +within ±10% of Luerl probably needs at least the framework even if not +every opcode is migrated initially. + +## Out of scope + +- A full custom JIT or NIF-backed executor. We stay on the BEAM. +- Migrating every opcode in one PR. Phase the work: arithmetic and + control flow first (highest dispatch density), then table ops, then + metamethod dispatch, then native calls. +- Eliminating the interpreter path. The interpreter remains as a + fallback for opcodes the compiler hasn't implemented yet, and for + any prototype that opts out (e.g. debug mode). +- Cross-prototype optimization (inlining one Lua function into another). + +## Success criteria + +- [ ] `Lua.Compiler.Erlang` module exists and converts a `%Prototype{}` + into an Erlang module (or anonymous function) that, when invoked + with `(registers, upvalues, state, varargs)`, returns + `{results, registers, state}`. +- [ ] At minimum these opcodes are compiled (not interpreted): + `:load_constant`, `:move`, `:add`, `:subtract`, `:multiply`, + `:less_than`, `:less_equal`, `:greater_than`, `:greater_equal`, + `:equal`, `:not_equal`, `:test`, `:return` (count=1), `:goto`, + `:label`, `:numeric_for`. Together these cover the fib hot path. +- [ ] Other opcodes fall back to the interpreter via a synthesized + `interpret_from_pc(pc, ...)` call. No opcode is unsupported. +- [ ] `mix test` passes; suite pass count does not regress. +- [ ] Microbenchmarks improve. Stretch target: **fib(25) reaches parity + with Luerl (±5%)** — currently ~1.13x slower. +- [ ] Compiled-mode failures fall back to interpretation, not crash. +- [ ] Generated modules are garbage-collected when no prototype holds + them. (Critical — otherwise long-running embedders leak code.) + +## Implementation notes + +### Strategy + +Generate Erlang **forms** (abstract syntax) and pass them to +`compile:forms/2` (or `:dynamic_compile.from_string/1`). Don't try to +generate Erlang source text — abstract forms skip the parser and are +the documented compiler entry point. + +```erlang +%% One generated module per Prototype. +-module(lua_proto_47). +-export([execute/4]). + +execute(Regs, Upvalues, State, Varargs) -> + %% R0..Rmax inlined as Erlang variables + R0 = element(1, Regs), + R1 = element(2, Regs), + ... + %% Instructions emitted in order, with control flow via goto/labels + %% mapped to Erlang case/recursion patterns. + Sum = R1 + R2, + Regs2 = setelement(3, Regs, Sum), + ... + {[ResultValue], Regs2, State}. +``` + +Reality is messier — Erlang has no `goto`, so labels become +function-letrec'd helpers or a single tail-recursive dispatch over the +remaining-instructions list (similar to today's interpreter, but +specialized per prototype). + +### Module lifecycle + +Each compiled prototype gets a unique module name (e.g. +`lua_proto_` or `lua_proto_`). Two paths: + +1. **`:code.load_binary/3`** with a generated `.beam` blob — fast load, + but loaded modules are global and persist until explicitly purged. +2. **Anonymous function via `erl_eval`** — no module loading, but the + BEAM doesn't JIT-compile interpreted funs as aggressively. Likely + slower than the loaded-module path. + +Recommend (1) with a per-prototype lifecycle: + +- Module name carries the prototype's content-addressable hash, so + identical prototypes share a module. +- A ref-counted registry tracks live prototypes; when the count hits + zero, `:code.purge/1` + `:code.delete/1` cleans up. +- The registry lives in a dedicated `Lua.VM.CodeCache` GenServer. + +This is the part most likely to leak in long-running embedders if done +wrong. Test it explicitly: create N prototypes, drop references, force +GC, confirm modules are purged. + +### Fallback path + +For opcodes not yet compiled, the generated module emits a call to +`Lua.VM.Executor.execute_from_pc/N` with the saved interpreter state. +The interpreter resumes from that PC and runs until either: + +- A return / frame pop transfers control back to the compiled module + (via a return-continuation passed in), or +- The whole call completes. + +This lets us land the framework with arithmetic + control flow +compiled, ship measurable wins, and migrate remaining opcodes +incrementally as separate plans (`B5a` for table ops, `B5b` for +metamethods, etc.). + +### Register representation + +Two options inside the compiled function: + +1. **Keep registers as a tuple**, use `element/2` and `setelement/3` — + minimal divergence from the interpreter. The BEAM can't avoid the + per-write `setelement` allocation. +2. **Promote each register to an Erlang variable** — true SSA, the + BEAM's optimizer can fold and register-allocate. Requires + single-assignment IR (rebinding `R0` becomes `R0_1`, `R0_2`, ...). + +Option (2) is the bigger payoff but requires SSA conversion in the +codegen. Option (1) ships sooner and still buys the dispatch-loop +elimination. Recommend (1) for the first PR; SSA conversion is a +follow-up plan (`B5c`). + +### Files + +- `lib/lua/compiler/erlang.ex` (new) — abstract-forms generator. +- `lib/lua/vm/code_cache.ex` (new) — ref-counted module registry. +- `lib/lua/compiler/prototype.ex` — add `:compiled_module` field, nil + when interpretation-only. +- `lib/lua/vm.ex` — when executing, dispatch to compiled module if + present, else interpreter. +- `lib/lua/compiler.ex` — opt-in flag to skip compilation (for + debugging, error-message fidelity). +- `test/lua/compiler/erlang_test.exs` (new) — compiles small prototypes, + asserts identical results vs interpreter. +- `test/lua/vm/code_cache_test.exs` (new) — module lifecycle tests. + +### Error message fidelity + +The interpreter currently raises with precise `line:` and `source:` +info threaded via the `line` parameter. Compiled mode must preserve +this. Options: + +- Inline `Process.put(@position_key, {line, source})` calls before any + potentially-raising opcode. Adds per-op overhead but matches + interpreter exactly. +- Emit a `try/catch` wrapping each compiled call that re-raises with + the right line info from a pc-to-line table. + +Recommend the try/catch approach — pays the cost only on the failure +path. The interpreter pays it on every native call anyway. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 + +# Benchmark before / after. +mix run benchmarks/fibonacci.exs +mix run benchmarks/oop.exs +mix run benchmarks/closures.exs +mix run benchmarks/table_ops.exs + +# Module lifecycle stress. +mix run -e ' +for _ <- 1..1000 do + lua = Lua.new() + {_, lua} = Lua.eval!(lua, "function f(n) return n + 1 end") + Lua.eval!(lua, "return f(42)") +end +:erlang.garbage_collect() +IO.inspect(:code.all_loaded() |> length(), label: "loaded modules after 1000 evals") +# Should be roughly constant, not growing by 1000. +' +``` + +## Risks + +- **Code load is slow.** `compile:forms/2` is hundreds of microseconds + per module. Embedders that compile-and-throw-away scripts (one-shot + `Lua.eval!`) may be net slower if compilation cost > execution cost. + Mitigation: opt out via flag, content-addressable module cache so + repeated compilation of the same source is free. +- **Module table is a global resource.** A misbehaving embedder that + compiles unique prototypes faster than they're collected exhausts + the atom table or `code:` server memory. Mitigation: ref-counted + registry + hard cap with eviction. Document the cap in the readme. +- **Hot reload / `mix test` interaction.** Compiled prototypes survive + module recompilation; stale modules could be referenced from in-flight + states. Mitigation: include a build hash in module names; reject + loading an old-build module into a new-build VM. +- **Error backtraces become harder to read.** Generated module names + in stack traces are noise to embedders. Mitigation: stack trimming + in `Lua.RuntimeException` (already trims VM internals; just extend + the prune list to include `lua_proto_*` modules). +- **The 1.13x current gap may not be reachable from here alone.** If + most of the remaining gap is `setelement` churn and not dispatch + overhead, this plan delivers less than expected. The B4 profile is + the gate — if B4 doesn't move `do_execute` self-time below ~30%, + reconsider whether B5 is the right next step or whether B6/B7 + (data structure reductions) come first. + +## Discoveries + +(populated during implementation) diff --git a/.agents/plans/B6-direct-table-refs.md b/.agents/plans/B6-direct-table-refs.md new file mode 100644 index 00000000..c5b721cf --- /dev/null +++ b/.agents/plans/B6-direct-table-refs.md @@ -0,0 +1,212 @@ +--- +id: B6 +title: Eliminate per-access Map.fetch! for table dereferencing +issue: null +pr: null +branch: perf/direct-table-refs +base: main +status: ready +direction: B +unlocks: + - measurable reduction in Map.get/2,3 cost across all table-heavy + workloads + - cleaner future ground for B7 (array+hash split) +--- + +## Goal + +Every read or write on a `{:tref, id}` value currently re-fetches the +table struct from `state.tables` via `Map.fetch!(state.tables, id)`. +For nested operations (table-of-tables, metatable chains) this fires +multiple times per opcode. Profiling shows `Map.get/2` + `Map.get/3` is +~6.4% of fib(22) and ~2.6% of the OOP workload — pure indirection +overhead. + +The fix: pass the resolved `%Lua.VM.Table{}` struct (or a more compact +mutable representation) directly through the operations that need it, +instead of re-resolving by ID. The tref atom remains the externally +visible identifier — only the executor's internal dispatch changes. + +## Why now + +This is independent of B4/B5 (dispatch shape) and pays back on every +table workload regardless of how the executor's outer loop is +structured. Pulling it forward also reduces noise in subsequent B4/B5 +profiles — every Map.fetch! that shows up there is a real signal, not +incidental indirection. + +## Out of scope + +- Eliminating `state.tables` entirely. The map of id-to-table is still + the source of truth; we're just avoiding re-resolution when the + resolved struct is already in hand. +- Switching to mutable references (ETS, `:persistent_term`, + process-dict). The state monad is preserved; this is a representation + change inside the executor, not a semantic change. +- Changing how `Lua.VM.Table` itself stores data. B7 may revisit that. +- Changing the public tref shape. `{:tref, id}` stays as the value Lua + code passes around. + +## Success criteria + +- [ ] `Lua.VM.Executor.table_index/3,4` and `table_newindex/4,5` accept + a resolved table struct (or a thin handle that carries it) where + they currently re-resolve from state. +- [ ] The hot opcode handlers `:get_field`, `:set_field`, `:get_table`, + `:set_table`, `:set_list`, `:self` resolve the table struct once + per opcode, then pass it down. +- [ ] Metatable chain traversal (in `index_value`, `table_newindex`, + `lookup_metamethod`) resolves each metatable once per step, not + twice. +- [ ] `mix test` passes; suite pass count does not regress. +- [ ] Profile after merge: combined `Map.get/2 + Map.get/3` self-time + drops below 3% on fib(22) (currently ~6.4%). +- [ ] Microbenchmarks improve. Stretch target: **10% reduction in + table_build+sum median** (currently ~180μs, target ~160μs). + Floor: no workload regresses by more than 2%. + +## Implementation notes + +### Current pattern (illustrative) + +```elixir +# Lua.VM.Executor.table_index/4 +defp table_index({:tref, id}, key, state, depth) do + table = Map.fetch!(state.tables, id) # fetch #1 + + case Table.get_data(table.data, key) do + nil -> + case table.metatable do + {:tref, mt_id} -> + mt = Map.fetch!(state.tables, mt_id) # fetch #2 + case Map.get(mt.data, "__index") do + {:tref, _} = idx_tbl -> + table_index(idx_tbl, key, state, depth + 1) # fetch #3 + ... + end + ... + end + v -> {v, state} + end +end +``` + +Three `Map.fetch!`s for a two-level metatable chain. + +### Proposed pattern + +Two viable shapes: + +1. **Pass the resolved struct alongside the tref.** Helpers take + `(tref, table_struct, ...)`. Slight signature churn but very + minimal change to data flow. + + ```elixir + defp table_index({:tref, _} = tref, table, key, state, depth) do + case Table.get_data(table.data, key) do + nil -> + case table.metatable do + {:tref, _} = mt_tref -> + mt = Map.fetch!(state.tables, elem(mt_tref, 1)) + ... + end + v -> {v, state} + end + end + ``` + +2. **Pass `{:tref, id, struct}` "fat refs"** through the executor's + internal channels (registers stay as `{:tref, id}` — Lua code never + sees the fat form). Helpers can be `tref`-shape-only. This requires + thinking carefully about when to re-resolve (writes invalidate the + struct). + +Recommend (1). It's clearer, easier to audit, and doesn't introduce a +parallel "internal vs external" tref representation that future +contributors have to track. + +### Write-through + +When a helper takes a resolved struct and then writes to the table, the +new struct must be committed back to `state.tables` before the next +read that depends on the change. The cleanest discipline: helpers +return both the new struct and the new state, callers always thread +state forward. The `set_field`/`set_table` fast paths landed in +PR #223 already do this — generalize the pattern. + +### Where Map.fetch! still belongs + +The opcode dispatch (`:get_field` etc.) does need exactly one +`Map.fetch!(state.tables, id)` per opcode entry — that's the +authoritative resolution. The goal is one fetch per opcode, not zero. + +The metatable chain is where the savings concentrate: a chain of N +metatables currently costs N+1 fetches; this plan brings it to 1 +fetch plus an internal walk that doesn't re-resolve. + +### Files + +- `lib/lua/vm/executor.ex` — `table_index/4,5`, `table_newindex/4,5`, + `index_value/5`, `lookup_metamethod/3`, `get_metatable/2`. +- `lib/lua/vm/state.ex` — possibly add a `get_table_struct/2` alias + that documents the contract. +- `lib/lua/vm/stdlib.ex` — anywhere stdlib re-resolves a tref it + already holds (e.g. iterating in `table.concat`, `table.sort`). + +### Audit checklist + +- [ ] grep `Map.fetch!(state.tables` — every site is a candidate. + Most should be the opcode entry points only. +- [ ] grep `state.tables\[` — same. +- [ ] grep `State.get_table` — same. Callers that already have the + tref's resolved struct shouldn't be calling this. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 + +# Profile to confirm Map.get drop. +mix profile.tprof -e ' +fib = "function fib(n) if n < 2 then return n end return fib(n-1) + fib(n-2) end" +lua = Lua.new() +{_, lua} = Lua.eval!(lua, fib) +{chunk, _} = Lua.load_chunk!(lua, "return fib(22)") +Lua.eval!(lua, chunk) +' +# Confirm `Map.get/2` and `Map.get/3` together are < 3% of total. + +# Microbenchmarks +mix run benchmarks/fibonacci.exs +mix run benchmarks/table_ops.exs +mix run benchmarks/oop.exs +``` + +## Risks + +- **Stale struct after mutation.** If helper A reads the table struct, + calls helper B which mutates the same table via state, then helper A + uses its stale struct — subtle correctness bug. Defense: any helper + that calls into something which threads state must either re-read + the struct after the call or not need it after the call. Test + coverage: metatable `__newindex` chains that mutate the parent table. +- **Signature churn touches many call sites.** The interpreter is + ~2300 lines; this is a wide refactor. Mitigation: dialyzer + the + full test suite catch shape mismatches; land the change with the + audit checklist completed. +- **Stdlib helpers that take only a tref now need state too.** Public + helpers like `Lua.VM.Executor.table_index/3` are documented; their + signatures should stay compatible. Add internal `_with_struct/N` + variants and keep the public `/3` arity calling the new internals + with a fresh resolution. +- **Diminishing returns.** If the Map.get cost is bound up in the + `table.data` per-key Map.get rather than the per-tref Map.fetch!, + this plan moves less than expected. That cost is structural to using + Erlang maps for table data and is B7's territory. + +## Discoveries + +(populated during implementation) diff --git a/.agents/plans/B7-table-array-hash-split.md b/.agents/plans/B7-table-array-hash-split.md new file mode 100644 index 00000000..3a9c92dc --- /dev/null +++ b/.agents/plans/B7-table-array-hash-split.md @@ -0,0 +1,253 @@ +--- +id: B7 +title: Split table storage into array + hash parts +issue: null +pr: null +branch: perf/table-array-hash-split +base: main +status: ready +direction: B +unlocks: + - O(1) `t[#t + 1] = x` (supersedes A10b) + - amortized O(1) `#t` length + - lower memory ratio vs Luerl on table-heavy workloads + - ipairs becomes a tight loop, not a Map walk +--- + +## Goal + +Reshape `Lua.VM.Table` so contiguous integer keys (the "array part") +live in a tuple-backed structure and other keys (the "hash part") stay +in the current map. This is what PUC-Lua and Luerl both do internally. + +For the common case `t = {1, 2, 3, ...}` or `for i = 1, n do t[i] = ...`, +this: + +- Cuts memory: the array part is a tuple of N values plus an integer + length, not an N-entry map (~6x smaller per element). +- Cuts time: integer-indexed reads are `elem/2`, integer-indexed + contiguous writes are `setelement/3` or `Tuple.append/2`. No map + hashing, no key normalization. +- Makes `#t` O(1) (length is stored, not scanned). +- Makes `ipairs` a fast loop over the tuple, not a sequence of + `Map.get` calls. + +## Why now + +The B-series wins so far closed the per-opcode dispatch gap. The +remaining table-workload gap vs Luerl (~1.27x slower on +table_build+sum) is dominated by allocation churn and Map operations +specific to using a single Erlang map as the backing store. No amount +of dispatch optimization will close that — the data shape itself is +the bottleneck. + +The memory ratio is the more glaring problem. The original benchee run +showed this library at **15,989x** memory vs C Lua and **5x more +memory than Luerl** on table_build. The array part lands a meaningful +chunk of that ratio. + +A10b (the deferred plan for `big.lua` perf) describes a smaller form +of this — cached sequence length only. This plan supersedes A10b: +shipping the full array+hash split delivers the sequence-length win +"for free" while also unlocking the broader benchmark improvements. +A10b stays open as a fallback if this plan is judged too large for one +PR; in that case A10b ships first as the sequence-length subset. + +## Out of scope + +- Migrating to a "true" mutable array via NIFs or ETS. We stay on the + BEAM with immutable tuples. +- Implementing Lua 5.4 borrowing semantics, weak tables, or finalizers. +- Reusing this representation for stdlib internal tables (string, + math, io). Those are stable-shape, hash-only tables; the array part + adds overhead with no payoff. +- The hashmap-side encoding. We keep `data: %{}` for non-integer keys. + +## Success criteria + +- [ ] `Lua.VM.Table` carries an `array :: tuple()` and `array_len :: + non_neg_integer()` field. Contiguous integer keys `1..array_len` + live in the tuple; gaps and non-integer keys live in `data`. +- [ ] `Lua.VM.Value.sequence_length/1` is O(1) — returns `array_len` + when the array part has no holes, falls back to today's logic + only when the array is empty or non-contiguous. +- [ ] `t[i]` for integer `i` in `1..array_len` is `elem(array, i)` — + no `Map.get`, no `normalize_key`. +- [ ] `t[#t + 1] = v` is amortized O(1): if `i == array_len + 1`, + append via `Tuple.append`, else fall back to `data` map. +- [ ] `ipairs(t)` iterates the tuple directly. +- [ ] `mix test` passes; `mix test --only lua53` does not regress. +- [ ] Microbenchmarks improve. Stretch targets: + - **table_build+sum (n=500): 30% faster** (currently ~180μs). + - **`big.lua` completes within 30 seconds** (currently times out + per A10b). + - **Table memory usage drops to 2-3x Luerl, from current 5x.** +- [ ] No workload regresses by more than 2%. + +## Implementation notes + +### Data layout + +```elixir +defstruct [ + array: {}, # tuple of values for keys 1..array_len + array_len: 0, # contiguous prefix length (#t value) + data: %{}, # hash part for non-integer keys, holes, large ints + order: [], # iteration order for hash part (B0 fix preserved) + order_tail: [], + dead: %{}, + metatable: nil +] +``` + +The `data` map remains the catchall for non-integer keys, integer keys +outside `1..array_len`, and any integer key written into a hole that +doesn't extend the contiguous prefix. + +### Promotion / demotion rules + +Per PUC-Lua's model, with simplifications: + +- **Write `t[i] = v` where `i == array_len + 1` and `v != nil`:** + append to `array`, increment `array_len`. Then check if any key + `i + 1` exists in `data` — if so, "absorb" it into `array` and + increment again (handles fill-in-the-blank patterns). +- **Write `t[i] = v` where `1 <= i <= array_len`:** `setelement(i, ...)`. + If `v == nil`, shrink `array_len` to `i - 1`, push the tail back to + `data`. (Lua semantics: `nil` removes the key.) +- **Write `t[i] = v` where `i > array_len + 1` or `i < 1` or `i` is + not an integer:** goes to `data`. +- **Read `t[i]` where `1 <= i <= array_len`:** `elem(array, i)`. +- **Read `t[i]` outside that range:** check `data`. + +### Hot-path read + +```elixir +def get(table, key) when is_integer(key) and key >= 1 and key <= table.array_len do + :erlang.element(key, table.array) +end + +def get(table, key) do + Table.get_data(table.data, key) +end +``` + +That's the entire fast path. Two function clauses, guard-tested, +no map lookup at all for in-range integer reads. + +### Sequence length + +```elixir +def sequence_length(%__MODULE__{array_len: n, data: data}) do + if map_size(data) == 0 do + n + else + # Fall through to current logic only when there might be a + # contiguous extension past the array part via the hash side. + sequence_length_with_hash(n, data) + end +end +``` + +For tables built linearly via `t[i] = x` or `t = {1, 2, 3}`, `data` is +empty and `array_len` is authoritative — O(1). + +### Migration + +The struct field changes break any caller that constructs `%Table{}` +literally with `data:` keyword. Audit: + +- `Lua.VM.Table.from_data/1` — used by `State.alloc_table/2` and stdlib + init. Needs to split incoming map into array + hash. +- `Lua.VM.Table.replace_data/2` — used by `table.sort`. Needs to + rebuild array_len. +- Anywhere `table.data` is read directly — there are several call + sites; replace with a `Lua.VM.Table.get/2` API that hides the split. + +### Files + +- `lib/lua/vm/table.ex` — the bulk of the change. +- `lib/lua/vm/value.ex` — `sequence_length/1` becomes a struct read. +- `lib/lua/vm/stdlib.ex` — `table.insert`, `table.remove`, `table.sort`, + `table.concat`, `ipairs` iterator. Each gets a fast path against the + array part. +- `lib/lua/vm/executor.ex` — `:get_table`, `:set_table`, `:get_field`, + `:set_field`, `:set_list`, `:length` opcodes. Most already dispatch + through `Lua.VM.Table` helpers; the helper changes cascade. +- `test/lua/vm/table_test.exs` (new or expanded) — explicit tests for + promotion, demotion, ipairs over mixed arrays. + +### Phasing + +If review pressure demands a smaller first PR, split: + +- **B7a**: add `array` / `array_len` fields, route reads through them, + keep writes going through `data` (no perf win yet, just plumbing). +- **B7b**: route writes through the array part, add promotion logic. +- **B7c**: fast-path stdlib (`ipairs`, `table.insert`, etc.). + +Default to landing as one plan unless a discovery makes phasing +necessary. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 + +# Specifically exercise the promotion rules. +mix test test/lua/vm/table_test.exs + +# Microbenchmarks +mix run benchmarks/table_ops.exs +mix run benchmarks/fibonacci.exs # confirm no regression on non-table workloads + +# big.lua suite test, which A10b was about +mix test --only lua53 test/lua53_suite_test.exs:NN # the big.lua test line + +# Memory check +mix run -e ' +script = "local t = {} for i = 1, 5000 do t[i] = i end return t" +lua = Lua.new() +{chunk, _} = Lua.load_chunk!(lua, script) +:erlang.garbage_collect() +{_, before_mem} = :erlang.process_info(self(), :memory) +Enum.each(1..100, fn _ -> Lua.eval!(lua, chunk) end) +:erlang.garbage_collect() +{_, after_mem} = :erlang.process_info(self(), :memory) +IO.puts("delta=#{after_mem - before_mem}B") +' +``` + +## Risks + +- **Promotion edge cases.** Lua semantics say `t[k] = nil` removes the + key — splitting that into "shrink array part" vs "drop from hash" is + where reference Lua implementations have classically had subtle bugs. + Test coverage must include: setting middle of array to nil, then + reading length; building array out of order; writing nil at the + array boundary; reviving a hole. +- **Iteration order via `pairs`** must match Lua 5.3 §6.1: iteration + is "unspecified" but must visit each live key exactly once. The + array part walks 1..array_len then `order` walks the hash part. + Tests exist for this via `nextvar.lua` — lean on them. +- **`table.sort` is hash-only-style today.** The fast path becomes + "sort the array tuple, leave hash alone". `table.sort` operating on + a sparse table is a weird case; the current implementation works + there. Make sure the split doesn't regress sparse-table sort. +- **Memory savings depend on workload shape.** Tables used as records + (`{name = "x", id = 1}`) don't benefit — they have no array part. + The fix doesn't make those slower, but the headline number is + for sequential-integer-key workloads. +- **`Tuple.append/2` is O(n) under the hood.** Repeated appends to a + large tuple are O(n²). Mitigation: pre-allocate when the size is + known (e.g. `t = {1, 2, 3, ..., N}` in source code → codegen emits + a single sized-tuple construction), use exponential growth for + dynamic appends. PUC-Lua doubles array capacity on grow; we should + too. + +## Discoveries + +(populated during implementation) diff --git a/.agents/plans/B8-inline-numeric-narrowing.md b/.agents/plans/B8-inline-numeric-narrowing.md new file mode 100644 index 00000000..ced56f6f --- /dev/null +++ b/.agents/plans/B8-inline-numeric-narrowing.md @@ -0,0 +1,158 @@ +--- +id: B8 +title: Inline `to_signed_int64/1` for the in-range fast path +issue: null +pr: null +branch: perf/inline-numeric-narrowing +base: main +status: ready +direction: B +unlocks: + - small but free win on all integer-arithmetic workloads +--- + +## Goal + +`Lua.VM.Numeric.to_signed_int64/1` is called on every integer arithmetic +result to wrap into signed 64-bit per Lua 5.3 §3.4.1. In the fib(22) +profile it accounts for **3.3%** of total time (85,968 calls). + +For the overwhelming common case where the result is already in +`[-2^63, 2^63 - 1]`, the masking and conditional subtraction are +wasted work — the input passes through unchanged. Adding a guarded +fast-path clause that returns the input as-is when it's already in +range eliminates the cost on that branch entirely. + +## Why now + +This is a small, self-contained, mechanical change. It has minimal +risk and no architectural overlap with B4-B7. It can ship before or +after any of them. Useful to have in the trunk so subsequent +benchmarks aren't muddied by this overhead. + +## Out of scope + +- Bypassing `to_signed_int64/1` calls entirely (the call sites still + call it; this plan only makes the function itself faster on the + common path). +- Changing the Lua wrap-around semantics. Behavior is identical. +- Turning the call sites into inline arithmetic at the executor level. + That would tangle with B5 and is not the right place to do it. + +## Success criteria + +- [ ] `Lua.VM.Numeric.to_signed_int64/1` has a guard-clause fast path + for inputs already in the signed 64-bit range. +- [ ] `Lua.VM.Numeric.signed?/1` is `@compile {:inline, signed?: 1}` + so the fast-path guard is cheap. +- [ ] `mix test` passes. +- [ ] Profile after merge: `Numeric.to_signed_int64` self-time drops + below 1.5% on fib(22). +- [ ] Microbenchmarks: fib(25) median improves by **3%+ stretch, 1% + floor**. +- [ ] No regression on overflow-heavy tests (the wrap-around path). + +## Implementation notes + +### Current implementation + +```elixir +@uint64_modulus 0x10000000000000000 +@uint64_mask 0xFFFFFFFFFFFFFFFF +@sign_bit 0x8000000000000000 + +@max_int 0x7FFFFFFFFFFFFFFF +@min_int -0x8000000000000000 + +def to_signed_int64(n) when is_integer(n) do + masked = band(n, @uint64_mask) + if masked >= @sign_bit, do: masked - @uint64_modulus, else: masked +end + +def signed?(n) when is_integer(n), do: n >= @min_int and n <= @max_int +``` + +Every call does a `band`, a comparison, and a branch — even for `n = 7`. + +### Proposed implementation + +```elixir +def to_signed_int64(n) when is_integer(n) and n >= @min_int and n <= @max_int do + n +end + +def to_signed_int64(n) when is_integer(n) do + masked = band(n, @uint64_mask) + if masked >= @sign_bit, do: masked - @uint64_modulus, else: masked +end +``` + +The BEAM compiles guards into native instruction sequences; the +in-range check is essentially two compare-and-branches with no +function call overhead. The slow path stays exactly as today. + +### Compile-time inlining + +```elixir +@compile {:inline, signed?: 1, to_signed_int64: 1} +``` + +`to_signed_int64/1` is small enough to inline at call sites; the BEAM +can then see the guards and short-circuit hot callers. + +This is one of the few places where `@compile {:inline, ...}` is +clearly worth it — the function is called inside the dispatch loop on +every arithmetic op. + +### Files + +- `lib/lua/vm/numeric.ex` — add the guard clause, add `@compile`. + +### Test coverage + +The existing tests in `test/lua/vm/numeric_test.exs` (and the doctests +in the module) cover both branches. Adding the guard splits the +existing `to_signed_int64/1` clause; verify the doctests still pass. + +If no module test exists, the integer-overflow tests in +`test/lua/vm/integration_test.exs` (or wherever the §3.4.1 wrap-around +asserts live) are the regression net. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test --only lua53 + +# Confirm overflow path is exercised +mix run -e ' +lua = Lua.new() +{[result], _} = Lua.eval!(lua, "return 9223372036854775807 + 1") +IO.inspect(result, label: "max_int + 1 should wrap to min_int") +^ -9223372036854775808 = result +' + +# Profile to confirm Numeric drop +mix profile.tprof -e ' +fib = "function fib(n) if n < 2 then return n end return fib(n-1) + fib(n-2) end" +lua = Lua.new() +{_, lua} = Lua.eval!(lua, fib) +{chunk, _} = Lua.load_chunk!(lua, "return fib(22)") +Lua.eval!(lua, chunk) +' +``` + +## Risks + +- **Almost none.** The behavior is bit-for-bit identical; the fast + path is purely a guard-tested return-as-is. +- **`@compile {:inline, ...}` can cause subtle behavior with dialyzer + type inference**, but for a function with this signature + (integer in, integer out) it's safe. Confirm `mix dialyzer` (if the + CI runs it) doesn't gain new warnings. + +## Discoveries + +(populated during implementation)