Skip to content

perf(table): O(1)-per-step hash iteration via memoized order index - #349

Merged
davydog187 merged 2 commits into
mainfrom
perf/table-hash-iteration
Jun 11, 2026
Merged

davydog187 merged 2 commits into
mainfrom
perf/table-hash-iteration

Conversation

@davydog187

@davydog187 davydog187 commented Jun 10, 2026 •

Copy link
Copy Markdown
Contributor

What changed and why

lua_next/flush_order now memoizes the hash-key iteration order (an :array of keys plus a key => index map) on the first step of an iteration, so each subsequent next is O(1) instead of re-scanning from the front. This turns pairs over a large hash-keyed table from O(n^2) into O(n).

Motivated by an external performance review flagging the quadratic pairs path on hash-keyed (string-keyed) tables.

Benchmark delta

Scenario: pairs over a large hash-keyed table (sum all values via for k,v in pairs(t) where t has N string keys "k1".."kN", forcing the hash/order iteration path, not the integer array fast path).

Ref N Mode Avg ips
baseline origin/main @016fa35 2000 full 11.43 ms 87.49
branch @95ee3e4 2000 full 0.858 ms 1.17 K
baseline origin/main @016fa35 4000 quick 57.06 ms 17.52
branch @95ee3e4 4000 quick 1.59 ms 627.76
  • Headline delta (apples-to-apples N=2000, full mode): 0.858 ms vs 11.43 ms => -92.5% (13.3x faster).
  • Baseline shows super-linear scaling (O(n^2)): 2000->4000 time grows ~5x. Branch is linear (O(n)): per-element time roughly flat (1.40->1.59 ms quick mode).
  • Memo build adds ~0.18 MB peak memory (1.38 vs 1.20 MB).
  • Cross-check at N=4000 quick mode: 57.06 ms -> 1.59 ms = -97.2% (36x); gap widens with N, consistent with O(n^2)->O(n).

Measurement notes: Wrote a minimal Benchee script benchmarks/hash_pairs.exs (this-impl-only, no Luerl/luaport; builds T=build(N) once into a global, then measures the chunk return sum_pairs(T)). Removed it after the run; NOT committed to this branch. Ran on both refs via detached-HEAD checkout in a worktree, MIX_ENV=benchmark. Sanity: sum_pairs returns n*(n+1)/2 on both refs. Branch quick-mode N=2000 showed high deviation (+/-140%) on the now-sub-ms path, which is why the stable full-mode N=2000 figure is the reported headline. Env: Elixir 1.20.0 / Erlang 29 JIT, single run per ref (not multi-trial), so absolute ms carry normal single-machine noise; the order-of-magnitude speedup is unambiguous.

Review verdict

State: ready. Tests pass after changes. No fixes required. Two nits noted (no action taken):

  • nit: flush_order now eagerly allocates :array.from_list/1 and a key => index map on the first lua_next of every iteration. For tables iterated once via pairs this is pure win (O(n^2)->O(n)), but for a next(t, k) called a single time on a large table it does more upfront work than the old single advance_past scan. Not a regression in practice (pairs always iterates fully) and the memo is correctly skipped on subsequent steps; noting only as a micro-tradeoff.
  • nit: In a pairs loop that nils keys, dead keys remain in order_arr and first_live_from scans past them via Map.fetch on data. Total work stays O(n) across the loop (scan only advances forward), so the O(1)-amortized claim holds, but worst-case a single resumed step can scan past an accumulated run of dead keys. Documented correctly in the next_hash_after comment.

Merge

Human-gated merge: this PR is intentionally a draft. Do not merge without human review and sign-off.

Stabilization + independent verification

No stabilization changes were required — the PR shipped ready-as-is. All review findings were nit-severity; each was independently re-confirmed, not just taken on trust.

Correctness

  • Full mix test: 2255 passed, 0 failures (19 skipped).
  • Gate: Lua 5.3 official nextvar.lua passes unconditionally (no skip ranges).
  • New test/lua/vm/table_iteration_test.exs (8 tests) + existing nextvar/table unit tests (95) all green.
  • §6.1 dead-key resume verified directly: on a 60-key table, clearing k1..k50 then next(t, "k1") returns {k51, 51}; the full live walk visits exactly k51..k60; an absent key returns :invalid_key via the memo. Clearing a value does not invalidate the memo (resume point preserved); inserting/reviving a key does.

Performance (interpreter path, LUA_BENCH_MODE=full, n=10k)

Bytecode stripped to force the interpreter (:lua_closure), baseline = origin/main @016fa35.

workload main branch delta
pairs large hash (n=10k) 261.59 ms 4.76 ms ~55x faster
pairs small (n=8) 0.00312 ms 0.00333 ms flat (within noise)
pairs large array (n=10k) 2.00 ms 2.16 ms within run-to-run noise

Memory: large hash 14.04 MB → 15.05 MB (+1 MB, expected from the one-time order-array/index memo); small table +~1.6 KB (the author-flagged build-memo-for-tiny-tables nit; trivial). The array path is untouched by the hash memo, so its small delta is pure variance.

The O(n²)→O(n) win for interpreter-routed hash iteration holds; small-table and array iteration show no regression.

Replace the linear advance_past/2 scan in next_entry/2 with an O(1)
position lookup. flush_order/1 now builds a memoized order array
(order_arr) and a key => index map (order_index) once at the start of an
iteration, so each hash step is an index lookup plus a forward scan to
the next live key — making a full pairs loop O(n) instead of O(n^2). Any
structural mutation of the order list invalidates the memo; clearing a
value does not, preserving the Lua 5.3 §6.1 dead-key resume point.

Plan: D
@davydog187
davydog187 marked this pull request as ready for review June 10, 2026 22:41
Adds StreamData properties asserting that a full next_entry/2 walk
visits exactly the live key set once each, that the memoized order
index agrees with the list-based fallback, and that clearing the
current key mid-walk still visits every live key (the §6.1 dead-key
resume guarantee) on both the memo and fallback paths.
@davydog187

Copy link
Copy Markdown
Contributor Author

🤖 Ultracode review — round 1

Two findings, both nits — durability/coverage gaps in the new table-iteration tests, not correctness bugs. The reviewer confirmed the underlying behavior works by hand; these are about pinning it down with tests.

  • [nit] test/lua/vm/table_iteration_test.exs:55-58, 230-234 — Equivalence property never exercises dead (deleted) hash keys — the one case where the memo and fallback paths diverge in mechanism

    entries_gen generates values via integer(1..1000) (always non-nil) and build_pair folds Table.put(t, k, v), so the three StreamData properties only ever build insert-only tables — dead stays empty and no dead key ever sits in order/order_arr during a property run. That matters because the memo path and the fallback path resolve a resume key by structurally different means: next_hash_after with a memo does Map.get(order_index, key); the fallback does advance_past via === over merged_order. They only diverge in mechanism on a key that is dead (present in order_arr/order_index, absent from data) — exactly the §6.1 resume-point this PR's memo touches (hash delete at table.ex:172-178 marks the key dead but deliberately leaves the memo intact). The property walk(flush_order(t)) == walk(t) claims to prove memo==fallback equivalence but never probes that divergence point. The dead-key resume IS covered by the hand-written "cleared key stays a valid resume point" test, but only on the memo side and only on a fixed shape — not as a randomized memo-vs-fallback comparison.

    Suggestion: Have entries_gen occasionally emit nil values (e.g. generate ops as {:put, k, v} | {:del, k}) and update build_pair/the oracle to drop nil'd keys, so the existing equivalence and full-walk properties exercise dead keys sitting in order_arr and pin memo==fallback agreement on the §6.1 dead-key resume path.

  • [nit] test/lua/vm/table_iteration_test.exs:57-68, 233-244 — No regression test drives an insert or dead-key revival mid-walk (only cursor-key deletion is interleaved)

    walk_deleting only ever deletes the key just returned (the cursor), and a hash delete preserves order_arr/order_index, so the memo is never invalidated mid-iteration in any test. The distinct path where a mutation nils the memo between two next_entry steps — inserting a brand-new key, or reviving a dead key, both of which set order_index/order_arr to nil (table.ex insert_hash) — forces the stdlib flush gate (order_tail == [] and order_index != nil) to rebuild the memo on the following step. That mid-walk rebuild is the core "index goes stale after mutation during traversal" risk and is only covered as discrete unit tests (insert clears memo; revival rebuilds memo), never interleaved into a live walk. The reviewer confirmed it works by hand, so this is a durability gap, not a known bug.

    Suggestion: Add a test (ideally a property interleaving a random mutation at a random step) that inserts a new key, and separately revives a dead key, mid-walk and asserts the resumed next_entry returns a valid pair, never :invalid_key on a live cursor, and the loop still terminates.

@davydog187
davydog187 merged commit 7988aee into main Jun 11, 2026
5 checks passed
@davydog187
davydog187 deleted the perf/table-hash-iteration branch June 11, 2026 11:40
@davydog187

Copy link
Copy Markdown
Contributor Author

🤖 Ultracode review — round 1

Two minor findings, both confirmed against the PR head. No blockers.

  • [MINOR] test/lua/vm/table_iteration_test.exs:95-128 — No regression test for inserting/clearing a key mid-iteration then resuming the same walk

    The new suite covers delete-during-walk, value-clear (memo preserved), revival (memo invalidated), and insert-then-FRESH-walk-from-nil. But two load-bearing arms of the memo handshake are unguarded: (1) insert a NEW hash key mid-walk and resume from a previously-returned key (insert nils order_index, so the resume must fall back to advance_past and still not raise); (2) first_hash_live(nil) when the FIRST entry in order_arr is value-cleared while the memo is still live (the skip-loop in first_live_from must step past a dead leading key). Both behave correctly on the PR head (next_entry(t,"a") after inserting "zzz" returns {"b",2}; next_entry(cleared,nil) returns {"b",2} with order_index still set), so these are coverage gaps, not bugs. A buggy first_live_from returning arr[0] unconditionally would still pass every current test. The insert-then-resume arc is exactly the invalidate/rebuild path the optimization rests on.

    Suggestion: Add a Table-level test that calls next_entry(t,nil), inserts a new hash key, then next_entry(t,k1) and asserts it is not :invalid_key and the live set stays reachable. Add a second: put a,b; flush_order; put(a,nil) (memo stays live); assert next_entry(table,nil) == {"b",2} and order_index != nil. Optionally extend walk_deleting into a walk_mutating property that also inserts on some steps.

  • [MINOR] lib/lua/vm/stdlib.ex:790 — package.loaded caching mutates Table.data directly, bypassing put/3 order and memo bookkeeping

    cache_module_result/3 writes %{loaded_table | data: Map.put(...)} — the only writer of Table.data outside table.ex (confirmed by grep). It neither appends to order/order_tail nor nils order_index/order_arr. This is a pre-existing order-bypass bug (new modnames are invisible to pairs/next regardless of this PR). For THIS PR the incremental risk is near-zero: new keys were already absent from order so the memo would not include them either, and overwrites of existing keys still read the live value via Map.fetch in first_live_from. So it does not make iteration more wrong. Flagged only because it is the one place a stale memo could in principle escape the single maintained code path, adjacent to this PR's "any structural mutation nils the memo" invariant.

    Suggestion: Route through Table.put/3 (e.g. State.update_table(state, ref, &Table.put(&1, modname, result))) so order and memo invalidation happen via the single maintained path; this also fixes the pre-existing pairs-visibility gap for newly loaded modules. Follow-up, not a blocker for this PR.

@davydog187

Copy link
Copy Markdown
Contributor Author

🤖 Ultracode review — round 2

All four findings are confirmed against the PR head. None are blockers — the Round 1 fix is correct; these are coverage and cleanup gaps.

[MAJOR] lib/lua/vm/stdlib.ex:787 — No regression test pins pairs(package.loaded) visibility for newly-required modules

cache_module_result/3 was rerouted from a direct %{loaded_table | data: Map.put(...)} write to Table.put/3 so caching a required module flows through order/order_tail/memo maintenance and becomes visible to for k in pairs(package.loaded). This is the load-bearing behavioral half of the Round 1 fix, and nothing tests it: a grep across test/ for pairs.*loaded / for .* in pairs(package returns nothing, and package_test.exs asserts module presence only via direct key access. A future refactor reverting line 787 to the direct .data write (the exact pre-existing bug) would pass the entire suite silently.

Suggestion: Add a regression test (in package_test.exs or table_iteration_test.exs): require a module, then run local seen=false; for k in pairs(package.loaded) do if k=='mymod' then seen=true end end; return seen and assert [true]. This pins the iteration-visibility contract the reroute exists to provide.

[MINOR] benchmarks/table_ops.exs:6 — Optimized hash-iteration path (next/pairs) has no benchmark; the O(n) vs O(n²) claim is unmeasured in-repo

The optimization targets the hash arm of next_entry/2 (replacing the linear advance_past/2 scan with an O(1) order_index lookup), making a full pairs loop O(n) rather than O(n²). The only in-repo iteration benchmark, run_table_sum, builds a dense array t[i]=i and sums via numeric-index reads through the array arm — it never touches the hash advance path this PR changes. A grep for pairs( / next( across benchmarks/ returns nothing, so the central performance claim has no measurement backing and a regression reintroducing the linear scan would go uncaught.

Suggestion: Add (or extend table_ops.exs with) a benchmark that builds a large string/sparse-integer-keyed table and runs a full for k,v in pairs(t) do ... end, so the optimized hash path is measured and the O(n) claim is reproducible via the run-bench harness.

[MINOR] lib/lua/vm/table.ex:213 — Absorb-after-flush memo invalidation (drop_hash_key during array promotion) is untested

drop_hash_key/2 now nils order_index/order_arr and is invoked from absorb_from_hash/1 when a contiguous array append promotes a sparse integer key parked in the hash — a distinct memo-invalidation trigger from the insert_hash and delete arms the tests cover. The property test's key_gen only generates integer(1..12)/integer(100..110) and calls flush_order once at the end (never interleaving), so it never builds the memo and then triggers absorb of a parked key. A direct test at the PR head confirms the behavior is correct (memo nils, reflush walk [1,2,3,'s']) — this is a coverage gap, not a bug.

Suggestion: Add a unit test in table_iteration_test.exs for the absorb path: put(1,..), put(3,..) (parked in hash), put('s',..), flush_order (assert memo present), then put(2,..) to trigger absorb of key 3, assert order_index == nil, reflush, and assert the walk equals [1,2,3,'s'].

[NIT] lib/lua/vm/table.ex:223 — Dead put_data/3 is a latent bare-map writer that bypasses memo invalidation

After the Round 1 reroute of cache_module_result/3 through Table.put/3, git grep put_data finds zero callers in lib/ and test/ (only its own def). It writes/deletes on a bare data map with no order/order_arr/order_index bookkeeping. It cannot itself produce a stale %Table{} (it returns a map, not a struct), so the footgun is speculative — it only bites if a future caller wraps its output back into a struct. Still, it is dead public code sitting in the memo-coherence blast radius.

Suggestion: Delete put_data/3 so the single maintained mutation path through Table.put/3 is the only struct writer. If it must remain for a bare-map use case, add a doc line that its output must never be folded back into a %Table{} without resetting order/memo.

@davydog187

Copy link
Copy Markdown
Contributor Author

🤖 Ultracode review — round 3

Blockers

  • [BLOCKER] lib/lua/vm/stdlib.ex:784 — PR perf(table): O(1)-per-step hash iteration via memoized order index #349 head is stale: reviewers and CI see 128c73e, but the fixes live at branch tip 1a3c6b6
    • Verified on the live remote: git ls-remote origin perf/table-hash-iteration → 1a3c6b6, and that commit resolves via gh api, so the round-1/2 fixes are pushed to GitHub (contrary to the earlier "never pushed" claim). However gh api .../pulls/349 .head.sha is still 128c73e and the PR lists only 2 commits (95ee3e4, 128c73e). Therefore gh pr diff 349, the PR UI, and any PR-triggered CI evaluate 128c73e, which lacks: the cache_module_result→Table.put fix, the package.loaded pairs-visibility test, the absorb-path memo test, the insert/clear mid-walk resume tests, the replace_data resets, and the pairs_hash benchmark. The branch is fixed; the PR is not pointing at the fix.
    • Suggestion: Re-sync PR perf(table): O(1)-per-step hash iteration via memoized order index #349 to the current branch tip 1a3c6b6 (an empty push / branch update, or close+reopen, to force GitHub to advance .head.sha off 128c73e). Then confirm gh pr diff 349 shows the cache_module_result Table.put fix, package_test.exs:109 "a newly required module is visible to pairs(package.loaded)", the absorb / insert-resume / value-cleared-leading-key tests, and benchmarks/table_ops.exs run_table_pairs_hash. This single sync resolves the four downstream "absent from PR head" findings, which are all artifacts of the stale head rather than real code gaps.

Minor

  • [MINOR] lib/lua/vm/table.ex:36 — replace_data/from_data memo reset is unpinned by any iteration test
    • Confirmed against the remote tip 1a3c6b6 (not just the stale PR head): replace_data/2 and from_data correctly reset order_index/order_arr to nil (table.ex:36-37, and the split paths at 49-50, 63-64), but no test in table_iteration_test.exs or elsewhere does a replace_data followed by a pairs/next_entry walk — grepping replace_data|from_data over the test at the tip returns nothing. The other invalidation arms (insert_hash new-key, dead-key resurrect, drop_hash_key) all gained regression tests on the branch; replace_data is the one reset arm left uncovered, so a future omission of the reset would silently survive.
    • Suggestion: Add one unit test — flush_order a table to build a live memo, replace_data it with a new map, then assert walk(flush_order(t)) equals the new key set exactly once each (and order_index/order_arr are nil immediately after replace_data). Guards the replace path against a dropped memo reset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant