Skip to content

fix(vm): align table key semantics with Lua 5.3 §3.4.11 - #200

Merged
davydog187 merged 3 commits into
mainfrom
fix/nextvar-suite
May 7, 2026
Merged

davydog187 merged 3 commits into
mainfrom
fix/nextvar-suite

Conversation

@davydog187

Copy link
Copy Markdown
Contributor

Fix nextvar.lua "concatenate nil" failure (partial)

Plan: .agents/plans/A7-nextvar-suite.md
Refs #168

This is the first half of plan A7. The second half — dead-key tracking
so next(t, k) survives t[k] = nil mid-iteration, plus the strict
"invalid key to 'next'" raise that depends on it — is split out to
plan A7a so this PR can land cleanly without regressing iteration.

Goal

Make nextvar.lua pass. The original "concatenate nil" symptom was
already gone after A1 landed; bisecting line by line surfaced four
distinct Lua 5.3 semantics gaps in the VM and stdlib, three of which
are fixed here.

Success criteria

  • mix test passes (1402 → 1420, no regressions; +18 new tests).
  • Each fixed issue has a unit test in test/lua/vm/nextvar_semantics_test.exs.
  • nextvar.lua passes end-to-end. Partial — three of four root
    causes fixed. The fourth (dead-key tracking for iterate-then-clear)
    requires a table-representation change and is deferred to plan A7a.

Changes

 lib/lua/vm/executor.ex                 |  17 ++--
 lib/lua/vm/state.ex                    |   2 +-
 lib/lua/vm/stdlib.ex                   |  24 ++++-
 lib/lua/vm/table.ex                    |  73 ++++++++++++++
 test/lua/vm/nextvar_semantics_test.exs | 152 +++++++++++++++++++++++++++++++++
 5 files changed, 255 insertions(+), 13 deletions(-)

A new Lua.VM.Table.put_data/3 helper centralizes the write-as-delete
and integer-coercion rules, with matching get_data/2 and has_data?/2
read helpers. Every table-data mutation now flows through them.

Discoveries (the four root causes)

  1. Assigning nil to a table key did not delete the key. set_table,
    set_field, set_list, table_newindex, and rawset all called
    Map.put(data, key, nil) instead of removing the entry. This violated
    Lua 5.3 §3.4.11 — fields with nil values are absent from the table.
    Consequences: #{nil} == 1 (should be 0), for k in pairs(t) do t[k] = nil end
    left every key visible, and next(t) returned keys whose values were nil.

  2. Float keys with an exact integer value were stored as floats.
    t[2.0] = v and t[2] = v lived in different slots, so the test at
    nextvar.lua:310 (a[#a] after for i=0,50 do a[2^i] = true end)
    read nil because 2^0 is 1.0. Lua 5.3 §3.4.11 mandates integer-valued
    floats collapse to integers as table keys.

  3. pairs and ipairs did not validate their argument. Calling them
    with no argument or with a non-table value crashed the BEAM with a
    FunctionClauseError instead of raising the Lua-level "bad argument"
    error that nextvar.lua:233-234 checks for.

  4. next(t, k) does not handle keys that were valid mid-iteration and
    are now nil.
    Real Lua keeps removed keys reachable in the hash chain
    (the "dead key" trick) so for k,v in pairs(t) do t[k] = nil end
    works. Without that, strict-next would error mid-iteration. The
    next error path here is left lenient (returns nil, nil) so this
    PR doesn't regress iteration; plan A7a will add dead-key tracking
    and the strict raise together.

Verification

mix format
mix compile --warnings-as-errors  # clean
mix test                          # 1420 tests, 0 failures, 31 skipped
mix test --only lua53             # 29 tests, 0 failures, 24 skipped

Out of scope (intentional)

  • Anything outside the table-key contract surfaced by nextvar.lua.
  • Dead-key tracking for next during iteration-with-deletion (A7a).
  • Promoting nextvar.lua from @skipped_tests to @ready_tests
    (deferred to A7a where the suite file actually passes end-to-end).
  • Stricter NaN-key handling. Table.invalid_key?/1 is added in this
    PR for the eventual NaN-key error path; A7a is the natural place to
    start using it.

davydog187 added 3 commits May 6, 2026 18:10
Three closely-related table-semantics bugs in the VM and stdlib:

* Assigning `nil` to a key did not delete the key, so `#{nil}` returned
  1, `t[k] = nil` left the entry visible to `pairs`, and `next(t)`
  returned a key whose value was nil.
* Float keys with an exact integer value lived in a different slot from
  their integer counterpart, so `t[2^0] = v` followed by `t[1]` read
  nothing.
* `pairs()` and `ipairs()` with no argument or a non-table argument
  crashed the BEAM with a `FunctionClauseError` instead of raising a
  Lua-level "bad argument" error.

A new `Lua.VM.Table.put_data/3` helper centralizes the write-as-delete
and integer-coercion rules, with matching `get_data/2` and `has_data?/2`
read helpers. Every table-data mutation in `executor.ex`, `state.ex`,
and `stdlib.ex` (rawset, rawget, next, set_global) now flows through
those helpers, so the contract holds uniformly across the four write
paths (set_table, set_field, set_list, table_newindex) and rawset.

Eighteen new unit tests in `test/lua/vm/nextvar_semantics_test.exs`
pin the §3.4.11 rules on small focused inputs so any future regression
points at the specific semantic that broke.

This is the first half of plan A7. The second half — dead-key tracking
so `next(t, k)` survives `t[k] = nil` mid-iteration, plus the strict
"invalid key" raise that depends on it — is split out to plan A7a.
With dead keys in place, `nextvar.lua` will pass end-to-end and can be
promoted to the ready set in `lua53_suite_test.exs`.

Plan: A7
Refs #168
A7 ships PR #200 with three of four nextvar.lua root causes fixed.
The remaining gap (dead-key tracking so `next(t, k)` survives
`t[k] = nil` mid-iteration) is captured as plan A7a so `nextvar.lua`
can be promoted to the ready set there.
@davydog187
davydog187 merged commit 8bcb3c7 into main May 7, 2026
4 checks passed
@davydog187
davydog187 deleted the fix/nextvar-suite branch May 7, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant