Skip to content

feat(A0): 64-bit integer overflow wrapping for arithmetic and bitwise ops - #177

Merged
davydog187 merged 3 commits into
mainfrom
feat/int-overflow-wrapping
May 5, 2026
Merged

davydog187 merged 3 commits into
mainfrom
feat/int-overflow-wrapping

Conversation

@davydog187

Copy link
Copy Markdown
Contributor

64-bit integer overflow wrapping for arithmetic and bitwise ops

Plan: .agents/plans/A0-int-overflow-wrapping.md
Closes #161

Goal

Make integer operations wrap to signed 64-bit per the Lua 5.3 spec, so maxint + 1 == minint and 1 << 63 == minint instead of overflowing into arbitrary-precision Erlang integers.

This is a deliberate divergence from Luerl, which inherits Erlang's bignum semantics. The new VM conforms to Lua 5.3 §3.4.1 ("In case of overflows in integer arithmetic, all operations wrap around, according to the usual rules of two-complement arithmetic").

Success criteria

  • mix test passes — 1273 → 1284 (11 new tests, no regressions)
  • New unit tests in test/lua/vm/arithmetic_test.exs:
    • maxint + 1 == minint
    • minint - 1 == maxint
    • maxint * 2 == -2
    • 1 << 63 == minint
    • ~0 == -1
    • ~0xffffffffffffffff == 0
    • 0x8000000000000000 // 1 == minint (no overflow into bignum)
  • mix test --only lua53 count unchanged (29/0/25 → 29/0/25). bitwise.lua is in the deferred list (A5) and depends on stdlib bits beyond this plan's scope; this PR removes one of its blockers but does not flip it green on its own.

Changes

 .agents/plans/A0-int-overflow-wrapping.md |  2 +-
 lib/lua/vm/executor.ex                    | 31 +++++++++---------
 lib/lua/vm/numeric.ex                     | 85 +++++++++++++++++++++++++++++++
 test/lua/vm/arithmetic_test.exs           | 65 +++++++++++++++++++++++
 4 files changed, 170 insertions(+), 13 deletions(-)
  • New Lua.VM.Numeric module with to_signed_int64/1, max_int/0, min_int/0, and signed?/1. The module docstring documents the Lua-vs-Luerl divergence so it's discoverable from generated docs.
  • Lua.VM.Executor:
    • safe_add, safe_subtract, safe_multiply, safe_negate route integer-only results through a new private narrow_if_integer/1 helper (floats untouched).
    • safe_floor_divide and safe_modulo wrap their integer paths.
    • :bitwise_and, :bitwise_or, :bitwise_xor, :bitwise_not wrap their results.
    • lua_shift_left/2 and lua_shift_right/2 previously masked to unsigned 64-bit, leaving 1 << 63 as a positive bignum. Both now flow their result through to_signed_int64/1, so high-bit results land at minint instead.

Notes on Lua-vs-Luerl behavior

This is the first behavior change on the new VM where users coming from Luerl will see different output from the same Lua source. The divergence is intentional and matches the Lua 5.3 reference. We should call this out in CHANGELOG / release notes for 0.5.0; that documentation update is out of scope here and belongs with A12.

Verification

mix format
mix compile --warnings-as-errors
mix test                  # 1284 tests, 0 failures, 32 skipped
mix test --only lua53     # 29 tests, 0 failures, 25 skipped
mix test test/lua/vm/arithmetic_test.exs  # 32 tests, 0 failures

Out of scope (intentional)

  • Float arithmetic (already IEEE 754).
  • Float-to-integer coercion edge cases.
  • Compiler/codegen changes.
  • Triage of remaining bitwise.lua failures unrelated to overflow — that is the A5 plan.

davydog187 added 3 commits May 5, 2026 06:17
Lua 5.3 §3.4.1 requires integer operations to wrap modulo 2^64 to the
signed 64-bit range. Without wrapping, results overflow into Erlang's
arbitrary-precision integers, breaking expectations like
`maxint + 1 == minint` and `1 << 63 == minint`. This conforms to
the Lua reference and is a deliberate divergence from Luerl, which
inherits Erlang's bignum semantics.

Adds `Lua.VM.Numeric` with `to_signed_int64/1` and applies it to
integer-only results in arithmetic ops (`+`, `-`, `*`, `//`, `%`,
unary `-`) and all bitwise ops (`&`, `|`, ~^`, `~`, `<<`, `>>`).
Floats are untouched. The previously partial wrapping in
`lua_shift_left`/`lua_shift_right` (masked to unsigned 64-bit, but
left as positive bignums for high-bit results) now produces the
correct signed values.

Closes #161
@davydog187
davydog187 merged commit b3cc76e into main May 5, 2026
3 checks passed
@davydog187
davydog187 deleted the feat/int-overflow-wrapping branch May 5, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

64-bit integer overflow wrapping for arithmetic and bitwise ops

1 participant