diff --git a/.agents/plans/A5-bitwise-suite.md b/.agents/plans/A5-bitwise-suite.md index 4a5fe32d..d6a9295c 100644 --- a/.agents/plans/A5-bitwise-suite.md +++ b/.agents/plans/A5-bitwise-suite.md @@ -2,10 +2,10 @@ id: A5 title: Fix bitwise.lua failing assertion issue: 166 -pr: null +pr: 198 branch: fix/bitwise-suite base: main -status: ready +status: review direction: A unlocks: - bitwise.lua @@ -24,10 +24,11 @@ assertion). ## Success criteria -- [ ] `mix test` passes (≥ 1273, no regressions) -- [ ] Each fixed assertion has a corresponding unit test in +- [x] `mix test` passes (≥ 1273, no regressions) +- [x] Each fixed assertion has a corresponding unit test in `test/lua/vm/bitwise_test.exs`. -- [ ] `bitwise.lua` passes end-to-end. +- [ ] `bitwise.lua` passes end-to-end. (Partially: lines 1–270 now pass. + Final third blocked on `math.fmod`, deferred to A5a.) ## Implementation notes @@ -61,4 +62,79 @@ mix test test/lua/vm/bitwise_test.exs ## Discoveries -(populated during implementation) +A0 fixed the integer arithmetic wrap but **not** integer literal lexing or +string-to-number coercion. Three distinct issues surfaced while running +`bitwise.lua`: + +1. **Hex integer literal wrapping (lexer).** `0xFFFFFFFFFFFFFFFF` was + parsed as the unsigned int `18446744073709551615`. Per Lua 5.3 §3.1, + hex integer literals overflow-wrap into the signed 64-bit range. Fixed + inline in `Lua.Lexer.scan_hex_number` (didn't reach for `Lua.VM.Numeric` + to keep the lexer free of VM-internal deps). + +2. **`Value.parse_number` (string → number coercion).** + - Same hex-overflow issue as the lexer. + - Did not handle a leading `-` or `+` (e.g. `"-0xfffffffffffffffe"` → + `nil`). `bitwise.lua` line 54-55 use this form for coercion via `&`. + - Did not handle hex *floats* in strings (`"0xAA.0"`). `bitwise.lua` + line 24-25 use them as bitwise operands. Fixed by adding hex-float + parsing matching the lexer's existing hex-float scanner. + +3. **Float → int coercion in bitwise ops (`Executor.to_integer!`).** Was + doing unconditional `trunc/1`. Per Lua 5.3 §3.4.3, the float must + represent an integer **exactly** and fit in the signed 64-bit range, + otherwise the bitwise op must error. `bitwise.lua` line 59 explicitly + tests this with `pcall`. + +4. **`require` did not consult `package.preload` (stdlib).** The + `bitwise.lua` test installs its own `bit32` implementation via + `package.preload.bit32 = function () ... end` and then calls + `require'bit32'`. Our `require` only checked `package.loaded` and the + filesystem search path. Added a narrow `package.preload` lookup ahead + of the path search. **Did not** introduce a full `package.searchers` + table — that's a separate concern. + +5. **`math.fmod` is not implemented.** `bitwise.lua` lines 278–279 use + `math.fmod` to verify `bit32.lshift` numerically. Out of scope for a + "bitwise" plan; deferred to **A5a** + (`A5a-bitwise-suite-math-fmod.md`). With `math.fmod` in place, + `bitwise.lua` is expected to pass end-to-end (no other gaps were found + downstream of line 278 in line-by-line bisect; see `## What changed`). + +## What changed + +PR: #198 + +**Files touched:** + +- `lib/lua/lexer.ex` — wrap hex integer literals to signed 64-bit + in `scan_hex_number`. Inlined `wrap_int64/1` using existing `Bitwise` + import; lexer stays VM-dep-free. +- `lib/lua/vm/value.ex` — rewrite `parse_number/1` to handle leading + `-`/`+` signs (with negation through `Numeric.to_signed_int64`) and + hex floats in strings (`"0xAA.0"`, `"0x1.8p3"`). Hex int strings now + also wrap. +- `lib/lua/vm/executor.ex` — split `to_integer!/1` so the float and + string-parsed-to-float paths go through `float_to_integer!/1`, which + raises unless the float represents an integer exactly and is in the + signed 64-bit range (Lua 5.3 §3.4.3). +- `lib/lua/vm/stdlib.ex` — `lua_require` now consults + `package.preload[modname]` before falling through to the path search. + Loader is called via `Executor.call_function`. Result caching matches + `parse_and_execute_module` (sentinel `true` before call, replaced + after). +- `test/lua/vm/bitwise_test.exs` — new file with 12 reduced-repro + assertions, one per failing case found in `bitwise.lua` lines 8–62. + +**Suite delta:** `bitwise.lua` was failing on its very first assertion +(line 8 area, no line info). Now runs cleanly through line 270, including +the entire `bit32` library block (lines 173–270). Final ~50 lines blocked +on `math.fmod`. + +**Test count:** 1382 → 1394 (+12 from new bitwise_test.exs). 0 failures, +32 skipped (unchanged). + +**Follow-ups opened:** + +- `.agents/plans/A5a-bitwise-suite-math-fmod.md` — implement `math.fmod` + to finish `bitwise.lua` end-to-end. Status `ready`. diff --git a/.agents/plans/A5a-bitwise-suite-math-fmod.md b/.agents/plans/A5a-bitwise-suite-math-fmod.md new file mode 100644 index 00000000..995d8950 --- /dev/null +++ b/.agents/plans/A5a-bitwise-suite-math-fmod.md @@ -0,0 +1,71 @@ +--- +id: A5a +title: Add math.fmod for bitwise.lua bit32 verification +issue: null +pr: null +branch: feat/math-fmod +base: main +status: ready +direction: A +unlocks: + - bitwise.lua (final third — bit32.lshift verification block) +--- + +## Goal + +Implement `math.fmod` so `bitwise.lua` can finish its bit32-library +verification block (line 278 onward). This was the last remaining gap +discovered while shipping A5. + +## Out of scope + +- Other missing math.* functions (math.modf, math.atan2, etc.) unless they + are also exercised by `bitwise.lua` and we hit them while wiring fmod in. +- A `package.searchers` table (currently we have a narrow `package.preload` + hook added in A5; full searchers list is a separate concern). + +## Success criteria + +- [ ] `math.fmod(x, y)` works per Lua 5.3 §6.7: + "Returns the remainder of the division of x by y that rounds the + quotient towards zero." +- [ ] `math.fmod` of two integers returns an integer; otherwise returns a + float. +- [ ] `mix test` passes, no regressions. +- [ ] Unit tests in `test/lua/vm/stdlib/math_test.exs` cover: integer/integer, + integer/float, float/float, negative dividend, divide-by-zero behavior + (raises for ints, returns NaN for floats per Lua semantics). +- [ ] `bitwise.lua` passes end-to-end. (This was deferred from A5.) + +## Implementation notes + +Add `math.fmod` to `lib/lua/vm/stdlib/math.ex` next to `math.floor`. For +two integers, use `Kernel.rem/2` (truncated remainder, sign of dividend), +matching C's `%` and Lua 5.3's `math.fmod` for ints. For floats, use +`:math.fmod/2` if available or compute as `x - trunc(x / y) * y`. + +Then re-run `bitwise.lua` to confirm. If new gaps surface (other missing +math functions, etc.), the triage-suite-failure skill applies the same +way: reduce → fix-now-or-defer → ship. + +## Verification + +```bash +mix format +mix compile --warnings-as-errors +mix test +mix test test/lua/vm/stdlib/math_test.exs +mix run --no-mix-exs -e 'Code.require_file("test/support/lua_test_case.ex"); Lua.TestCase.run_lua_file("test/lua53_tests/bitwise.lua")' +``` + +## Risks + +- Lua 5.3 specifies `math.fmod(x, 0)` for integers raises "bad argument #2 + ... (zero)" but for floats returns NaN. We need to honor both cases. +- Bit-perfect float fmod is subtle for edge cases (NaN, infinities). Lua + defers to C's `fmod`; Erlang's `:math.fmod/2` is the same OS-level + implementation, so we should match. + +## Discoveries + +(populated during implementation) diff --git a/lib/lua/lexer.ex b/lib/lua/lexer.ex index 09a383e6..3bbeafb3 100644 --- a/lib/lua/lexer.ex +++ b/lib/lua/lexer.ex @@ -685,7 +685,9 @@ defmodule Lua.Lexer do defp scan_hex_number(rest, hex_acc, acc, pos, start_pos) do case Integer.parse(hex_acc, 16) do {num, ""} -> - token = {:number, num, start_pos} + # Per Lua 5.3 §3.1: hex integer literals overflow-wrap into the + # signed 64-bit range. e.g. 0xFFFFFFFFFFFFFFFF == -1. + token = {:number, wrap_int64(num), start_pos} do_tokenize(rest, [token | acc], pos) _ -> @@ -693,6 +695,17 @@ defmodule Lua.Lexer do end end + # Wrap an unsigned hex integer to the signed 64-bit range. Inlined here so + # the lexer doesn't depend on Lua.VM.Numeric (kept VM-internal). + @uint64_mask 0xFFFFFFFFFFFFFFFF + @uint64_modulus 0x10000000000000000 + @sign_bit 0x8000000000000000 + + defp wrap_int64(n) when is_integer(n) do + masked = band(n, @uint64_mask) + if masked >= @sign_bit, do: masked - @uint64_modulus, else: masked + end + # Scan hex fractional digits after the dot defp scan_hex_frac(<>, int_acc, frac_acc, acc, pos, start_pos) when c in ?0..?9 or c in ?a..?f or c in ?A..?F do diff --git a/lib/lua/vm/executor.ex b/lib/lua/vm/executor.ex index 8f45826a..d8bc44f0 100644 --- a/lib/lua/vm/executor.ex +++ b/lib/lua/vm/executor.ex @@ -1863,7 +1863,7 @@ defmodule Lua.VM.Executor do defp to_number(v), do: {:error, v} defp to_integer!(v) when is_integer(v), do: v - defp to_integer!(v) when is_float(v), do: trunc(v) + defp to_integer!(v) when is_float(v), do: float_to_integer!(v) defp to_integer!(v) when is_binary(v) do case Value.parse_number(v) do @@ -1873,8 +1873,11 @@ defmodule Lua.VM.Executor do error_kind: :bitwise_on_non_integer, value_type: :string - n -> - trunc(n) + n when is_integer(n) -> + n + + n when is_float(n) -> + float_to_integer!(n) end end @@ -1885,6 +1888,21 @@ defmodule Lua.VM.Executor do value_type: value_type(v) end + # Per Lua 5.3 §3.4.3: a float converts to integer for bitwise ops only if + # it represents an integer exactly and fits in the signed 64-bit range. + defp float_to_integer!(f) when is_float(f) do + truncated = trunc(f) + + if f == truncated * 1.0 and Numeric.signed?(truncated) do + truncated + else + raise TypeError, + value: "number has no integer representation", + error_kind: :bitwise_on_non_integer, + value_type: :number + end + end + # ── Lua 5.3 shift semantics ──────────────────────────────────────────────── defp lua_shift_left(_val, shift) when shift >= 64, do: 0 diff --git a/lib/lua/vm/stdlib.ex b/lib/lua/vm/stdlib.ex index f36f5a86..31ae4b72 100644 --- a/lib/lua/vm/stdlib.ex +++ b/lib/lua/vm/stdlib.ex @@ -580,9 +580,15 @@ defmodule Lua.VM.Stdlib do # Check if already loaded case Map.get(loaded_table.data, modname) do nil -> - # Not loaded, need to search and load - search_path = Map.get(package.data, "path", "?.lua") - load_module(modname, search_path, state) + # Not loaded. First consult package.preload, then fall back to path. + case lookup_preload(modname, package, state) do + {:ok, loader} -> + load_from_preload(modname, loader, state) + + :not_found -> + search_path = Map.get(package.data, "path", "?.lua") + load_module(modname, search_path, state) + end result -> # Already loaded, return cached result @@ -602,6 +608,41 @@ defmodule Lua.VM.Stdlib do raise ArgumentError.value_expected("require", 1) end + # Look up `package.preload[modname]`. Returns `{:ok, loader}` if a callable + # is registered, `:not_found` otherwise. This is the narrow searcher + # equivalent for `package.preload` — we don't yet expose the full + # `package.searchers` table. + defp lookup_preload(modname, package, state) do + case Map.get(package.data, "preload") do + {:tref, preload_id} -> + preload_table = Map.fetch!(state.tables, preload_id) + + case Map.get(preload_table.data, modname) do + nil -> :not_found + loader -> {:ok, loader} + end + + _ -> + :not_found + end + end + + # Call a preload loader (closure or native_func) with the module name + # and cache its result, mirroring `parse_and_execute_module`. + defp load_from_preload(modname, loader, state) do + state = cache_module_result(state, modname, true) + {results, state} = Executor.call_function(loader, [modname], state) + + result = + case results do + [value | _] -> value + [] -> true + end + + state = cache_module_result(state, modname, result) + {[result], state} + end + # Load a module by searching the path defp load_module(modname, search_path, state) do patterns = String.split(search_path, ";", trim: true) diff --git a/lib/lua/vm/value.ex b/lib/lua/vm/value.ex index 51113135..250407ca 100644 --- a/lib/lua/vm/value.ex +++ b/lib/lua/vm/value.ex @@ -7,6 +7,7 @@ defmodule Lua.VM.Value do the executor and stdlib. """ + alias Lua.VM.Numeric alias Lua.VM.State @doc """ @@ -63,28 +64,94 @@ defmodule Lua.VM.Value do @doc """ Parses a string to a number (integer or float), supporting hex notation. + Hex integer literals overflow-wrap into the signed 64-bit range per Lua 5.3 + §3.1 (e.g. `"0xFFFFFFFFFFFFFFFF"` → `-1`). Hex floats with a fractional + part or binary exponent (e.g. `"0xAA.0"`, `"0x1.8p3"`) are also supported. + Returns `nil` if the string cannot be parsed. """ @spec parse_number(String.t()) :: number() | nil def parse_number(str) do str = String.trim(str) - if String.starts_with?(str, "0x") or String.starts_with?(str, "0X") do - case Integer.parse(String.slice(str, 2..-1//1), 16) do - {n, ""} -> n - _ -> nil + {sign, body} = + case str do + "-" <> rest -> {-1, String.trim_leading(rest)} + "+" <> rest -> {1, String.trim_leading(rest)} + _ -> {1, str} end - else - case Integer.parse(str) do - {n, ""} -> - n - - _ -> - case Float.parse(str) do - {f, ""} -> f - _ -> nil - end + + parsed = + if String.starts_with?(body, "0x") or String.starts_with?(body, "0X") do + parse_hex_number(String.slice(body, 2..-1//1)) + else + case Integer.parse(body) do + {n, ""} -> + n + + _ -> + case Float.parse(body) do + {f, ""} -> f + _ -> nil + end + end end + + case parsed do + nil -> nil + n when sign == 1 -> n + n when is_integer(n) -> Numeric.to_signed_int64(-n) + n when is_float(n) -> -n + end + end + + # Parse the body of a hex literal (after "0x"). Supports: + # * integer: "FF" → 255 (wrapped to signed int64) + # * fractional: "FF.8" → 255.5 + # * exponent only: "FFp4" → 4080.0 + # * fractional+exp: "1.8p3" → 12.0 + defp parse_hex_number(body) do + case parse_hex_int(body) do + {int_val, ""} -> + Numeric.to_signed_int64(int_val) + + {int_val, "." <> rest} -> + parse_hex_frac(int_val, rest) + + {int_val, <>} when p in [?p, ?P] -> + parse_hex_exp(int_val + 0.0, rest) + + _ -> + nil + end + end + + defp parse_hex_int(""), do: :error + defp parse_hex_int(body), do: Integer.parse(body, 16) + + defp parse_hex_frac(int_val, frac_and_rest) do + case Integer.parse(frac_and_rest, 16) do + {frac_int, rest} -> + # Determine how many hex digits the fractional part used. + digits = byte_size(frac_and_rest) - byte_size(rest) + frac = if digits == 0, do: 0.0, else: frac_int / :math.pow(16, digits) + base = int_val + frac + + case rest do + "" -> base + <> when p in [?p, ?P] -> parse_hex_exp(base, exp_rest) + _ -> nil + end + + :error -> + nil + end + end + + defp parse_hex_exp(base, exp_str) do + case Integer.parse(exp_str) do + {exp, ""} -> base * :math.pow(2, exp) + _ -> nil end end diff --git a/test/lua/vm/bitwise_test.exs b/test/lua/vm/bitwise_test.exs new file mode 100644 index 00000000..02eb3954 --- /dev/null +++ b/test/lua/vm/bitwise_test.exs @@ -0,0 +1,77 @@ +defmodule Lua.VM.BitwiseTest do + use ExUnit.Case, async: true + + # Reduced repros of failing assertions from test/lua53_tests/bitwise.lua. + # Per Lua 5.3 §3.1: "Hexadecimal numerals with neither a radix point nor + # an exponent always denote an integer value; if the value overflows, it + # wraps around to fit into a valid integer." + + defp run(code) do + {results, _lua} = Lua.eval!(Lua.new(), code) + results + end + + describe "hex integer literal overflow wrapping" do + # bitwise.lua:16 + test "0xFFFFFFFFFFFFFFFF == -1" do + assert [true] = run("return 0xFFFFFFFFFFFFFFFF == -1") + end + + test "0xFFFFFFFFFFFFFFFF & -1 == 0xFFFFFFFFFFFFFFFF" do + assert [true] = run("local a = 0xFFFFFFFFFFFFFFFF; return a & -1 == a") + end + + # bitwise.lua:19 + test "0xF0F0F0F0F0F0F0F0 ~ 0 == 0xF0F0F0F0F0F0F0F0" do + assert [true] = run("local a = 0xF0F0F0F0F0F0F0F0; return a ~ 0 == a") + end + + test "0xF0F0F0F0F0F0F0F0 ~ ~a == -1" do + assert [true] = run("local a = 0xF0F0F0F0F0F0F0F0; return a ~ ~a == -1") + end + + test "0x8000000000000000 == math.mininteger" do + assert [true] = run("return 0x8000000000000000 == math.mininteger") + end + + test "0xFFFFFFFFFFFFFFFF prints as -1" do + assert [-1] = run("return 0xFFFFFFFFFFFFFFFF") + end + end + + describe "string-to-number coercion for bitwise operators" do + # bitwise.lua:25 — hex floats in strings should coerce + test "'0xAA.0' bor 0 == 0xAA" do + assert [170] = run(~S{return "0xAA.0" | 0}) + end + + test "'0xFD.0' band 0xFF == 0xFD" do + assert [253] = run(~S{return "0xFD.0" & 0xFF}) + end + + # tonumber should also handle hex floats in strings + test "tonumber('0xAA.0') == 170.0" do + assert [170.0] = run(~S{return tonumber("0xAA.0")}) + end + + test "tonumber('0xff.8') == 255.5" do + assert [255.5] = run(~S{return tonumber("0xff.8")}) + end + + # bitwise.lua:59 — out-of-range hex float string still errors via pcall + test "string with hex float that doesn't fit in int raises" do + assert [false] = + run(~S{local ok = pcall(function() return "0xffffffffffffffff.0" | 0 end); return ok}) + end + end + + describe "Lua 5.3 mininteger boundary" do + test "1 << (numbits - 1) == math.mininteger" do + assert [true] = + run(""" + local numbits = string.packsize('j') * 8 + return (1 << (numbits - 1)) == math.mininteger + """) + end + end +end