Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 82 additions & 6 deletions .agents/plans/A5-bitwise-suite.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
id: A5
title: Fix bitwise.lua failing assertion
issue: 166
pr: null
pr: 198
branch: fix/bitwise-suite
base: main
status: ready
status: review
direction: A
unlocks:
- bitwise.lua
Expand All @@ -24,10 +24,11 @@ assertion).

## Success criteria

- [ ] `mix test` passes (≥ 1273, no regressions)
- [ ] Each fixed assertion has a corresponding unit test in
- [x] `mix test` passes (≥ 1273, no regressions)
- [x] Each fixed assertion has a corresponding unit test in
`test/lua/vm/bitwise_test.exs`.
- [ ] `bitwise.lua` passes end-to-end.
- [ ] `bitwise.lua` passes end-to-end. (Partially: lines 1–270 now pass.
Final third blocked on `math.fmod`, deferred to A5a.)

## Implementation notes

Expand Down Expand Up @@ -61,4 +62,79 @@ mix test test/lua/vm/bitwise_test.exs

## Discoveries

(populated during implementation)
A0 fixed the integer arithmetic wrap but **not** integer literal lexing or
string-to-number coercion. Three distinct issues surfaced while running
`bitwise.lua`:

1. **Hex integer literal wrapping (lexer).** `0xFFFFFFFFFFFFFFFF` was
parsed as the unsigned int `18446744073709551615`. Per Lua 5.3 §3.1,
hex integer literals overflow-wrap into the signed 64-bit range. Fixed
inline in `Lua.Lexer.scan_hex_number` (didn't reach for `Lua.VM.Numeric`
to keep the lexer free of VM-internal deps).

2. **`Value.parse_number` (string → number coercion).**
- Same hex-overflow issue as the lexer.
- Did not handle a leading `-` or `+` (e.g. `"-0xfffffffffffffffe"` →
`nil`). `bitwise.lua` line 54-55 use this form for coercion via `&`.
- Did not handle hex *floats* in strings (`"0xAA.0"`). `bitwise.lua`
line 24-25 use them as bitwise operands. Fixed by adding hex-float
parsing matching the lexer's existing hex-float scanner.

3. **Float → int coercion in bitwise ops (`Executor.to_integer!`).** Was
doing unconditional `trunc/1`. Per Lua 5.3 §3.4.3, the float must
represent an integer **exactly** and fit in the signed 64-bit range,
otherwise the bitwise op must error. `bitwise.lua` line 59 explicitly
tests this with `pcall`.

4. **`require` did not consult `package.preload` (stdlib).** The
`bitwise.lua` test installs its own `bit32` implementation via
`package.preload.bit32 = function () ... end` and then calls
`require'bit32'`. Our `require` only checked `package.loaded` and the
filesystem search path. Added a narrow `package.preload` lookup ahead
of the path search. **Did not** introduce a full `package.searchers`
table — that's a separate concern.

5. **`math.fmod` is not implemented.** `bitwise.lua` lines 278–279 use
`math.fmod` to verify `bit32.lshift` numerically. Out of scope for a
"bitwise" plan; deferred to **A5a**
(`A5a-bitwise-suite-math-fmod.md`). With `math.fmod` in place,
`bitwise.lua` is expected to pass end-to-end (no other gaps were found
downstream of line 278 in line-by-line bisect; see `## What changed`).

## What changed

PR: #198

**Files touched:**

- `lib/lua/lexer.ex` — wrap hex integer literals to signed 64-bit
in `scan_hex_number`. Inlined `wrap_int64/1` using existing `Bitwise`
import; lexer stays VM-dep-free.
- `lib/lua/vm/value.ex` — rewrite `parse_number/1` to handle leading
`-`/`+` signs (with negation through `Numeric.to_signed_int64`) and
hex floats in strings (`"0xAA.0"`, `"0x1.8p3"`). Hex int strings now
also wrap.
- `lib/lua/vm/executor.ex` — split `to_integer!/1` so the float and
string-parsed-to-float paths go through `float_to_integer!/1`, which
raises unless the float represents an integer exactly and is in the
signed 64-bit range (Lua 5.3 §3.4.3).
- `lib/lua/vm/stdlib.ex` — `lua_require` now consults
`package.preload[modname]` before falling through to the path search.
Loader is called via `Executor.call_function`. Result caching matches
`parse_and_execute_module` (sentinel `true` before call, replaced
after).
- `test/lua/vm/bitwise_test.exs` — new file with 12 reduced-repro
assertions, one per failing case found in `bitwise.lua` lines 8–62.

**Suite delta:** `bitwise.lua` was failing on its very first assertion
(line 8 area, no line info). Now runs cleanly through line 270, including
the entire `bit32` library block (lines 173–270). Final ~50 lines blocked
on `math.fmod`.

**Test count:** 1382 → 1394 (+12 from new bitwise_test.exs). 0 failures,
32 skipped (unchanged).

**Follow-ups opened:**

- `.agents/plans/A5a-bitwise-suite-math-fmod.md` — implement `math.fmod`
to finish `bitwise.lua` end-to-end. Status `ready`.
71 changes: 71 additions & 0 deletions .agents/plans/A5a-bitwise-suite-math-fmod.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
id: A5a
title: Add math.fmod for bitwise.lua bit32 verification
issue: null
pr: null
branch: feat/math-fmod
base: main
status: ready
direction: A
unlocks:
- bitwise.lua (final third — bit32.lshift verification block)
---

## Goal

Implement `math.fmod` so `bitwise.lua` can finish its bit32-library
verification block (line 278 onward). This was the last remaining gap
discovered while shipping A5.

## Out of scope

- Other missing math.* functions (math.modf, math.atan2, etc.) unless they
are also exercised by `bitwise.lua` and we hit them while wiring fmod in.
- A `package.searchers` table (currently we have a narrow `package.preload`
hook added in A5; full searchers list is a separate concern).

## Success criteria

- [ ] `math.fmod(x, y)` works per Lua 5.3 §6.7:
"Returns the remainder of the division of x by y that rounds the
quotient towards zero."
- [ ] `math.fmod` of two integers returns an integer; otherwise returns a
float.
- [ ] `mix test` passes, no regressions.
- [ ] Unit tests in `test/lua/vm/stdlib/math_test.exs` cover: integer/integer,
integer/float, float/float, negative dividend, divide-by-zero behavior
(raises for ints, returns NaN for floats per Lua semantics).
- [ ] `bitwise.lua` passes end-to-end. (This was deferred from A5.)

## Implementation notes

Add `math.fmod` to `lib/lua/vm/stdlib/math.ex` next to `math.floor`. For
two integers, use `Kernel.rem/2` (truncated remainder, sign of dividend),
matching C's `%` and Lua 5.3's `math.fmod` for ints. For floats, use
`:math.fmod/2` if available or compute as `x - trunc(x / y) * y`.

Then re-run `bitwise.lua` to confirm. If new gaps surface (other missing
math functions, etc.), the triage-suite-failure skill applies the same
way: reduce → fix-now-or-defer → ship.

## Verification

```bash
mix format
mix compile --warnings-as-errors
mix test
mix test test/lua/vm/stdlib/math_test.exs
mix run --no-mix-exs -e 'Code.require_file("test/support/lua_test_case.ex"); Lua.TestCase.run_lua_file("test/lua53_tests/bitwise.lua")'
```

## Risks

- Lua 5.3 specifies `math.fmod(x, 0)` for integers raises "bad argument #2
... (zero)" but for floats returns NaN. We need to honor both cases.
- Bit-perfect float fmod is subtle for edge cases (NaN, infinities). Lua
defers to C's `fmod`; Erlang's `:math.fmod/2` is the same OS-level
implementation, so we should match.

## Discoveries

(populated during implementation)
15 changes: 14 additions & 1 deletion lib/lua/lexer.ex
Original file line number Diff line number Diff line change
Expand Up @@ -685,14 +685,27 @@ defmodule Lua.Lexer do
defp scan_hex_number(rest, hex_acc, acc, pos, start_pos) do
case Integer.parse(hex_acc, 16) do
{num, ""} ->
token = {:number, num, start_pos}
# Per Lua 5.3 §3.1: hex integer literals overflow-wrap into the
# signed 64-bit range. e.g. 0xFFFFFFFFFFFFFFFF == -1.
token = {:number, wrap_int64(num), start_pos}
do_tokenize(rest, [token | acc], pos)

_ ->
{:error, {:invalid_hex_number, start_pos}}
end
end

# Wrap an unsigned hex integer to the signed 64-bit range. Inlined here so
# the lexer doesn't depend on Lua.VM.Numeric (kept VM-internal).
@uint64_mask 0xFFFFFFFFFFFFFFFF
@uint64_modulus 0x10000000000000000
@sign_bit 0x8000000000000000

defp wrap_int64(n) when is_integer(n) do
masked = band(n, @uint64_mask)
if masked >= @sign_bit, do: masked - @uint64_modulus, else: masked
end

# Scan hex fractional digits after the dot
defp scan_hex_frac(<<c, rest::binary>>, int_acc, frac_acc, acc, pos, start_pos)
when c in ?0..?9 or c in ?a..?f or c in ?A..?F do
Expand Down
24 changes: 21 additions & 3 deletions lib/lua/vm/executor.ex
Original file line number Diff line number Diff line change
Expand Up @@ -1863,7 +1863,7 @@ defmodule Lua.VM.Executor do
defp to_number(v), do: {:error, v}

defp to_integer!(v) when is_integer(v), do: v
defp to_integer!(v) when is_float(v), do: trunc(v)
defp to_integer!(v) when is_float(v), do: float_to_integer!(v)

defp to_integer!(v) when is_binary(v) do
case Value.parse_number(v) do
Expand All @@ -1873,8 +1873,11 @@ defmodule Lua.VM.Executor do
error_kind: :bitwise_on_non_integer,
value_type: :string

n ->
trunc(n)
n when is_integer(n) ->
n

n when is_float(n) ->
float_to_integer!(n)
end
end

Expand All @@ -1885,6 +1888,21 @@ defmodule Lua.VM.Executor do
value_type: value_type(v)
end

# Per Lua 5.3 §3.4.3: a float converts to integer for bitwise ops only if
# it represents an integer exactly and fits in the signed 64-bit range.
defp float_to_integer!(f) when is_float(f) do
truncated = trunc(f)

if f == truncated * 1.0 and Numeric.signed?(truncated) do
truncated
else
raise TypeError,
value: "number has no integer representation",
error_kind: :bitwise_on_non_integer,
value_type: :number
end
end

# ── Lua 5.3 shift semantics ────────────────────────────────────────────────

defp lua_shift_left(_val, shift) when shift >= 64, do: 0
Expand Down
47 changes: 44 additions & 3 deletions lib/lua/vm/stdlib.ex
Original file line number Diff line number Diff line change
Expand Up @@ -580,9 +580,15 @@ defmodule Lua.VM.Stdlib do
# Check if already loaded
case Map.get(loaded_table.data, modname) do
nil ->
# Not loaded, need to search and load
search_path = Map.get(package.data, "path", "?.lua")
load_module(modname, search_path, state)
# Not loaded. First consult package.preload, then fall back to path.
case lookup_preload(modname, package, state) do
{:ok, loader} ->
load_from_preload(modname, loader, state)

:not_found ->
search_path = Map.get(package.data, "path", "?.lua")
load_module(modname, search_path, state)
end

result ->
# Already loaded, return cached result
Expand All @@ -602,6 +608,41 @@ defmodule Lua.VM.Stdlib do
raise ArgumentError.value_expected("require", 1)
end

# Look up `package.preload[modname]`. Returns `{:ok, loader}` if a callable
# is registered, `:not_found` otherwise. This is the narrow searcher
# equivalent for `package.preload` — we don't yet expose the full
# `package.searchers` table.
defp lookup_preload(modname, package, state) do
case Map.get(package.data, "preload") do
{:tref, preload_id} ->
preload_table = Map.fetch!(state.tables, preload_id)

case Map.get(preload_table.data, modname) do
nil -> :not_found
loader -> {:ok, loader}
end

_ ->
:not_found
end
end

# Call a preload loader (closure or native_func) with the module name
# and cache its result, mirroring `parse_and_execute_module`.
defp load_from_preload(modname, loader, state) do
state = cache_module_result(state, modname, true)
{results, state} = Executor.call_function(loader, [modname], state)

result =
case results do
[value | _] -> value
[] -> true
end

state = cache_module_result(state, modname, result)
{[result], state}
end

# Load a module by searching the path
defp load_module(modname, search_path, state) do
patterns = String.split(search_path, ";", trim: true)
Expand Down
Loading
Loading