diff --git a/.agents/plans/A26-error-message-quality.md b/.agents/plans/A26-error-message-quality.md index 8b5a9514..23b8c100 100644 --- a/.agents/plans/A26-error-message-quality.md +++ b/.agents/plans/A26-error-message-quality.md @@ -1,25 +1,25 @@ --- id: A26 title: Error message quality pass — make the rendered output world-class -issue: null -pr: null +issue: 263 +pr: 304 branch: errors/quality-pass base: main -status: blocked +status: review direction: A unlocks: - "world-class error messages" promise of the library --- -## Blocked on - -- A19 — every raise site needs to carry line/source first; otherwise - this pass has nothing to render in many cases. - ## Goal A18 + A19 wired the *data* into every exception (`:line`, `:source`, -`:call_stack`, `:value_type`, `:error_kind`). This plan answers the +`:call_stack`, `:value_type`, `:error_kind`). A19 (PR #215, status +`review`) does this by auto-populating `:line`/`:source`/`:call_stack` +inside each exception module's `exception/1` from +`Lua.VM.Executor.current_position/0`, so the four user-visible VM +exception types (`TypeError`, `RuntimeError`, `AssertionError`, +`ArgumentError`) all carry location data. This plan answers the question: **is the rendered output actually world-class?** Audit every error message a user can see. For each, evaluate: @@ -31,20 +31,30 @@ Audit every error message a user can see. For each, evaluate: - Is the call stack readable? - Does the message look right on a non-TTY (stripped of ANSI codes)? -Then fix the rough edges. +Then fix the rough edges. The audit is grounded in concrete findings +captured under **Implementation notes → Confirmed audit findings**; +the implementation work is to close those gaps, lock the output with +fixture tests, and ship a docs gallery. ## Out of scope - Changing the exception struct shape (A18 settled this). -- Adding new error categories. +- Adding new error *categories* (the existing `error_kind` set is + fixed). Wiring up suggestions for kinds that already exist is in + scope; inventing new kinds is not. - Implementing `traceback()` or `debug.traceback` — we already render the stack we have. - Source maps past `load()` chunks. +- Fixing the two pre-existing data oddities A19 logged in its + Discoveries (`table.insert(t, nil, 5)` reports arg `#1` instead of + `#2`; `setmetatable(5, {})` on line 1 reports `line 0`). These are + data-layer bugs, not rendering bugs; if confirmed during the audit, + log them and open follow-ups rather than expanding scope. ## Success criteria -- [ ] A doc test or fixture file exists showing the rendered output - for each error category: +- [ ] A gallery test pins the rendered output (as an inline `expected` + string) for each error category: - [ ] arithmetic on non-number - [ ] indexing nil / non-table - [ ] calling nil / non-function @@ -56,63 +66,122 @@ Then fix the rough edges. - [ ] `assert(false, msg)` and `assert(false)` - [ ] `error("msg")` and `error({tbl})` - [ ] runtime stack overflow / infinite recursion -- [ ] Each message prominently shows `at ::` on its own - line, before the message body. -- [ ] Suggestions (when present) are specific to the error category. - Generic "check your logic" filler is replaced or removed. -- [ ] Color rendering: messages render readably with ANSI on TTY and - cleanly without ANSI on `IO.puts/2` to a file or non-TTY pipe. -- [ ] A small "error gallery" lives in the docs (probably - `guides/errors.md` or as part of the README) showing - before/after for each category. -- [ ] `mix test` passes. +- [ ] Each message leads with `at ::` before the message + body (currently it renders *after* the header line — fix the + ordering in `format/3`). Note the public exception still prefixes + `Lua runtime error: `, so the first rendered line is + `Lua runtime error: at ::`. +- [ ] The redundant double label is gone: today the public exception + prefixes `Lua runtime error: ` and the formatter header also says + `Runtime Type Error`. Pick one consistent presentation. +- [ ] Suggestions (when present) are specific to the error category and + reachable. The formatter's `build_suggestion/3` matches + `:arithmetic_type_error`, but the executor emits + `:arithmetic_on_non_number` — that clause is dead. Align the + atoms, and add category-specific suggestions for the emitted + kinds that currently get none (`:compare_incompatible_types`, + `:length_not_integer`). The generic + `"...Check your logic."` assertion suggestion is filler — replace + or remove it. +- [ ] Color rendering: messages render readably with ANSI on a TTY and + cleanly without ANSI when written to a file or non-TTY pipe. + Today `format/3` emits `IO.ANSI.*` unconditionally, so escape + codes leak into non-TTY output — gate ANSI behind a TTY/enabled + check (e.g. `IO.ANSI.enabled?/0`) so the same render path is safe + either way. +- [ ] A small "error gallery" pins before/after for each category. (Shipped + as inline `expected` strings in `test/lua/error_gallery_test.exs`; the + standalone `guides/errors.md` draft was dropped during review.) +- [ ] `mix test` passes (1585 → 1585+, no regressions). ## Implementation notes -### Audit step +### Confirmed audit findings (grounded, not speculative) -Run a fixture script that triggers each error category, capture -stdout, and write the rendered output to -`test/fixtures/error_gallery/.txt`. These become regression -tests: any change to the formatter that alters output requires -updating the fixture. - -Suggested gallery file format: +Rendering `local x = nil; print(x + 1)` via `Lua.eval!` today produces: ``` -=== category: arithmetic on non-number === -=== source === -local x = "5" -local y = nil -print(x + y) -=== expected output === -** (Lua.RuntimeException) Lua runtime error: Runtime Type Error - at gallery.lua:3: - attempt to perform arithmetic on a nil value - ... +Lua runtime error: [31m[1mRuntime Type Error[0m + + at :2: + + attempt to perform arithmetic on a nil value (local 'x') ``` -### Likely improvements to make +This single example confirms four of the things to fix: + +1. **Double label.** `Lua runtime error: ` (from + `Lua.RuntimeException`, see `@runtime_prefix`) plus + `Runtime Type Error` (from `ErrorFormatter.format_header/1`) — two + redundant "runtime"/"error" labels stacked. +2. **Location not prominent.** `at :2:` renders on its own line + but *after* the header, not before the message body. The success + criterion wants it up top. +3. **Raw ANSI leak.** `format/3` calls `IO.ANSI.red()` etc. + unconditionally; piping to a file embeds raw escapes. Gate on + `IO.ANSI.enabled?/0`. +4. **Missing / dead suggestions.** Executor emits these `error_kind`s: + `:call_nil`, `:call_non_function`, `:index_non_table`, + `:concatenate_type_error`, `:arithmetic_on_non_number`, + `:compare_incompatible_types`, `:length_not_integer`, + `:bitwise_on_non_integer`, `:for_loop_non_number`. The formatter's + `build_suggestion/3` only matches `:call_nil`, + `:call_non_function`, `:index_non_table`, `:arithmetic_type_error` + (a name that is never emitted — dead clause), and + `:concatenate_type_error`. So arithmetic gets *no* suggestion (atom + mismatch), and compare / length get none either. + +### Audit / gallery step -- The current TypeError prefix "Lua runtime error: " followed by - "Runtime Type Error" is redundant. Pick one. -- Suggestion text appears even when not useful. Consider making it - category-specific or omitting it entirely when generic. -- Stack traces from native functions and Lua functions interleave; - make sure the formatting distinguishes them clearly. -- Multi-line values in error messages (e.g. when a printed table is - shown) should be indented consistently. +The gallery test triggers each error category listed in Success +criteria via `Lua.eval!`, captures the rendered `Exception.message/1` +output, and compares it to an inline `expected` string in the test +itself. Output is captured with ANSI disabled (`IO.ANSI.enabled? == +false` path) so the expectations are plain text and stable across +terminals. The test asserts the rendered output equals the inline +expectation; updating it is an explicit edit when output intentionally +changes. + +Each table-driven case carries its source snippet and expected render +inline, e.g.: + +``` +arithmetic on non-number -> + source: print(x + y) # x = "5", y = nil + expected: "at gallery.lua:3:\n\n attempt to perform arithmetic ..." +``` ### Files -- `lib/lua/error_formatter.ex` — main render path. -- `lib/lua/runtime_exception.ex` — `Exception.message/1`. -- `lib/lua/vm/type_error.ex` — `Exception.message/1`. -- `lib/lua/vm/runtime_error.ex` — `Exception.message/1`. -- `lib/lua/vm/assertion_error.ex` — `Exception.message/1`. -- `test/lua/error_gallery_test.exs` (new) — fixture comparison tests. -- `test/fixtures/error_gallery/*.txt` (new) — expected outputs. -- `guides/errors.md` (new) or `README.md` section. +- `lib/lua/vm/error_formatter.ex` — main render path + (`format/3`, `to_map/3`). Reorder location-before-body, gate ANSI on + `IO.ANSI.enabled?/0`, align `build_suggestion/3` atoms with the + kinds the executor actually emits, add the missing category + suggestions, replace the filler assertion suggestion. +- `lib/lua/runtime_exception.ex` — `Exception.message/1` and + `@runtime_prefix`: resolve the double-label with the formatter + header. +- `lib/lua/vm/type_error.ex` — `exception/1` builds the message via + `ErrorFormatter.format/3`; adjust if the prefix/ordering change + requires it. +- `lib/lua/vm/runtime_error.ex` — same. +- `lib/lua/vm/assertion_error.ex` — same; drop or specialize the + generic suggestion. +- `lib/lua/vm/argument_error.ex` — `message/1` routes stdlib bad-arg + errors through `ErrorFormatter.format(:type_error, ...)`; make sure + the stdlib bad-arg case stays correct after formatter changes. +- `test/lua/error_gallery_test.exs` (new) — gallery tests with inline + `expected` strings for every reachable category. +- A standalone `guides/errors.md` was drafted then dropped during review; + the before/after gallery is carried by the test above instead. + +Note: the plan's earlier draft pointed at `lib/lua/error_formatter.ex` +and a `Lua.VM.TypeError.message/1`. The real formatter lives at +`lib/lua/vm/error_formatter.ex`, and the VM exceptions build their +message string eagerly inside `exception/1` (storing it in `:message`) +rather than via an `Exception.message/1` callback — +`ArgumentError` is the one that uses a `message/1` callback. The file +list above reflects the actual code. ## Verification @@ -129,17 +198,140 @@ Manual: pipe to a file, confirm no raw ANSI escape codes leak. elixir -e 'IO.puts(Lua.eval!(Lua.new(), ~S{print(nil + 1)}))' > /tmp/out 2>&1 || cat /tmp/out ``` +(Expected: the rescued message text, with no `\e[` escape sequences in +`/tmp/out`.) + ## Risks - "World-class" is subjective. Resist the urge to bikeshed; the minimum bar is "every error has line/source up top, every error has - a category-appropriate body, no filler suggestions." -- Fixture-based tests are brittle if the format keeps changing. Lock - the format here and treat changes as opt-in. -- If the audit surfaces structural issues (e.g. a category that - doesn't carry the data it should), open follow-up plans rather than - expanding scope. + a category-appropriate body, no filler suggestions, no ANSI leak off + a TTY." +- Pinned-output tests are brittle if the format keeps changing. Lock + the format here and treat changes as opt-in (update the inline + `expected` strings deliberately, never auto-pass). +- ANSI gating must not change the `to_map/3` wire-safe path, which is + already documented to contain no escapes — only `format/3` should be + affected. +- If the audit surfaces structural data issues (a category that + doesn't carry the line/source it should), open follow-up plans + rather than expanding scope. The two A19-flagged data oddities are + explicitly out of scope here. ## Discoveries (populated during implementation) + +Pre-implementation notes carried over from the audit: + +- **A19 is `status: review`, not merged to `main`.** If the branch is + cut from a `main` that predates A19's merge, the four exception + modules will not auto-populate `:line`/`:source`/`:call_stack` and + many fixtures will render without a location line. Before + generating fixtures, confirm `Lua.VM.TypeError.exception([value: + "x"])` returns a struct with `:line`/`:source` populated when called + inside a Lua execution; if not, A19 has not landed and this plan + should wait (it remains blocked in practice even though the data is + designed to be present). Audit what *is* present and note any + category still missing line info here. (Resolved below: A19 data is + present, and the gallery pins output via inline `expected` strings + rather than checked-in fixture files.) +- **Dead suggestion clause.** `build_suggestion(:type_error, + :arithmetic_type_error, _)` is never reached — the executor emits + `:arithmetic_on_non_number`. Confirmed by grep of `lib/`. +- **Categories with no emitted suggestion today:** + `:compare_incompatible_types`, `:length_not_integer`, + `:bitwise_on_non_integer`, `:for_loop_non_number`. The first two are + in the Success-criteria gallery list and get suggestions here; the + latter two are noted for completeness. +- **Table-index-nil/NaN raise sites:** `lib/lua/vm/table.ex` documents + the contract (§3.4.11) but the actual raise happens in the executor + set path. Confirm the rendered message and whether it carries an + `error_kind` before writing that case; if it has no kind, the + case pins the body only (no suggestion) — do not invent a kind. + +### Confirmed during implementation + +- **A19 data IS present on this base.** `Lua.VM.Executor.current_position/0` + exists, all nine `error_kind`s are emitted, and the four VM exceptions + auto-populate `:line`/`:source`. Every type/argument/runtime/assertion + error reachable from `Lua.eval!` carries a location, so the gallery + cases render `at gallery.lua::` up top — except the + stack-overflow case and the numeric `for`-loop control coercion case + (see below). + +- **Out-of-scope data-layer gaps (logged, not fixed):** + - `#5` (length operator on a number) and `#true` return `0` rather than + raising. The `:length_not_integer` kind is wired into the formatter as + a suggestion, but the executor never raises it for these inputs, so + "length on non-string/table" has no rendered message to pin. Data-layer + bug; out of scope. + - `t[nil] = 1` and `t[0/0] = 1` (nil / NaN table key) succeed silently + instead of raising "table index is nil/NaN". No rendered message to + pin. Data-layer bug; out of scope. These two categories are therefore + documented here under "Known gaps" (and in the PR description) rather than + pinned in the gallery. + - Stack-overflow runtime errors carry no originating line, so they render + without an `at ::` header and the stack frames show line + `0`. The renderer is correct given the data; the missing line is a + data-layer gap. + - Numeric `for`-loop control coercion errors (`for_loop_non_number`) also + render without an `at ::` header: the `raise` in + `coerce_for_value/2` receives no position from `current_position/0`, so + the location is `nil`. Identical on `main`, not a regression; the + gallery honestly pins the location-less output. Data-layer gap. + +- **`error()` with a non-string object** previously leaked an internal + Elixir term (`runtime error: {:tref, 12}`). `Lua.VM.RuntimeError.stringify/1` + now renders PUC-Lua's `(error object is a TYPE value)` for any non-string, + non-number value, while strings and numbers still render verbatim. This is + a rendering fix on a file already in scope. + +- **`assert(false)` still double-texts** as `assertion failed: assertion + failed!` because `AssertionError.raw_message/1` prefixes + `"assertion failed: "` and the stdlib passes the literal default value + `"assertion failed!"`. Removing the prefix is a behavioral change that + ripples into `test/lua53_tests/errors.lua` and + `test/lua/compiler/integration_test.exs`; the plan scoped the assertion + change to its *suggestion* (now removed), not the message prefix. Left as + a follow-up. + +- **Tests updated as a direct consequence of the formatter change** (the + golden snapshots tested the old header + unconditional ANSI): + `test/lua/vm/error_to_map_test.exs` and `test/lua/error_messages_test.exs`. + These are not in the plan's file list but directly assert the formatter's + output; updating them is a necessary consequence of the in-scope change, + not a scope expansion. + +## What changed + +Shipped in PR #304. + +Files touched: + +- `lib/lua/vm/error_formatter.ex` — `format/3` now leads with the location + line, drops the standalone header (resolving the double label), gates all + ANSI behind a `color/2` helper keyed on `IO.ANSI.enabled?/0`, realigns the + dead `:arithmetic_type_error` suggestion to the emitted + `:arithmetic_on_non_number`, adds suggestions for + `:compare_incompatible_types`, `:length_not_integer`, + `:bitwise_on_non_integer`, `:for_loop_non_number`, and removes the filler + assertion suggestion. +- `lib/lua/vm/runtime_error.ex` — non-string/number `error()` objects render + PUC-Lua's `(error object is a TYPE value)` instead of an internal term. +- `test/lua/error_gallery_test.exs` (new) — pins the rendered output for + every reachable category with an inline `expected` string per case (no + separate fixture files); update those strings deliberately when the format + changes on purpose. +- The standalone `guides/errors.md` gallery was dropped during review: the + before/after renders are pinned by `test/lua/error_gallery_test.exs` and the + "Known gaps" data-layer holes are captured in this plan's Discoveries and the + PR description instead of a separate guide. +- `test/lua/vm/error_to_map_test.exs`, `test/lua/error_messages_test.exs` — + golden snapshots updated to the new format. + +Tests: `mix test` 2104 passed / 0 failed; `mix test --only lua53` 17 passed. + +Follow-ups to open (out of scope here): length operator / nil-NaN table key +not raising, stack-overflow errors carrying no line, and the +`assert(false)` double-text prefix. diff --git a/lib/lua/vm/error_formatter.ex b/lib/lua/vm/error_formatter.ex index 4d0c8557..4e2e9865 100644 --- a/lib/lua/vm/error_formatter.ex +++ b/lib/lua/vm/error_formatter.ex @@ -4,15 +4,33 @@ defmodule Lua.VM.ErrorFormatter do Two entry points share the same underlying data: - * `format/3` renders an ANSI-colored multi-line string for terminal output - (headers, source context with pointer, stack trace, suggestion). + * `format/3` renders a multi-line string for terminal output (location, + message body, source context with pointer, stack trace, suggestion). + ANSI color is applied only when `IO.ANSI.enabled?/0` is true, so the + same render path is safe whether the output goes to a TTY or is piped + to a file. * `to_map/3` returns a wire-safe structured map for non-terminal consumers (HTML rendering, JSON payloads, structured logs). No ANSI escapes appear in any string field. + + The rendered layout leads with the source location so the reader sees + *where* before *what*: + + at demo.lua:3: + + attempt to perform arithmetic on a nil value + ... + + There is deliberately no separate "Runtime Type Error" header line — the + public `Lua.RuntimeException` already prefixes `Lua runtime error: `, and + the message body itself ("attempt to ...", "assertion failed: ...") names + the category. Stacking a second header would be redundant. """ @doc """ - Formats a runtime error into a multi-line ANSI-colored string. + Formats a runtime error into a multi-line string. + + ANSI color is applied only when `IO.ANSI.enabled?/0` returns true. ## Options @@ -32,16 +50,16 @@ defmodule Lua.VM.ErrorFormatter do error_kind = Keyword.get(opts, :error_kind) value_type = Keyword.get(opts, :value_type) - header = format_header(error_type) location = format_location(source, line) message_section = format_message(message) context = format_source_context(source_code, line) stack_trace = format_stack_trace(call_stack) suggestion = format_suggestion(error_type, error_kind, value_type) - [header, location, message_section, context, stack_trace, suggestion] + [location, message_section, context, stack_trace, suggestion] |> Enum.reject(&is_nil/1) - |> Enum.join("\n") + |> Enum.join("") + |> String.trim_leading() end @doc """ @@ -92,28 +110,23 @@ defmodule Lua.VM.ErrorFormatter do } end - defp format_header(:type_error) do - IO.ANSI.red() <> IO.ANSI.bright() <> "Runtime Type Error" <> IO.ANSI.reset() - end - - defp format_header(:runtime_error) do - IO.ANSI.red() <> IO.ANSI.bright() <> "Runtime Error" <> IO.ANSI.reset() - end - - defp format_header(:assertion_error) do - IO.ANSI.red() <> IO.ANSI.bright() <> "Assertion Failed" <> IO.ANSI.reset() + # ANSI helpers — no-ops unless the terminal actually supports color, so + # piping to a file or a non-TTY never embeds raw escape codes. + defp color(text, ansi) do + if IO.ANSI.enabled?() do + ansi <> text <> IO.ANSI.reset() + else + text + end end - defp format_header(_), do: IO.ANSI.red() <> IO.ANSI.bright() <> "Error" <> IO.ANSI.reset() - defp format_location(nil, nil), do: nil - defp format_location(source, nil), do: "\n at #{source}:" - defp format_location(nil, line), do: "\n at line #{line}:" - defp format_location(source, line), do: "\n at #{source}:#{line}:" + defp format_location(source, nil), do: color("at #{source}:", IO.ANSI.faint()) <> "\n\n" + defp format_location(nil, line), do: color("at line #{line}:", IO.ANSI.faint()) <> "\n\n" - defp format_message(message) do - "\n #{message}" - end + defp format_location(source, line), do: color("at #{source}:#{line}:", IO.ANSI.faint()) <> "\n\n" + + defp format_message(message), do: " #{message}" defp format_source_context(source_code, line) do case build_source_context(source_code, line) do @@ -131,14 +144,14 @@ defmodule Lua.VM.ErrorFormatter do pointer_offset = String.length(format_line_number(num)) + 3 pointer = - String.duplicate(" ", pointer_offset) <> IO.ANSI.red() <> "^" <> IO.ANSI.reset() + String.duplicate(" ", pointer_offset) <> color("^", IO.ANSI.red()) [ - "\n" <> IO.ANSI.red() <> line_str <> IO.ANSI.reset(), - pointer + "\n" <> color(line_str, IO.ANSI.red()), + "\n" <> pointer ] else - ["\n" <> IO.ANSI.faint() <> line_str <> IO.ANSI.reset()] + ["\n" <> color(line_str, IO.ANSI.faint())] end end) @@ -194,7 +207,7 @@ defmodule Lua.VM.ErrorFormatter do |> truncate_frames() |> Enum.map_join("\n", &format_frame/1) - "\n\n" <> IO.ANSI.cyan() <> "Stack trace:" <> IO.ANSI.reset() <> "\n" <> frames + "\n\n" <> color("Stack trace:", IO.ANSI.cyan()) <> "\n" <> frames end defp truncate_frames(frames) do @@ -221,8 +234,7 @@ defmodule Lua.VM.ErrorFormatter do defp build_call_stack(_), do: [] defp format_frame({:omitted, count}) do - IO.ANSI.faint() <> - " ... #{count} more frames ..." <> IO.ANSI.reset() + color(" ... #{count} more frames ...", IO.ANSI.faint()) end defp format_frame(%{source: source, line: line, name: name}) do @@ -256,22 +268,34 @@ defmodule Lua.VM.ErrorFormatter do "You can only index tables. Make sure the value you're indexing is a table, not a #{format_type_name(value_type)}." end - defp build_suggestion(:type_error, :arithmetic_type_error, _value_type) do - "Arithmetic operations require numbers. Make sure both operands are numbers or strings that can be converted to numbers." + defp build_suggestion(:type_error, :arithmetic_on_non_number, _value_type) do + "Arithmetic requires numbers. Make sure both operands are numbers (or strings that can be coerced to numbers)." end defp build_suggestion(:type_error, :concatenate_type_error, _value_type) do - "String concatenation (..) requires strings or numbers, not other types." + "Concatenation (..) requires strings or numbers. Convert other values with tostring() first." + end + + defp build_suggestion(:type_error, :compare_incompatible_types, _value_type) do + "Relational operators (< <= > >=) only compare two numbers or two strings. Convert one operand so both sides share a type." + end + + defp build_suggestion(:type_error, :length_not_integer, _value_type) do + "The length operation returned a non-integer value. If you defined a __len metamethod, make sure it returns an integer." + end + + defp build_suggestion(:type_error, :bitwise_on_non_integer, _value_type) do + "Bitwise operators require integers. Floats with no fractional part are accepted; everything else must be converted first." end - defp build_suggestion(:assertion_error, _error_kind, _value_type) do - "The assertion condition evaluated to false or nil. Check your logic." + defp build_suggestion(:type_error, :for_loop_non_number, _value_type) do + "Numeric for loops need number values for the start, limit, and step. Check the loop bounds." end defp build_suggestion(_, _, _), do: nil defp suggestion(text) do - "\n\n" <> IO.ANSI.cyan() <> "Suggestion:" <> IO.ANSI.reset() <> "\n " <> text + "\n\n" <> color("Suggestion:", IO.ANSI.cyan()) <> "\n " <> text end defp format_type_name(:number), do: "number" diff --git a/lib/lua/vm/runtime_error.ex b/lib/lua/vm/runtime_error.ex index 3e466d91..a42055a1 100644 --- a/lib/lua/vm/runtime_error.ex +++ b/lib/lua/vm/runtime_error.ex @@ -14,6 +14,7 @@ defmodule Lua.VM.RuntimeError do """ alias Lua.VM.ErrorFormatter + alias Lua.VM.Value defexception [:value, :source, :message, :call_stack, :line] @@ -61,7 +62,13 @@ defmodule Lua.VM.RuntimeError do defp raw_message(value), do: "runtime error: #{stringify(value)}" - defp stringify(nil), do: "nil" + # PUC-Lua renders string and number error objects verbatim, but any other + # Lua value (table, function, boolean, nil, userdata) becomes the message + # "(error object is a TYPE value)" rather than leaking an internal term. defp stringify(v) when is_binary(v), do: v - defp stringify(v), do: inspect(v) + defp stringify(v) when is_integer(v) or is_float(v), do: Value.to_string(v) + + defp stringify(value) do + "(error object is a #{Value.type_name(value)} value)" + end end diff --git a/test/lua/error_gallery_test.exs b/test/lua/error_gallery_test.exs new file mode 100644 index 00000000..d60c0eff --- /dev/null +++ b/test/lua/error_gallery_test.exs @@ -0,0 +1,160 @@ +defmodule Lua.ErrorGalleryTest do + @moduledoc """ + Locks the user-visible rendered output for every error category a Lua + program can hit. Each case evaluates a snippet and compares the public + `Exception.message/1` against an expected string defined inline next to + the source. + + The suite runs with ANSI disabled, so the expected text is stable across + terminals. When the format changes on purpose, update the `expected` + string for the affected case. + """ + + use ExUnit.Case, async: true + + # {name, lua source, eval opts, expected rendered message}. A finite + # max_call_depth is set on the stack-overflow case so the recursion + # terminates deterministically. + @cases [ + {"arithmetic_on_non_number", "local x = nil\nprint(x + 1)", [], + """ + Lua runtime error: at gallery.lua:2: + + attempt to perform arithmetic on a nil value (local 'x') + + Suggestion: + Arithmetic requires numbers. Make sure both operands are numbers (or strings that can be coerced to numbers).\ + """}, + {"index_nil", "local t = nil\nprint(t.field)", [], + """ + Lua runtime error: at gallery.lua:2: + + attempt to index a nil value (local 't') + + Suggestion: + You can only index tables. Make sure the value you're indexing is a table, not a nil.\ + """}, + {"call_nil", "local f = nil\nf()", [], + """ + Lua runtime error: at gallery.lua:2: + + attempt to call a nil value (local 'f') + + Suggestion: + The value you're trying to call as a function is nil. Check that the function exists and is defined before this point.\ + """}, + {"concat_non_string", "local t = {}\nprint(t .. \"x\")", [], + """ + Lua runtime error: at gallery.lua:2: + + attempt to concatenate a table value + + Suggestion: + Concatenation (..) requires strings or numbers. Convert other values with tostring() first.\ + """}, + {"compare_incompatible", "print(1 < \"x\")", [], + """ + Lua runtime error: at gallery.lua:1: + + attempt to compare number with string + + Suggestion: + Relational operators (< <= > >=) only compare two numbers or two strings. Convert one operand so both sides share a type.\ + """}, + {"bitwise_on_non_integer", "print(1 & \"x\")", [], + """ + Lua runtime error: at gallery.lua:1: + + attempt to perform bitwise operation on a string value + + Suggestion: + Bitwise operators require integers. Floats with no fractional part are accepted; everything else must be converted first.\ + """}, + {"for_loop_non_number", "for i = 1, \"x\" do end", [], + """ + Lua runtime error: 'for' limit must be a number + + Suggestion: + Numeric for loops need number values for the start, limit, and step. Check the loop bounds.\ + """}, + {"stdlib_bad_arg", "string.upper(nil)", [], + """ + Lua runtime error: at gallery.lua:1: + + bad argument #1 to 'string.upper' (string expected, got nil)\ + """}, + {"assert_with_message", "assert(false, \"boom\")", [], + """ + Lua runtime error: at gallery.lua:1: + + assertion failed: boom\ + """}, + {"assert_no_message", "assert(false)", [], + """ + Lua runtime error: at gallery.lua:1: + + assertion failed: assertion failed!\ + """}, + {"error_string", "error(\"something broke\")", [], + """ + Lua runtime error: at gallery.lua:1: + + runtime error: something broke\ + """}, + {"error_table", "error({code = 1})", [], + """ + Lua runtime error: at gallery.lua:1: + + runtime error: (error object is a table value)\ + """}, + {"error_number", "error(1.5)", [], + """ + Lua runtime error: at gallery.lua:1: + + runtime error: 1.5\ + """}, + {"stack_overflow", "local function f(n) return 1 + f(n + 1) end\nf(1)", [max_call_depth: 30], + """ + Lua runtime error: runtime error: stack overflow + + Stack trace: + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + ... 20 more frames ... + gallery.lua:0: in function 'f' + gallery.lua:0: in function 'f' + gallery.lua:2: in function 'f'\ + """} + ] + + setup do + refute IO.ANSI.enabled?(), "gallery assumes ANSI is disabled in the test env" + :ok + end + + for {name, source, opts, expected} <- @cases do + test "gallery: #{name}" do + source = unquote(source) + opts = unquote(opts) + expected = unquote(expected) + + assert render(source, opts) == expected + end + end + + defp render(source, opts) do + lua = Lua.new(opts) + + try do + Lua.eval!(lua, source, source: "gallery.lua") + flunk("expected #{inspect(source)} to raise") + rescue + e -> Exception.message(e) + end + end +end diff --git a/test/lua/error_messages_test.exs b/test/lua/error_messages_test.exs index ac7622c3..6d01a263 100644 --- a/test/lua/error_messages_test.exs +++ b/test/lua/error_messages_test.exs @@ -281,9 +281,9 @@ defmodule Lua.ErrorMessagesTest do message = "attempt to call a nil value" formatted = ErrorFormatter.format(:type_error, message, source: "test.lua", line: 5) - assert formatted =~ "Runtime Type Error" - assert formatted =~ "test.lua" - assert formatted =~ "5" + # Location leads the output, on its own line before the message body. + assert String.starts_with?(formatted, "at test.lua:5:") + assert formatted =~ message end test "includes suggestions for common errors" do diff --git a/test/lua/vm/error_to_map_test.exs b/test/lua/vm/error_to_map_test.exs index 35c43699..58b4c1c8 100644 --- a/test/lua/vm/error_to_map_test.exs +++ b/test/lua/vm/error_to_map_test.exs @@ -95,27 +95,52 @@ defmodule Lua.VM.ErrorToMapTest do assert map.suggestion =~ "number" refute String.contains?(map.suggestion, @ansi_escape) end + + test ":length_not_integer suggestion addresses the non-integer result" do + error = + VMTypeError.exception( + value: "object length is not an integer", + source: "t.lua", + line: 1, + error_kind: :length_not_integer, + value_type: :number + ) + + map = VMTypeError.to_map(error) + + assert map.suggestion =~ "non-integer" + assert map.suggestion =~ "__len" + end end describe "Lua.VM.AssertionError.to_map/2" do - test "returns assertion suggestion without ANSI" do + test "carries the message and no filler suggestion" do error = VMAssertionError.exception(value: "expected true", source: "t.lua", line: 1) map = VMAssertionError.to_map(error) assert map.type == :assertion_error assert map.message == "assertion failed: expected true" - assert is_binary(map.suggestion) - refute String.contains?(map.suggestion, @ansi_escape) + # Assertions no longer carry a generic "check your logic" suggestion; + # the message body already says what failed. + assert map.suggestion == nil assert map.error_kind == nil end end describe "Lua.VM.ErrorFormatter.format/3 stability" do - # Golden snapshots locking the terminal output. If a refactor changes - # what terminal callers see, these fail loudly. + # Golden snapshots locking the rendered output. The test suite runs with + # ANSI disabled, so these pin the plain-text render: location first, no + # redundant header, category-specific suggestion. If a refactor changes + # what callers see, these fail loudly. + + test "ANSI is gated on IO.ANSI.enabled?" do + refute IO.ANSI.enabled?(), "test env should have ANSI disabled" + output = ErrorFormatter.format(:type_error, "attempt to call a nil value", source: "t.lua", line: 2) + refute String.contains?(output, @ansi_escape) + end - test "type_error with source_code, stack, and suggestion" do + test "type_error leads with location and emits a category suggestion" do output = ErrorFormatter.format(:type_error, "attempt to call a nil value", source: "t.lua", @@ -126,7 +151,7 @@ defmodule Lua.VM.ErrorToMapTest do ) expected = - "\e[31m\e[1mRuntime Type Error\e[0m\n\n at t.lua:2:\n\n attempt to call a nil value\n\n\n\e[2m 1 │ local x\e[0m\n\e[31m 2 │ x()\e[0m \e[31m^\e[0m\n\e[2m 3 │ return 0\e[0m\n\n\n\e[36mStack trace:\e[0m\n t.lua:2: in main chunk\n\n\n\e[36mSuggestion:\e[0m\n The value you're trying to call as a function is nil. Check that the function exists and is defined before this point." + "at t.lua:2:\n\n attempt to call a nil value\n\n 1 │ local x\n 2 │ x()\n ^\n 3 │ return 0\n\nStack trace:\n t.lua:2: in main chunk\n\nSuggestion:\n The value you're trying to call as a function is nil. Check that the function exists and is defined before this point." assert output == expected end @@ -140,21 +165,15 @@ defmodule Lua.VM.ErrorToMapTest do ) expected = - "\e[31m\e[1mRuntime Error\e[0m\n\n at t.lua:1:\n\n runtime error: boom\n\n\n\e[36mStack trace:\e[0m\n t.lua:1: in function 'f'" + "at t.lua:1:\n\n runtime error: boom\n\nStack trace:\n t.lua:1: in function 'f'" assert output == expected end - test "assertion_error with empty stack still emits suggestion" do + test "assertion_error carries the body and no filler suggestion" do output = ErrorFormatter.format(:assertion_error, "assertion failed: nope") - expected = - "\e[31m\e[1mAssertion Failed\e[0m" <> - "\n\n assertion failed: nope" <> - "\n\n\n\e[36mSuggestion:\e[0m" <> - "\n The assertion condition evaluated to false or nil. Check your logic." - - assert output == expected + assert output == "assertion failed: nope" end end end diff --git a/test/lua/vm/for_loop_register_test.exs b/test/lua/vm/for_loop_register_test.exs index 2c54fd15..6efcd272 100644 --- a/test/lua/vm/for_loop_register_test.exs +++ b/test/lua/vm/for_loop_register_test.exs @@ -12,7 +12,7 @@ defmodule Lua.VM.ForLoopRegisterTest do # register file: the second loop saw `nil` where its loop counter should # have been, producing: # - # Runtime Type Error — attempt to perform arithmetic on a nil value + # Lua runtime error: attempt to perform arithmetic on a nil value # # The original Phase 17 fix (commit e7c50e5) ensured the for-loop's # internal counter/limit/step registers did not leak into the surrounding