Repository navigation
feat(vm): conformant goto/label on both engines - #364
Conversation
The list interpreter resolved `goto` with a forward-only `find_label`
scan of the current instruction list. Backward jumps (manual loops),
`continue`-style jumps out of an `if`, and break-style jumps out of a
loop all raised "goto target not found"; reused label names could match
the wrong label. A tail-recursive walker has already consumed the block
head at the goto, so labels cannot be resolved at run time.
Resolve them ahead of execution instead. `Lua.Compiler.GotoResolution`
rewrites each `{:goto, name}` to `{:goto, id}` and records
`proto.goto_targets[id] = {depth, level, target_tail}`:
* `depth` — `cont` entries to drop to leave the blocks between the
goto and its target (`:test` = 1, loop body = 2), so the remaining
`cont` is exactly the destination block's entry continuation.
* `level` — the target label's live-local register watermark (recorded
by scope resolution on `{:label, name, level}`); the goto closes open
upvalue cells at or above it, matching Lua 5.3 §3.3.4 block-exit.
* `target_tail` — the destination block's suffix after the label.
The pass runs after bytecode encoding, so the encoder still reads the
raw goto/label stream (the dispatcher port is separate). New conformance
tests in goto_test.exs cover backward/continue/break/nested/reused-label
and upvalue-capture cases on both engines.
goto.lua: skip the `debug.upvalueid` upvalue-cell-identity assertions
(163-195) — `debug.upvalueid` is a stub returning nil. The goto control
flow those closures exercise stays covered by `assert(#a == 6)`.
With the interpreter's goto now conformant, port goto/label to the
dispatcher so goto programs stop falling back. The encoder resolves
gotos over the encoded tuple tree (mirroring GotoResolution for the
interpreter, with PCs instead of list tails):
* `:label` → `{@op_label, name, level}`, a no-op anchoring a target PC.
* `:goto` → `{@op_goto, depth, target_pc, level}` via `resolve_gotos/2`:
`depth` `cont` entries to drop (`:test` = 1, loop body = 2), the
label's PC in the destination tuple, and the upvalue-close level.
The dispatcher gains `@op_label` (no-op) and `@op_goto` handlers: close
open upvalues at/above `level`, drop `depth` `cont` entries, and resume
the destination tuple (recovered from the unwound markers via
`unwind_goto/2`, reusing the shape `:break`'s `find_loop_exit/1` relies
on) at `target_pc`. The hot path is untouched — three new jump-table
clauses for a rare opcode.
`Bytecode.fully_compiled?/1`'s corpus now includes goto programs (they
compile end-to-end); the one remaining documented exception is
short-circuit `and`/`or` (`:test_and` / `:test_or`), which the original
exploration had overlooked and which is left for a follow-up. The
dispatcher_test `encodes_op?` helper now descends into nested body
tuples so opcodes emitted inside a block are detected.
🤖 Automated multi-agent review — stabilization cycle5 reviewers fanned out across correctness/conformance/dispatcher/tests/quality dimensions; every finding was then adversarially verified by an independent agent that ran repros against the branch tip and PUC-Lua. Raised 15 → 11 confirmed (3 HIGH, 8 MEDIUM), 4 refuted. The 11 confirmed findings collapse into 4 distinct issues. The first directly contradicts the PR's central "conformant goto/label" claim, so flagging it before merge. 🔴 HIGH — Flat-inlined
|
….3.4)
Codegen flat-inlines `do ... end` blocks into the parent instruction list,
so a do-block's labels leaked into the parent's flat label search. Both goto
resolvers (interpreter `GotoResolution`, dispatcher `Bytecode.resolve_gotos`)
used a scope-blind "nearest after, else nearest before" heuristic over that
flattened list, binding a goto to labels in nested or sibling blocks.
Scope resolution now tags every `{:label, name, level, block_path}` and
`{:goto, name, block_path}` with the lexical do-block path (innermost-last,
unique per block). A label is visible to a goto only when the label's path is
a prefix of the goto's — its own block or an enclosing one, never a nested or
sibling block. Both engines apply the same prefix test, so they agree.
Adds `Lua.Compiler.GotoValidation`, a static legality pass that runs before
both resolvers and returns a compile error (surfaced as `Lua.CompilerException`
via `compile/2`/`compile!`) for the programs PUC-Lua rejects at compile time:
duplicate label in a scope, undefined/out-of-scope goto, and a forward jump
into the scope of a local. This removes the dispatcher's bare "codegen should
not emit this" RuntimeError on user-reachable undefined gotos and makes the
two engines diverge no more.
The suite's `load` helper now returns `nil, message` on a compile error (not
raise), matching PUC, so `goto.lua`'s illegal-goto `errmsg(...)` rejections
pass; un-skips goto.lua lines 12-40 (only the debug.upvalueid range remains).
Repros (both engines), PUC value | before -> after:
local x=0; goto l; do ::l:: x=100 end; ::l:: x=x+5; return x 5 | 105 -> 5
do ::a:: ... goto a end do goto a ... ::a:: ... end 1002 | 1001 -> 1002
do ::l:: ... end do ::l:: ... end (sibling reuse) 2,3 | 1,3 -> 2,3
`Lua.Parser.parse/1` and `Lua.Compiler.compile/2` both return a binary error message, so the non-binary `format_load_error/1` clause was dead (`pattern_match_cov`). Inline the binary message into `load`'s second return value and remove the helper.
✅ Conformance fixes landed (commits
|
| repro | before | after / PUC |
|---|---|---|
jump into nested do label |
105 |
5 |
| backward goto vs forward sibling label | 1001 |
1002 |
| sibling-block reuse | [1,3] |
[2,3] |
Rejections now compile-error (both engines), matching PUC wording:
goto nowhere→no visible label 'nowhere' for goto::a:: ::a::→label 'a' already defineddo goto skip; local y=10; ::skip:: end→jumps into the scope of local 'y'
This also removes the bare "codegen should not emit this" internal raise (BUG 3) — undefined/out-of-scope gotos are now a clean compile error consistently across both engines — and the mis-categorized goto.lua suite skip (BUG 4): the illegal-goto rejection block (lines 12–40) now runs and passes; only the debug.upvalueid range stays skipped.
Verification: mix test 2450 passed · mix test --only lua53 17 passed (goto.lua's errmsg block now exercised) · mix format clean · mix compile --warnings-as-errors clean · Dialyzer clean · full CI green.
Fixes implemented by an automated agent, then independently verified against PUC-Lua (lua-5.5/luac-5.5) and CI by the orchestrator. The first push tripped CI Dialyzer (a dead format_load_error clause); 009b880 fixes it.
Context
Second PR in the effort to drive the bytecode Dispatcher to full opcode coverage (follows #362). The remaining real gap was
:goto/:label.While implementing the dispatcher port, I found the premise was wrong: the interpreter's
gotowas itself non-conformant. It resolved labels with a forward-onlyfind_labelscan of the current instruction list, so:continue(out of anif)So there was no correct oracle to mirror. This PR makes
goto/labelcorrect on both engines.Approach
A tail-recursive list walker has already consumed the block head at a goto, so labels are resolved ahead of execution rather than scanned at runtime. Both engines have analogous
contnesting (:testpushes one entry, loops two), so a goto resolves to{depth-to-unwind, target, close-level}on each — differing only in the target representation.Lua.Compiler.GotoResolution, runs after codegen): rewrites{:goto, name}→{:goto, id}and recordsproto.goto_targets[id] = {depth, level, target_tail}. The goto handler dropsdepthcontentries, closes upvalues>= level, and resumestarget_tail.Bytecode.resolve_gotos/2, over the encoded tuple tree)::label→{@op_label, …}no-op;:goto→{@op_goto, depth, target_pc, level}. The handler closes upvalues, unwindsdepthcontentries (unwind_goto/2, reusing:break's marker shape), and resumes the destination tuple attarget_pc.{:label, name, level}; a goto closes open upvalue cells>= level, matching Lua 5.3 §3.3.4 block-exit (verified via the upvalue-capture test).doblocks are flat-inlined, so reused label names collapse into one list; resolution picks the nearest label after the goto (else before), matching lexical intent for thecontinue/break/backward-loop idioms.Coverage status (honest)
goto/labelare now covered. One gap remains that the original exploration missed: short-circuitand/or(:test_and/:test_or) still falls back to the interpreter. It's now the single documented exception in thefully_compiled?/1corpus guard, left for a focused follow-up. So this PR closes the known gap but the dispatcher is not yet at literal 100%.Tests / verification
test/lua/vm/goto_test.exs— runs each program on both engines (bytecode stripped vs kept) and asserts equal, correct results: forward/backward/continue/break/nested/reused-label/upvalue-capture.dispatcher_test.exs— a:goto / :labelblock asserting@op_gotois encoded (helper now descends into nested body tuples) and runs correctly.bytecode_test.exs— goto programs added to the coverage corpus; fallback/cascade tests reworked aroundand/or.fib(20) dispatcher == interpreter. (Benchee not installed in this env, so the perf comparison wasn't run; the hot path gains only three jump-table clauses for a rare opcode.)goto.lua skip
goto.lua lines 163-195 (the
debug.upvalueidcell-identity assertions) are now skipped:debug.upvalueidis a stub returningnil, so those assertions can't pass once the (now-correct) goto control flow reaches them. The control flow itself stays covered by the precedingassert(#a == 6).