Repository navigation
fix(parser): report syntax errors at the real offending token - #357
Merged
Merged
Conversation
A deeply-nested sub-parse failure was swallowed by the partial-list
recovery branch in `parse_expr_list_acc/2`: once one or more arguments
had parsed, any later failure returned the partial list with the stream
left pointing at the start of the failed expression, so the caller's
`expect(rparen)` blamed that token at the list boundary instead of the
actual mistake.
For example:
w:step("a", "b", "c", {x = 1}, function()
bar(
end)
reported "Expected :delimiter::rparen, got :keyword::function" at the
`function` keyword, three lines above the unclosed `bar(` / unexpected
`end`. Reference Lua (`luac -p`) points at the `end`; we now agree.
Adopt the recursive-descent "commit on progress" rule: only recover
(treat the list as ended) when the sub-parse made no progress — i.e. the
next token simply cannot begin an expression. A committed failure (input
consumed, or EOF) propagates as the real error. Progress is detected via
the monotonic `byte_offset` already on every token.
Also surface genuinely-unclosed delimiters precisely: `parse_call_args`,
`parse_index`, `parse_table`, and parenthesised expressions now capture
their opening position and, when the closer is missing at EOF, emit the
existing `Error.unclosed_delimiter/2` ("Unclosed (… add a closing ) to
match the opening at line N") pointing at the opener rather than a bare
"got :eof" at end-of-file.
Add `test/lua/parser/error_position_test.exs`: a CI-deterministic golden
corpus pinning the reported line for deep errors, over-propagation
guards, and unclosed constructs, plus a `:differential` test that
cross-checks the reported line against `luac -p` when available.
The new luac-differential error-position tests shell out to a reference `luac` and assert exact line-number agreement. That is environment- and version-dependent: CI has no compatible `luac`, so the suite (which only excluded `:slow`) ran them and they failed. They were always intended as an opt-in local check alongside the CI-deterministic golden corpus. Add `:differential` to the default exclude list; run them explicitly with `mix test --include differential`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A user pasted a workflow program with a genuine syntax error — an unclosed call whose
)is never supplied before the enclosingend:The parser reported:
…with the caret on the
functionkeyword on the outerw:step(...)line — ~3 lines above the real mistake. Reference Lua (luac -p) points at theend(unexpected symbol near 'end').Root cause
The error-swallowing recovery branch in
parse_expr_list_acc/2:Parsing the
w:step(...)argument list, 4 arguments parse, then the 5th (thefunctionliteral) fails deep inside (the uncloseddevice.doesntExist(). Becauseaccis non-empty, the branch throws away that real error and returns the partial list with the stream still pointed atfunction; the caller'sexpect(rparen)then blamesfunction.This was the only error-swallowing recovery site in the parser (audited).
Fix
Adopt the recursive-descent commit-on-progress rule: recover (treat the list as ended) only when the sub-parse made no progress — the next token simply can't begin an expression. A committed failure (input consumed, or EOF) propagates as the real error. Progress is detected via the monotonic
byte_offsetalready present on every token.Also surface genuinely-unclosed delimiters precisely:
parse_call_args,parse_index,parse_table, and parenthesised expressions now capture their opening position and, when the closer is missing at EOF, emit the existingError.unclosed_delimiter/2("Unclosed ( … add a closing ) to match the opening at line N") pointing at the opener instead of a baregot :eof.After the fix, the reported program points at the
end, matchingluac.Tests
New
test/lua/parser/error_position_test.exs:@tag :differential, skipped withoutluac): cross-checks our reported line againstluac -pfor every concrete-offending-token case — all agree.Verification
mix test— 2353 passed, 0 failures (19 pre-existing skips)mix test test/lua/parser/error_position_test.exs --include differential— 13 passedmix format --check-formattedclean🤖 Generated with Claude Code