Skip to content

fix(parser): propagate deep errors past function-call argument lists - #366

Merged
davydog187 merged 3 commits into
mainfrom
fix/parser-deep-error-position
Jun 15, 2026
Merged

davydog187 merged 3 commits into
mainfrom
fix/parser-deep-error-position

Conversation

@davydog187

@davydog187 davydog187 commented Jun 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A parse error deep inside a function-call argument was reported at the opening line of the call, not where the mistake actually is. Given:

w:step("w.h42zel", "code", "My Code", { continueOnError = false }, function(output)
    local usercode = function()
      device.doesntExist)   -- line 3: the real error
    end
    return usercode()
end)

the parser emitted Expected :delimiter::rparen, got :keyword::function pinned at line 1 (the w:step( opener) instead of line 3.

Root cause

In parse_expr_list_acc/2, when an argument sub-parse fails, committed?/2 decides whether the failure is a real deep error to propagate or a clean signal that the argument list ended. Its fallback clause derives "committed" by comparing the error's byte offset against the start token via error_position/1, which only understands :unexpected_token errors:

defp error_position({:unexpected_token, _type, pos, _message}), do: pos
defp error_position(_), do: nil

The actual failure is a {:bare_expression, …} error (device.doesntExist is a bare table access, not a valid statement). That shape returned nil, so the committed? fallback reported "not committed", the list "recovered" as if the arguments ended just before function, and expect_closing produced the misleading "Expected rparen" at the opener. This is exactly the failure mode the comment at parse_expr_list_acc/2 warns about — it just never covered non-:unexpected_token shapes.

Fix

Special-case the position-independent error shapes in committed?/2 so they are always committed, rather than position-gating them through error_position/1:

defp committed?({:unexpected_end, _message, _pos}, _tokens), do: true
defp committed?({:bare_expression, _pos, _expr_struct}, _tokens), do: true
defp committed?({:invalid_assign_target, _pos, _expr_struct}, _tokens), do: true
defp committed?({:unclosed_delimiter, _delimiter, _open_pos}, _tokens), do: true

These shapes can never represent a clean list terminator:

  • :bare_expression / :invalid_assign_target are statement-level errors only reachable through a function() … end body — i.e. only after a full prefix expression has been consumed.
  • :unclosed_delimiter has by construction consumed an opener and its content. Gating it on position would make blame depend on whether the opener happens to be the argument's first token versus opened mid-element; always committing it blames the inner opener uniformly.

Committing them unconditionally also sidesteps the missing-position problem: an invalid-assign target on a meta: nil postfix node (e.g. an Expr.Call LHS) has no target_position/1. To keep convert_error/2 pointing at the right line, validate_assign_targets/2 now takes the = operator position and uses it as a fallback:

other -> {:halt, {:error, {:invalid_assign_target, target_position(other) || assign_pos, other.__struct__}}}

committed?/2 now classifies these as committed and the genuine deep error propagates. The example above now reports:

at line 3, column 7:
A bare table-access expression isn't a Lua statement.

This is the inverse companion to #365 ("blame the opener"): don't let a shallow opener-blame swallow a genuine deep error.

Tests

  • New regression rows in the @deep_propagation_errors table in test/lua/parser/error_position_test.exs, covering each newly-committed shape (bare expression, invalid assign target with a position-less Expr.Call node, and unclosed delimiters both mid-element and beginning a non-first argument).
  • Full suite green: mix test → 2475 passed, 7 skipped.

When an argument sub-parse failed with a non-:unexpected_token error
(e.g. a bare-expression statement deep inside a `function` argument),
parse_expr_list_acc/2 treated it as a clean list terminator because
error_position/1 only understood :unexpected_token. The list "recovered"
and expect_closing emitted a misleading "Expected rparen, got function"
pinned at the enclosing call's opener line instead of the real error.

Teach error_position/1 to read the position out of :bare_expression,
:invalid_assign_target, and :unclosed_delimiter errors so committed?/2
classifies them correctly and the genuine deep error propagates.
…mitted

Stabilization follow-up from adversarial review of the deep-error-position
fix. Three refinements:

- :bare_expression and :invalid_assign_target are now classified as
  always-committed in committed?/2 rather than position-based. They are
  statement-level errors only reachable through a function-literal body, so
  they always represent consumed input. This closes a nil-position
  fall-through: an Expr.Call/Expr.Paren assignment target (built with
  meta: nil) yielded a nil position, so the old position-based check still
  swallowed the deep error and blamed the enclosing call opener.

- validate_assign_targets/2 falls back to the `=` operator position when the
  offending target node carries no position, so the propagated error points
  at the right line instead of degrading to nil.

- error_position/1 keeps the :unclosed_delimiter clause (position-based via
  the opener) so a delimiter that begins an argument still recovers at the
  outer boundary.

Adds regression rows locking the invalid-assign and non-first-argument
unclosed-delimiter paths, which were previously untested.
@davydog187

Copy link
Copy Markdown
Contributor Author

Stabilization cycle (ultracode review)

Ran a multi-agent review (4 dimensions: regression-risk, edge-cases, completeness, test-coverage) with adversarial verification (each finding refuted by 2 independent skeptics) plus a validation gate. 11 findings, 7 confirmed. Results actioned in 5e73e77:

Blocker — formatting. The new test tuple wasn't mix format-clean. Fixed.

High — nil-position fall-through (real bug, same class as the original). The original :invalid_assign_target clause only helped when target_position/1 returned a real position — but it returns nil for an Expr.Call/Expr.Paren LHS (those nodes are built with meta: nil). So this still reproduced the bug:

outer("ok", function()
  f() = 1        -- blamed the `function` keyword on line 1, not here
end)

Fix: classify :bare_expression and :invalid_assign_target as always-committed in committed?/2 (they're statement-level errors only reachable through a function-literal body, so they always represent consumed input — like :unexpected_end), independent of position. Also threaded the = operator position into validate_assign_targets/2 as a fallback so the propagated error gets a real line instead of nil. Now reports syntax error near '=' at line 2.

Medium — :unclosed_delimiter clause was untested. Deleting it kept all tests green. Added a row for a non-first-argument unclosed delimiter (outer(1, inner(), plus a row for the invalid-assign case above.

Not actioned (intentional):

  • {/(-prefix unclosed argument inconsistency (low): an unclosed delimiter that begins an argument is still blamed at the outer boundary because its opener position coincides with the sub-parse start token. Relaxing > to >= in committed?/2 would change this, but it's a broader behavior change with its own trade-offs — left as a possible follow-up, not folded into this PR.

Validation after changes: mix test → 2477 passed, 7 skipped; mix format --check-formatted clean; credo not configured in this repo.

…t argument

An unclosed (/{/[ that begins a non-first call argument was reported as a
misleading "Expected )" at the outer call boundary, while the same
delimiter as the first argument or opened mid-element was correctly blamed
at its opener. The begins-a-non-first-argument case fell through because its
open_pos equals the sub-parse start offset, so committed?/2's strict
position test (error_offset > start_offset) read it as "not committed" and
the list recovered.

Make :unclosed_delimiter unconditionally committed, joining :unexpected_end,
:bare_expression, and :invalid_assign_target. An opened-but-unclosed
delimiter has by construction consumed an opener and content, so it can
never be a clean list terminator and must always propagate as the genuine
deep error. The position-gated path is left for :unexpected_token, which
still relies on it for clean trailing-comma/stray-closer recovery.
@davydog187

Copy link
Copy Markdown
Contributor Author

Folded the deferred follow-up into this PR (65a7c04).

{/(-prefix unclosed argument inconsistency — now addressed. Rather than the broad > → >= relaxation in committed?/2 (which would also make :unexpected_token errors at the start token "committed" and break clean trailing-comma / stray-closer recovery, e.g. f(1, ) and f(1, ]), :unclosed_delimiter is now made unconditionally committed — joining :unexpected_end, :bare_expression, and :invalid_assign_target. An opened-but-unclosed delimiter has by construction consumed an opener and content, so it can never be a clean list terminator and always propagates as the genuine deep error. :unexpected_token keeps its position-gated recovery.

Result — an unclosed delimiter beginning a non-first argument is now blamed at its own opener, consistent with the first-arg and mid-element cases:

source before after
outer(1, {a = 1 Expected rparen, got lbrace Unclosed opening brace '{' @ col 10
outer(1, (2 + 3 Expected rparen, got lparen Unclosed opening parenthesis @ col 10

Two regression rows added to @deep_propagation_errors. Validation: mix test → 2479 passed, 7 skipped; mix format --check-formatted clean.

@davydog187
davydog187 merged commit 41db5d2 into main Jun 15, 2026
5 checks passed
@davydog187
davydog187 deleted the fix/parser-deep-error-position branch June 15, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant