Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .agents/plans/A46-pattern-position-capture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
id: A46
title: "Position capture () in string pattern matching"
issue: 257
pr: 288
branch: feat/pattern-position-capture
base: main
status: review
direction: A
unlocks:
- pm.lua
---

## Goal

Land position capture `()` in the pattern engine with regression coverage
across `string.find`/`match`/`gmatch`/`gsub`, and advance `pm.lua`'s skip
from `:all` to the next genuine failure point.

## Out of scope

Issue #257 is a multi-feature pattern-engine epic. This PR ships only
position capture. The following stay deferred to their own follow-up plans
(noted in ## Discoveries with the precise next failure points):

- `%f[set]` frontier pattern.
- `%b xy` balanced match (e.g. `%b()`).
- `%1`..`%9` backreferences in pattern *bodies* and the matching
`string.gsub` "invalid capture index" / "invalid replacement value"
error validation (pm.lua:233-238).
- The C-stack / "pattern too complex" depth-limit and big-string tests
(pm.lua:240-262).
- `string.gsub` table/string replacement fall-back-to-whole-match
semantics when the first capture is a position capture (a pre-existing
table-replacement bug, surfaced via `().`; see ## Discoveries).

## Success criteria

- [ ] `()` captures the current 1-based byte position as an integer in
`find`, `match`, `gmatch`, and `gsub`.
- [ ] New regression tests in `test/lua/vm/stdlib/pattern_test.exs` pass.
- [ ] `pm.lua` skip narrows from `lines: :all` to a precise range with an
issue reference, and the lua53 suite is green.
- [ ] `mix test` passes.
- [ ] `mix test test/lua53_suite_test.exs --only lua53` passes.

## Implementation notes

- `lib/lua/vm/stdlib/pattern.ex` already compiles `()` to
`:position_capture` and emits `{:done, pos + 1}`; the mechanic is in
place. This plan verifies it end-to-end across all four entry points
and pins it with regression tests, then advances the skip.
- The first genuine post-position-capture failure in `pm.lua` is line 233
(`string.gsub` "invalid replacement value"/"invalid capture index"
error checks) — out of scope. The remaining deferred features are
interleaved through lines 233-374, so the smallest contiguous skip that
lands green is `233..374`.

## Verification

- `mix format`
- `mix compile --warnings-as-errors`
- `mix test`
- `mix test test/lua53_suite_test.exs --only lua53`

## Risks

- Position captures interact with other capture kinds (nested, mixed with
substring captures). Tests cover mixed `()(%w+)()` ordering to guard the
opening-order capture model.

## Discoveries

- `string.gsub("alo alo", "().", {'x','yy','zzz'})` returns `"xyyzzz4567"`
instead of the PUC-Lua `"xyyzzz alo"`. The table-replacement path in
`lib/lua/vm/stdlib/string.ex` keys the table by the first capture and
falls back to that same key when the lookup misses; Lua semantics
require the fall-back to be the **whole match** substring. Surfaced via
position captures because the first capture there is an integer. This is
a pre-existing table/string-replacement bug, not a position-capture
defect — deferred to a follow-up plan.
- pm.lua next failure points after position capture: line 233 (gsub error
validation), line 250 ("pattern too complex" depth limit), then `%b`,
`%f`, and backreference assertions through line 374.
74 changes: 74 additions & 0 deletions test/lua/vm/stdlib/pattern_test.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
defmodule Lua.VM.Stdlib.PatternTest do
use ExUnit.Case, async: true

# Pins Lua 5.3 §6.4.1 position-capture semantics: an empty
# parenthesised group `()` captures the current 1-based position in the
# subject as an integer, not a substring.

alias Lua.VM.Stdlib.Pattern

describe "position capture in Pattern.match/find/gmatch" do
test "() at the start captures position 1" do
assert {:match, [1]} = Pattern.match("hello", "()")
end

test "() captures the 1-based byte position as an integer" do
assert {:match, [3, 5]} = Pattern.match("hello", "()ll()")
end

test "find returns the captured position alongside the match span" do
assert {3, 3, [4]} = Pattern.find("hello", "l()")
end

test "position capture mixes with a substring capture in opening order" do
assert {:match, [1, "alo", 4]} = Pattern.match("alo jose", "()(%w+)()")
end

test "gmatch yields each match's position captures" do
assert [[1, "a", 2], [3, "alo", 6], [7, "jose", 11], [13, "joao", 17]] =
Pattern.gmatch("a alo jose joao", "()(%w+)()")
end

test "gmatch with a bare () walks every position including end+1" do
positions = "abcde" |> Pattern.gmatch("()") |> List.flatten()
assert positions == [1, 2, 3, 4, 5, 6]
end
end

describe "position capture through the Lua string library" do
test "string.match returns integer positions" do
assert {[3, 5], _} = Lua.eval!(~S|return string.match("hello", "()ll()")|)
end

test "string.find appends the captured position" do
assert {[3, 3, 4], _} = Lua.eval!(~S|return string.find("hello", "l()")|)
end

test "string.gsub passes positions to the replacement function" do
script = ~S"""
local t = {}
local s = 'a alo jose joao'
local r = string.gsub(s, '()(%w+)()', function (a, w, b)
assert(string.len(w) == b - a)
t[a] = b - a
end)
return r, t[1], t[3], t[7], t[13]
"""

assert {["a alo jose joao", 1, 3, 4, 4], _} = Lua.eval!(script)
end

test "string.gmatch iterates positions in a for loop" do
script = ~S"""
local a = 0
for i in string.gmatch('abcde', '()') do
assert(i == a + 1)
a = i
end
return a
"""

assert {[6], _} = Lua.eval!(script)
end
end
end
8 changes: 7 additions & 1 deletion test/lua53_skips.exs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,13 @@
}
],
"pm.lua" => [
%{lines: :all, category: :unimplemented, reason: "pending initial triage (pattern engine work in A9)", issue: nil}
%{
lines: 233..374,
category: :unimplemented,
reason:
"gsub capture-index/replacement error validation, 'pattern too complex' depth limit, %b balanced match, %f frontier, and %1-%9 backreferences not yet implemented; position capture passes through line 232",
issue: 257
}
],
"sort.lua" => [
%{
Expand Down
Loading