Fix slow hosted Poetry lock rewrite (#1473)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Refs #1473. This is the rewrite half of the issue. The VEX half is left for a follow-up, see below.

Summary

A hosted scan of a Poetry or PDM project rewrites every patched
package in poetry.lock / pdm.lock in one batch
(utils::lock_fragments::rewrite_batch). This PR cuts the work that
batch and the Poetry redirect repeat once per package:

• Fragment lookups: the batch searched the whole lock text 2–3 times
  per changed fragment: text.matches(old).count() and text.find(old)
  on the original, then result.matches(new).count() on the result.
  It now finds every fragment in one pass over the original and one
  over the result (sole_occurrences). That pass is an Aho-Corasick search
  for each fragment's first 64 bytes, with the full fragment compared at
  each hit. A fragment that isn't there exactly once still ends the batch.
• Stale-install check: patch::redirect::poetry::pre_1_4_writer parsed
  the whole rewritten lock just to read [metadata] lock-version. The
  batch now hands back the version it read from its own parse
  (rewrite_poetry_lock_all_with_version). Only the step-by-step fallback
  still parses for it.

Lock output is unchanged. Using overlapping counts in sole_occurrences
only rejects more: a self-overlapping fragment sends the batch to the
step-by-step path, which is the reference result. The batch is never
accepted where that path would refuse.

Evidence

Callgrind, poetry/hosted bench scenario (400 packages, 12 patched), same fixture:

	total Ir	rewrite_batch	extra lock parse in rewrite_poetry
main 15c2c83	505.9M	205.4M	~50M
this PR	402.8M (−20%)	152.2M	0

The rewritten poetry.lock and the --json report are byte-identical to
main's (ports and paths normalized). On the 4-vCPU sandbox, wall-clock
socket-patch-bench compare stayed inside its noise band (poetry/rescan
−7.2% [−11.8, −2.8], poetry/hosted −1.8% [−5.6, +4.1]). The first version of the
change built the automaton over whole fragments, which cost more than it
saved. The second commit fixes that.

Regression tests (red on main, green here):

• patch::redirect::poetry::equivalence_tests::many_patches_render_the_lock_once
  now also asserts 2 whole-lock parses and 2 whole-lock scans for 12
  patched packages. On main it fails with PARSES left: 3, right: 2.
• patch::redirect::pdm::parse_reuse_equivalence_tests::many_patches_render_the_lock_once
  asserts 2 scans. On main it fails with SCANS left: 144, right: 2.
• utils::lock_fragments::tests::sole_occurrences_finds_each_needle_once_or_refuses
  covers shared 64+ byte prefixes, missing, repeated, empty and
  self-overlapping needles.
• utils::poetry_lock batch-vs-steps equivalence now also checks the version
  the batch reads equals the output lock's lock_version.

Local checks (head bbb3185):

• cargo fmt --all -- --check: OK
• cargo clippy --workspace --all-features -- -D warnings: OK
• cargo test --workspace --all-features --no-fail-fast: 219 test binaries pass. 13 tests fail, and none of them is in the Poetry/PDM rewrite path:
  • 12 write-failure or unremovable-file tests that make a path read-only.
    This sandbox runs as uid 0, which ignores those permissions
    (covgap_commands_vendor ×3, in_process_redirect ×3, repair ×2,
    core copy_tree/vlt_heal/pypi_poetry/pypi_requirements wire-failure tests).
  • e2e_bun_lockb::binary_shared_bundled_record_hosted_pin_is_managed
    needs patches-api.socket.dev, which the sandbox proxy blocks.
  • CI runs these as non-root with network.
• e2e_vex_build -- poetry:: --ignored with real Poetry 2.4.3: 4/4 pass.
• e2e_vex_build -- pdm:: --ignored with real PDM 2.29.2 (SOCKET_PATCH_PDM_E2E_REQUIRED=1): 2/2 pass.
• No wrapper changes (npm/, pypi/, gem/ don't touch this code).

/code-review high: no correctness findings. Fixed in bbb3185: the
counter docs now say exactly which sites they instrument, renders are
counted through the same tick helper, the "[metadata] untouched" doc is
narrowed to lock-version, and the shadowing local is renamed. Not changed:

• The lock version travels out through the plan closure rather than a
  LockBatch field. LockBatch is shared with PDM, which has no such
  field.
• rewrite_poetry_lock_all is kept as the version-less entry point the
  equivalence tests and docs use.
• The 64-byte prefix only affects speed, never output. Poetry/PDM units
  start with their name key within that window.
• AhoCorasick::new failing only sends the batch step by step.
• The empty "Start fix" commit is the claim marker this agent workflow
  uses. Squash-merge drops it.

Not in this PR (follow-up for #1473)

vex::discover::pypi_locks::extract is now ~35% of the scan's instructions.
Both VEX discovery passes (before and after the rewrite) parse poetry.lock
into a DocumentMut again. Sharing the rewrite's parse with VEX, or
reading it as a spanned Document, means changing the uv/pylock/PDM VEX
helpers that take &DocumentMut. That is a separate change, so #1473
stays open for it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UELYzooW2PT3YpQweak8tS

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bbb3185. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Labeled Ready for review by the burn-down agent.

• Head: bbb31854a93ebd2977055b793be597580621d98e (mergeable against main, no conflicts; no CHANGELOG.md change)
• CI: 36/36 green (50 skipped) on the head, no main-wide failures
• Bugbot: reviewed bbb3185, no new issues; no unresolved review threads

Slack announcement not sent this run (Slack send tool unavailable); the next run will retry.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Final review brief

What it does: The batched Poetry/PDM lock rewrite used to search the whole lock text 2-3 times for every changed fragment. It now finds all fragments in one pass over the original and one over the result, using an Aho-Corasick automaton over each fragment's first 64 bytes and comparing the full fragment at each hit. The Poetry redirect also stops re-parsing the rewritten lock just to read lock-version. This is a perf fix for #1473 (−20% instructions on the poetry/hosted bench), and the lock output doesn't change.

Risk: low. The new search can only refuse more often than the old one (it counts overlapping matches). A refused batch falls back to the step-by-step path, which is the reference result, and an accepted batch must still match a full re-render byte for byte. The new counters are #[cfg(test)] only.

Look here

• lock_fragments.rs:297: sole_occurrences, the core of the change
• lock_fragments.rs:430: start positions are zipped with the flattened edits, so the order has to match olds
• poetry_lock.rs:579: rewrite_poetry_lock_all_with_version, the version captured inside the plan
• poetry.rs:27: pre_1_4_writer uses the batch's version, or parses the lock itself when it doesn't have one

Verified

• Read the full diff.
• Lib tests pass: lock_fragments 3/3, pdm 64/64 and poetry 82/83. The one failure is a chmod-based write-failure test that can't pass as root in the sandbox and is unrelated.
• A scratch test covered a multi-byte char straddling byte 64 (no panic, since it slices bytes), needles that are prefixes of other needles or share a prefix, empty and duplicate needles, CRLF, and fragments that overlap each other. All behaved as described.
• aho-corasick was already a core dependency, and Cargo.toml is unchanged.
• CHANGELOG.md is untouched and there's no debug code.
• CI is green on this head (ci-ok, clippy). Bugbot reviewed this head and left no open threads.

Changes I made: none.

Open questions: none. #1473 stays open for the VEX-parse follow-up the description lists.

Auto-merge is armed, so approving sends this straight to the merge queue.

---

Generated by Claude Code