Fix uv vendored revert half-reverting after a relock (#806, #821)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #806
Fixes #821

Summary

Before this change, a vendored uv project broke after an ordinary relock. Running uv add --dev zipp with six in a dev group (#821), or uv add idna beside a user-authored override-dependencies (#806), and then vendor --revert, remove, rollback or a vendored→hosted takeover:

• reverted pyproject.toml but left uv.lock wired;
• made uv sync --locked fail with "The lockfile at uv.lock needs to be updated";
• still let vendor --revert and the takeover exit 0 with status: "success", and the takeover warned "left in place" after it had already written pyproject.toml.

Now:

• Revert recognizes our element inside uv's re-serialized array and unwinds both files cleanly.
• When something really has drifted, or can't be read, revert writes neither file, so the pair stays consistent.

Root cause (shared by both issues)

revert_uv (crates/socket-patch-core/src/vendor/pypi_uv.rs) had two defects:

1. Whole-array text matching. The uv_lock_requires_dev record (Vendored uv package in a dependency group: after uv add --dev / uv remove --dev, vendor --revert, remove, rollback and the hosted takeover revert pyproject.toml but keep uv.lock's vendored requires-dev entry, so uv sync --locked fails (exit 0, "success") #821) and the Rewritten uv_lock_manifest_overrides record (Vendored uv with a user-authored override-dependencies: after any relock (uv add, uv lock --upgrade-package), vendor --revert / remove revert pyproject.toml but keep the vendored [manifest] overrides entry in uv.lock, so uv sync --locked fails (vendor --revert exits 0) #806) capture the entire <key> = [ … ] array, sibling elements included. Any relock that touches a sibling makes uv re-serialize the array:

   • uv add --dev x rewrites the group line;
   • uv add y sorts [manifest] overrides and splits it one element per line.

   Our element stays byte-identical, but the recorded text no longer matches, so revert called it drift. The convergence shortcut also searched the whole lock for the recorded array, so a byte-identical array under another key could fake convergence. That was found in review: see the resolved thread.

2. No pair gate. A drift-kept uv.lock record only added a warning. The pyproject.toml records were still reverted and both files written. That contradicts docs/testing/uv-compatibility.md ("conflicting changes preserve both files … rather than restoring only one side").

Change

• Key-anchored array revert (revert_array_elements). It handles uv_lock_requires_dev, uv_lock_manifest_constraints and the Rewritten uv_lock_manifest_overrides, none of which is text-matched across the whole lock any more. It finds the array under the record's own key, in the root unit's [package.metadata.requires-dev] or in [manifest]:
  • Untouched array: restored verbatim.
  • Re-serialized array: the recorded old and new arrays are compared to find our edits (an element rewritten in place, or one appended). Just that element is restored or removed, and the array is re-rendered the way uv writes it: one element inline, two or more one per line, in the lock's own line ending.
  • Our element edited and still routing through the artifact: drift.
  • Our element gone with nothing in the array routing through the artifact, or the key/section provably absent: silent convergence, per the LIVENESS CONTRACT.
  • Key present but unreadable (re-spelled, unbalanced, root unit missing): drift, fail-closed. This was found by Bugbot: see the resolved thread.
• Pair gate: if any record is drift-kept, revert_uv writes neither file. The vendor_lock_entry_drifted warning still keeps the artifact and ledger entry, so a re-run finishes once the drift is undone. With this, the hosted takeover's redirect_vendored_revert_failed warning ("left in place") is accurate: revert_keeps_wiring in scan/hosted.rs keys on the same drift signal.
• remove, rollback and the takeover all go through revert_pypi → revert_uv, so they get the same fix.
• Docs: docs/testing/uv-compatibility.md now describes the relock behaviour.

Behaviour change to existing tests

Six existing unit tests asserted the half-revert that these issues report. #821 points at one of them, revert_warns_and_skips_on_drifted_lock_fragment ("The pyproject side (undrifted) was still reverted"). They now assert the pair gate: neither file is written on drift. One test, renamed to revert_removes_our_element_from_a_reshaped_manifest_overrides_array, covered a user reshaping the overrides array around our unchanged element. That is no longer drift: our element is removed and the user's reshaping is kept. No test was skipped or weakened. Every drift case still warns vendor_lock_entry_drifted and keeps the artifact.

Test evidence (red → green)

Test	Covers	Without fix	With fix
pypi_uv::tests::revert_survives_uv_reserializing_the_dev_group_line	#821 (uv add --dev)	fail (drift warning, half-revert)	pass
pypi_uv::tests::revert_survives_a_sibling_leaving_the_dev_group	#821 (uv remove --dev)	fail	pass
pypi_uv::tests::revert_survives_uv_reserializing_manifest_overrides	#806	fail	pass
pypi_uv::tests::revert_removes_our_override_from_a_sorted_multi_line_array	#806 (two user overrides)	fail	pass
pypi_uv::tests::revert_converges_on_the_overrides_key_not_a_lookalike_array	#806 (review: lookalike array fakes convergence)	fail (element stranded, artifact deleted)	pass
pypi_uv::tests::revert_treats_an_unreadable_array_key_as_drift	Bugbot: a locator miss is not convergence	fail	pass
pypi_uv::tests::revert_writes_neither_file_when_a_lock_record_is_drift_kept	pair gate (#806, #821)	fail (pyproject reverted alone)	pass
e2e_vendor_pypi_build::uv_vendor_revert_after_dev_group_relock (real uv: vendor → uv add --dev zipp → vendor --revert → uv lock --check)	#821	fail (uv.lock fragment … changed since vendoring; left untouched)	pass
e2e_vendor_pypi_build::uv_vendor_revert_after_manifest_overrides_relock (real uv: user override-dependencies + transitive six → uv add idna==3.7 → vendor --revert → uv lock --check)	#806	fail (same)	pass

The e2e results were run locally against uv 0.8.17 with SOCKET_PATCH_UV_E2E_REQUIRED=1, so skips count as failures. The red runs used main's pypi_uv.rs with the new tests. CI runs this suite on its uv legs (0.2.37 through 0.12.17, Linux and macOS); releases without uv lock --check report n/a, as the existing capstones do.

Local gates (head 77686c4)

• cargo clippy --workspace --all-features -- -D warnings: clean.
• rustfmt --check on both touched Rust files: clean (both were clean on main).
• cargo test -p socket-patch-core --lib vendor::pypi_uv: 98/98 pass.
• cargo test -p socket-patch-cli --test e2e_vendor_pypi_build -- --include-ignored: 25/25 pass.
• cargo test --workspace --all-features --lib --bins (on c35bf96): every test passes except 4 write-permission tests in files this PR doesn't touch (copy_tree, vlt_heal, pypi_poetry, pypi_requirements). They expect a read-only path to refuse a write, which can't happen as root, and this sandbox runs as root. The integration suites are left to CI; a full local workspace build of every test binary didn't fit in the sandbox disk.
• The npm, PyPI and gem wrappers only dispatch to the binary, so no change was needed there.

Per-issue checklist

• Vendored uv package in a dependency group: after uv add --dev / uv remove --dev, vendor --revert, remove, rollback and the hosted takeover revert pyproject.toml but keep uv.lock's vendored requires-dev entry, so uv sync --locked fails (exit 0, "success") #821 vendor --revert after uv add --dev / uv remove --dev → the two dev-group unit tests and the real-uv e2e
• Vendored uv package in a dependency group: after uv add --dev / uv remove --dev, vendor --revert, remove, rollback and the hosted takeover revert pyproject.toml but keep uv.lock's vendored requires-dev entry, so uv sync --locked fails (exit 0, "success") #821 remove / rollback / hosted takeover → the same revert_uv path; the pair gate makes the takeover's "left in place" true
• Vendored uv with a user-authored override-dependencies: after any relock (uv add, uv lock --upgrade-package), vendor --revert / remove revert pyproject.toml but keep the vendored [manifest] overrides entry in uv.lock, so uv sync --locked fails (vendor --revert exits 0) #806 vendor --revert / remove after a relock re-serializes [manifest] overrides → the overrides unit tests (including the lookalike-array case) and the real-uv e2e
• Both: genuine drift or an unreadable array keeps both files wired → revert_writes_neither_file_when_a_lock_record_is_drift_kept, revert_treats_an_unreadable_array_key_as_drift, plus the five updated drift tests

Follow-ups (out of scope)

• When wiring, add_manifest_override still appends our element in socket-patch's own spelling rather than uv's sorted multi-line form. With key-anchored revert this no longer causes drift, so the wiring format is left as is.

🤖 Generated with Claude Code

https://claude.ai/code/session_016MCKtjJk6MF8zQWSFYWg7N

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 77686c4. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[agent] One check is red on head 77686c4: PDM patch compatibility → native (ubuntu-latest, 2.22.4). One cell failed, 2.22.4 optional hosted (rescanIdempotent, 40.7s against the usual ~20s). The other 46 cells passed.

I don't think this PR caused it:

• This PR only changes the uv backend (vendor/pypi_uv.rs), a uv e2e test and the uv docs. PDM hosted rewriting and PDM vendored wiring live in other modules. In scripts/backtest-pdm.py, uv is only used to bootstrap the PDM release under test.
• The failing cell moves between runs. On f9b7cfa, a different leg failed in a different cell: native (macos-latest, 2.8.2), direct vendored (rescanAfterRelockApplies, rescanReusesWheel). That leg passes on 77686c4, and every cell passed on f4e9b7b.
• Recent PDM runs on other PRs (Fix vendored requirements.txt re-vendoring removed pins (#786) #787, Fix gem crawler missing Bundler 4 standalone bundle (#796) #797, Fix npm VEX attesting a patch the twin lock lacks (#798) #799) needed a second attempt to go green.

No fix exists to port. Once this run finishes I'll re-run the failed job once. If it fails again, I'll treat it as real and dig into the case artifacts.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Labeled Ready for review.

• Head: 77686c45
• CI: 482/482 check runs completed on head, 0 failing
• Bugbot: reviewed 77686c45, no new issues; both earlier review threads resolved
• Reviewer focus: the uv pyproject/uv.lock pair gate and the fail-closed path when a lock array can't be read (crates/socket-patch-core/src/vendor/pypi_uv.rs).

Slack announcement not sent this run (no Slack send tool available), so the next run will retry.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Stacked follow-up: #841 fixes #840. That's a different cause in the same revert_uv function: the revert restores a stale specifier after the user edits the declaration. It builds on this branch.

---

Generated by Claude Code