Skip to content

Fix Poetry/PDM lock splice drift (#694, #695) - #703

Open
Mikola Lysenko (mikolalysenko) wants to merge 3 commits into
mainfrom
agent/fix-python-lock-fragment-engine
Open

Mikola Lysenko (mikolalysenko) wants to merge 3 commits into
mainfrom
agent/fix-python-lock-fragment-engine

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #694
Fixes #695

Summary

poetry.lock and pdm.lock rewrites (hosted and vendored) now share one fragment-splice engine. On a lock that mixes CRLF and LF lines, the patched package used to come back in opposite styles depending on the tool: Poetry made it all CRLF and PDM made it all LF. Now both give the replaced package the line ending most of its own lines already had, and every line outside it keeps its own. LF-only and CRLF-only locks produce the same bytes as before.

Root cause

utils/poetry_lock.rs and utils/pdm_lock.rs each carried a full copy of one engine: the rewrite struct and edits(), the parse holder, next_header_end, extend_span, fragment pairing, and the render → reparse → pair → splice finish step. The copies had drifted on the line-ending rule. Poetry converted the whole rendering to CRLF if the input had any CRLF. PDM used preserve_line_endings, which converts only CRLF-only input. Both splice only the package's fragments back, so the rule decided the endings of every line in the patched unit (#695).

Change

  • New utils/lock_fragments.rs, holding one copy of each piece:
    • LockParse (with parsed, take, restore)
    • FragmentRewrite and its edits()
    • next_header_end
    • extend_span (the Poetry variant, a superset)
    • pair_fragments (with PDM's shape check, now also guarding Poetry)
    • finish
  • finish applies one rule per fragment: each replacement fragment takes the line ending that dominates the original fragment it replaces. If that fragment has no line break, the file's majority is used. A fragment anchored at the preceding line break (Poetry's legacy [metadata.files] entry) keeps that leading \n as is, because its \r lies outside the fragment.
  • Poetry and PDM keep only their planner, the document mutation and *_lock_fragments_in. PoetryLockParse/PdmLockParse and PoetryLockRewrite/PdmLockRewrite are now type aliases, so callers (redirect/{poetry,pdm}.rs, vendor/pypi_{poetry,pdm}.rs, vex discovery) are unchanged.
  • Net: −261 / +200 lines in the two format modules, plus the new 330-line module (about 100 of those lines are tests).

Tests

Issue Regression test Without fix With fix
#695 Poetry utils::poetry_lock::tests::mixed_line_ending_unit_keeps_its_majority_ending: lock generations 0.12.17 → 2.4.3, CRLF file with an LF name line and the reverse, hosted url and vendored file, forward rewrite plus byte-exact replay of the recorded edits fails (13 wrong-ending lines on 0.12.17) passes
#695 PDM utils::pdm_lock::tests::mixed_line_ending_unit_keeps_its_majority_ending: PDM 0.12.3 → 2.29.2, the same variants, path and url fails (7 wrong-ending lines on 0.12.3) passes
#694 utils::{poetry,pdm}_lock::tests::pairing_refuses_a_fragment_shape_change and utils::lock_fragments::tests::pair_fragments_refuses_a_shape_change (one copy, shape check for both formats); finish_spells_each_fragment_like_the_one_it_replaces n/a (new shared API) passes

The tests that pin byte-identical output pass unchanged: native_formats_rewrite_and_reverse_byte_exactly, rewrite_edits_equal_pdm_lock_edits, rewrite_edits_are_rederived_when_the_splice_differs_from_the_document, the Poetry fixture tests, the redirect::poetry / redirect::pdm golden equivalence suites (rewrite_matches_golden_on_every_lock_generation), and both parse-reuse golden sweeps. No golden file changed.

Commands run locally:

  • cargo test -p socket-patch-core --lib -- utils::pdm_lock utils::poetry_lock utils::lock_fragments utils::python_lock redirect::pdm redirect::poetry: 53 passed.
  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • rustfmt --check on the changed files: clean. A repo-wide cargo fmt --all -- --check is already dirty on main (CI does not run it), so I left the other files alone.
  • cargo test --workspace --all-features --no-fail-fast: 214 test binaries ok. 12 tests fail only because this sandbox runs as root (uid 0). Each one forces a write failure by making a directory or file read-only (set_mode(0o555) and similar), which root ignores. None of them is in code this PR touches beyond its callers, and CI runs them as non-root.
  • e2e_vex_build poetry:: with real Poetry 2.1.1: the vendored test passed. The hosted test's rollback could not reach pypi.org from inside the test harness (sandbox network), so the CI e2e matrix covers it.

CI: 323/323 checks green on b4d8e12, including the Poetry/PDM e2e matrix. Bugbot found no issues on b4d8e12.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LUZxuUEoBuooYdJGxN7p1B


Note

Medium Risk
Changes how Poetry/PDM lockfiles are rewritten and how rollback fragments are recorded; behavior is heavily tested but mistakes could break installs or byte-exact rollback.

Overview
Extracts a shared fragment-splice engine (lock_fragments) used by both poetry.lock and pdm.lock rewriters, replacing duplicated parse caching, fragment pairing, span helpers, and the render→splice finish path. Public types stay as PoetryLockRewrite / PdmLockRewrite and *LockParse aliases over FragmentRewrite / LockParse.

Line endings (#695): finish now respells each replacement fragment to match the dominant line ending of the original fragment it replaces (file majority when the fragment has no breaks), instead of Poetry forcing CRLF when any CRLF exists and PDM using a coarser file-level rule. Unpatched bytes and rollback edits stay byte-exact.

Safety (#694): Shared pair_fragments rejects mismatched fragment counts for both formats (PDM’s shape check now applies to Poetry too).

Adds regression tests for mixed CRLF/LF locks, shape-change refusal, and unit-level ending preservation across lock generations.

Reviewed by Cursor Bugbot for commit b4d8e12. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
poetry.lock and pdm.lock rewrites each carried their own copy of the
engine that splices a rewritten package back into the original lock,
and the copies disagreed on line endings. On a lock that mixes CRLF
and LF lines, Poetry turned the patched package's lines all CRLF while
PDM turned them all LF, so the same input churned in opposite ways.

Both rewriters now use one engine in utils/lock_fragments.rs. Each
replaced fragment takes the line ending most of the original
fragment's lines had, so lines outside the patched package never
change. LF-only and CRLF-only locks come out byte-identical to before.

Fixes #694, #695.

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 3, 2026 16:12
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b4d8e12. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 3, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent).

  • Head: b4d8e129f8ee1b966a44f1dfd0577184f5f3a9a9, 0 commits behind main
  • CI: 319/319 check runs passed (4 skipped, 0 failed or pending)
  • Bugbot: reviewed b4d8e12 and found no issues. There are no open review threads.
  • Reviewers should look at: the per-fragment line-ending rule in finish (utils/lock_fragments.rs), and the now-shared shape check in pair_fragments.

Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

2 participants