diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index a1a7b20f1..33f2ff3b7 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -1540,10 +1540,10 @@ command's integration tests check its output. | `vendor_lock_descriptor_unlocked` | revert `warnings[]` | vendor `--revert` / rollback / remove (yarn berry, #759): a descriptor of the vendored package was added while it was vendored, and the restored registry version does not satisfy it (or its range is one socket-patch cannot read, such as a dist-tag), so it has no `yarn.lock` entry after the revert. The revert still succeeds and the artifact is removed. Run `yarn install` and commit `yarn.lock`. | | `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run ` lock`. | | `pypi_poetry_integrity_unverified` | `skipped` (warning) | vendor (pypi / poetry): the lock was written by Poetry < 1.4 (0.12 `[metadata.hashes]`, lock 1.0/1.1, or a 2.0 lock without a `@generated by Poetry X.Y.Z` header — 1.3 wrote those). That installer does not verify local wheel hashes (the committed wheel bytes are the protection) and does not replace an already-installed package at the same version; recreate the virtualenv or `pip uninstall` the package before `poetry install`, or upgrade Poetry. | -| `pypi_poetry_source_constraint` | `failed` / `skipped` (warning) | vendor (pypi / poetry): the pyproject pins the package to a named source (`source = "pypi"`), which Poetry records nothing for in the lock. Over a lock written by Poetry 1.3 – 2.2 (whose installer refuses that constraint on a file-sourced package, so every `poetry install` would fail) the package is refused before any write, `failed`; an already-wired lock is refused the same way instead of reading as in sync. Over any other lock the wiring stands and this is a warning naming the 1.3 – 2.2 range. The detail names the remedy: drop the `source` key or upgrade to Poetry >= 2.3, re-lock and re-run; when the lock is already wired it first says to run `socket-patch vendor --revert` (which reverts EVERY vendored package), since re-locking discards the wiring. | +| `pypi_poetry_source_constraint` | `failed` / `skipped` (warning) | vendor (pypi / poetry): the pyproject pins the package to a named source (`source = "pypi"`), which Poetry records nothing for in the lock. Over a lock written by Poetry 1.3 – 2.2 (whose installer refuses that constraint on a file-sourced package, so every `poetry install` would fail), or one with `lock-version` 1.0 / 1.1 (which every Poetry 1.3+ re-solves on install and then refuses the same way, #1503), the package is refused before any write, `failed`, and the detail says which; an already-wired lock is refused the same way instead of reading as in sync. Over a Poetry 2.3+ (or 0.x) lock the wiring stands and this is a warning naming the 1.3 – 2.2 range. The detail names the remedy: drop the `source` key or upgrade to Poetry >= 2.3, re-lock and re-run; when the lock is already wired it first says to run `socket-patch vendor --revert` (which reverts EVERY vendored package), since re-locking discards the wiring. | | `redirect_poetry_stale_install_risk` | `warnings[]` (warning) | scan `--mode hosted` (poetry): same writer test as above — a warm virtualenv keeps the upstream package after the redirect on Poetry < 1.4 (1.4+ re-installs from the new source); fresh installs pick up the patched wheel. Emitted once per rewritten lock, only on the run that rewrites it. | -| `redirect_poetry_source_constraint` | `warnings[]` (warning) | scan `--mode hosted` (poetry): the pyproject pins the redirected package to a named source (`source = "pypi"`) and the lock was not written by Poetry 1.3 – 2.2 (those writers are refused with `redirect_poetry_lock_unsupported`, also when the lock is already redirected and before a vendored → hosted takeover reverts anything). The rewrite stands, and an already-redirected lock is still confirmed; the detail says a Poetry 1.3 – 2.2 installer would refuse the lock and to drop the `source` key if the project is installed with one. Exit code and `status` unchanged. | -| `redirect_poetry_entry_not_found` / `redirect_poetry_missing_sha256` / `redirect_poetry_lock_unsupported` | `warnings[]` (warning) | scan `--mode hosted` (poetry): the lock has no `[[package]]` at the granted version (uv-parity twin of `redirect_uv_entry_not_found`); the grant carries no SHA-256 (gated once per dep, not per lock); the lock is refused — Poetry 0.12 layout (URL sources ignored), an unsupported `lock-version`, a forked package listed at several versions, a user-authored `[package.source]` on another origin (an earlier Socket URL for the same wheel is superseded in place), a package the pyproject pins to a named source (`source = "pypi"`) over a lock written by Poetry 1.3 – 2.2, whose installer refuses that constraint on a URL-sourced package, a malformed `[metadata.files]`/`[metadata.hashes]`, or a wheel whose filename does not match the locked package. Exit code and `status` unchanged (hosted-refusal posture). A vendored → hosted takeover over a Poetry 0.x lock is retracted with this code (wet and `--dry-run`, see **Staged takeover**): the purl stays vendored and patched and is skipped with this code as `skipped` (hosted event). The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), upgrade to Poetry >= 1.0 and re-lock, then re-run `scan --mode hosted`. | +| `redirect_poetry_source_constraint` | `warnings[]` (warning) | scan `--mode hosted` (poetry): the pyproject pins the redirected package to a named source (`source = "pypi"`) and the lock is in format 2.1+ and was not written by Poetry 1.3 – 2.2 (those writers, and any `lock-version` 1.0 / 1.1 lock, which every Poetry 1.3+ re-solves and refuses (#1503), are refused with `redirect_poetry_lock_unsupported`, also when the lock is already redirected and before a vendored → hosted takeover reverts anything). The rewrite stands, and an already-redirected lock is still confirmed; the detail says a Poetry 1.3 – 2.2 installer would refuse the lock and to drop the `source` key if the project is installed with one. Exit code and `status` unchanged. | +| `redirect_poetry_entry_not_found` / `redirect_poetry_missing_sha256` / `redirect_poetry_lock_unsupported` | `warnings[]` (warning) | scan `--mode hosted` (poetry): the lock has no `[[package]]` at the granted version (uv-parity twin of `redirect_uv_entry_not_found`); the grant carries no SHA-256 (gated once per dep, not per lock); the lock is refused — Poetry 0.12 layout (URL sources ignored), an unsupported `lock-version`, a forked package listed at several versions, a user-authored `[package.source]` on another origin (an earlier Socket URL for the same wheel is superseded in place), a package the pyproject pins to a named source (`source = "pypi"`) over a lock written by Poetry 1.3 – 2.2, whose installer refuses that constraint on a URL-sourced package, or over a `lock-version` 1.0 / 1.1 lock, which every Poetry 1.3+ re-solves and refuses the same way (#1503), a malformed `[metadata.files]`/`[metadata.hashes]`, or a wheel whose filename does not match the locked package. Exit code and `status` unchanged (hosted-refusal posture). A vendored → hosted takeover over a Poetry 0.x lock is retracted with this code (wet and `--dry-run`, see **Staged takeover**): the purl stays vendored and patched and is skipped with this code as `skipped` (hosted event). The detail names the remedy and its reach: run `socket-patch vendor --revert` (it reverts EVERY vendored package in the project, not just this one), upgrade to Poetry >= 1.0 and re-lock, then re-run `scan --mode hosted`. | | `redirect_pdm_refused` / `redirect_pdm_legacy_sync_required` | `warnings[]` (warning) | scan `--mode hosted` (pdm): the `pdm.lock` rewrite was refused — an unsupported `[metadata] lock_version` (the identity-losing `3.1` / `4.0`–`4.2` formats or an untested future format), an unsupported `strategy`, a package listed at several versions (fork) or absent, a user-authored `url`/`path`/VCS/`editable` source, hash-less or malformed `files`, or a wheel whose filename does not match the locked package (`redirect_pdm_refused`); or the lock was written in format `2` (PDM 0.12–1.4), whose upstream freshness bug lets `pdm install` regenerate the lock — use `pdm sync` (`redirect_pdm_legacy_sync_required`). A refused uuid is withheld from every other PyPI rewriter when `pdm.lock` is the install driver, and its patch is not confirmed. Exit code and `status` unchanged (hosted-refusal posture). | | `redirect_bun_lock_unsupported` | `warnings[]` (warning) | scan/get `--mode hosted` (bun): the text lock's `lockfileVersion` is not 0, 1 or 2 (a newer version: update socket-patch, re-locking would reproduce it; no integer: re-lock with Bun ≥ 1.2 — the shared gate's text, identical to vendored's `vendor_lockfile_version_unsupported`), or its `packages` section is not bun's single-line grammar. Nothing rewritten; exit 0 (hosted-refusal posture). | | `redirect_bun_workspace_unsupported` | `warnings[]` (warning) | scan/get `--mode hosted` (bun): a lockfileVersion-0 lock (Bun 1.1.39–1.1.45 `--save-text-lockfile`) holds `workspace:` packages; frozen installs of that grammar cannot keep the hosted tuple. Detail: "Bun version-0 workspace locks cannot preserve hosted tarballs on frozen installs; delete bun.lock and re-run `bun install` with Bun >= 1.2 (which writes lockfileVersion 1, accepted by hosted mode) — a plain in-place `bun install` bumps the version only when a workspace depends on another workspace (e.g. root -> member); otherwise it keeps version 0 or fails to resolve" (measured: Bun 1.2.0 keeps 0, 1.2.23–1.4.2 exit 1 "failed to resolve" on a root that does not depend on its members). Version-1/2 workspace locks are rewritten. Exit 0. | diff --git a/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs b/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs index ec6fc908a..6401c6355 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_build/poetry.rs @@ -1179,9 +1179,10 @@ fn poetry_vendored_revendors_to_a_superseding_patch() { /// `six = {version = "1.16.0", source = "pypi"}` leaves no trace in the /// lock, and Poetry 1.3 – 2.2 refuse to install a lock that then sources six /// from a URL or file ("Repository \"pypi\" does not exist."; 1.3 and 1.4 -/// report a solver failure). Over a lock one of them wrote, both modes -/// refuse and leave the lock installable; other releases install the -/// rewired lock, with a warning (#1282, #1447). +/// report a solver failure), and every Poetry 1.3+ re-solves, then refuses, +/// a lock-version 1.0 / 1.1 lock (Poetry 1.0 – 1.2 write those). Over such a lock +/// both modes refuse and leave the lock installable; 2.3+ install the +/// rewired lock, with a warning (#1282, #1447, #1503). #[test] #[ignore = "real Poetry + PyPI; run by the CI e2e matrix per Poetry release"] fn poetry_source_pypi_constraint_never_breaks_install() { @@ -1206,7 +1207,10 @@ fn poetry_source_pypi_constraint_never_breaks_install() { return; }; // The lock was written by this Poetry, so it is also the installer. - let refused = ((1, 3)..(2, 3)).contains(&poetry.major_minor()); + // 1.1 and 1.2 install their own lock, but they write lock format 1.1, + // which every Poetry 1.3+ re-solves and then refuses (#1503): refused + // too, since the installer usually outlives the lock. + let refused = poetry.major_minor() < (2, 3); let mut patched = pristine.clone(); patched.extend_from_slice(MARKER); let wheel = build_wheel(&patched); diff --git a/crates/socket-patch-core/src/patch/redirect/poetry.rs b/crates/socket-patch-core/src/patch/redirect/poetry.rs index e5c505426..936aae853 100644 --- a/crates/socket-patch-core/src/patch/redirect/poetry.rs +++ b/crates/socket-patch-core/src/patch/redirect/poetry.rs @@ -11,8 +11,8 @@ use super::{DepOverride, FileEdit, RewriteResult, RewriteWarning}; use crate::crawlers::python_crawler::canonicalize_pypi_name; use crate::utils::poetry_lock::{ generated_by_version, lock_version, lock_wiring, pyproject_source_constraints, - rewrite_poetry_lock_all_with_version, source_constraint_advisory, - source_constraint_breaks_writer, source_constraint_refusal, LockBatch, LockStep, PoetryLockDep, + rewrite_poetry_lock_all_with_version, source_constraint_advisory, source_constraint_break, + source_constraint_refusal, LockBatch, LockStep, PoetryLockDep, }; /// Whether the lock was written by a Poetry release older than 1.4. Those @@ -71,8 +71,9 @@ pub(super) fn rewrite_poetry( for (path, original) in locks { // A pyproject `source = "…"` constraint on the dep (#1282). Poetry // 1.3 – 2.2 fail every install of a lock that sources it from a URL, - // so a lock one of them wrote is refused; other writers are warned. - // The constraint lives in the pyproject beside the lock. + // and every 1.3+ fails a lock-version 1.0 / 1.1 one (#1503), so such + // a lock is refused; Poetry 2.3+'s are warned. The constraint lives in + // the pyproject beside the lock. let pyproject_path = format!( "{}pyproject.toml", path.strip_suffix("poetry.lock").unwrap_or("") @@ -83,7 +84,12 @@ pub(super) fn rewrite_poetry( .unwrap_or_default(); let constraint = |dep: &DepOverride| constraints.get(&canonicalize_pypi_name(&dep.name)); let mut usable = usable.clone(); - if !constraints.is_empty() && source_constraint_breaks_writer(original) { + let breaks = if constraints.is_empty() { + None + } else { + source_constraint_break(original) + }; + if let Some(why) = &breaks { usable.retain(|&(dep, _)| { // A dep the lock doesn't list keeps its `entry_not_found` // report; one it lists (even already redirected) is refused. @@ -108,6 +114,7 @@ pub(super) fn rewrite_poetry( source, "URL", wired, + why, ) ), }); @@ -359,12 +366,13 @@ mod equivalence_tests { /// A dependency the pyproject pins to a named source (`source = "pypi"`) /// has no trace in the lock. Rewriting it to a URL source breaks every - /// `poetry install` on Poetry 1.3 – 2.2, so a lock one of those wrote is - /// refused before any write; other writers are rewritten with a warning - /// (#1282, #1447). A sibling lock whose pyproject has no constraint is - /// rewritten as before. + /// `poetry install` on Poetry 1.3 – 2.2, and on every Poetry 1.3+ over a + /// lock-version 1.0 / 1.1 lock (which they re-solve), so such a lock is + /// refused before any write; Poetry 2.3+'s locks are rewritten with a + /// warning (#1282, #1447, #1503). A sibling lock whose pyproject has no + /// constraint is rewritten as before. #[test] - fn source_constrained_dependency_is_refused_for_poetry_1_3_to_2_2() { + fn source_constrained_dependency_is_refused_unless_poetry_2_3_installs_it() { const PYPROJECTS: &[&str] = &[ "[tool.poetry.dependencies]\nurllib3 = {version = \"1.26.18\", source = \"pypi\"}\n", "[tool.poetry.dependencies.URLLib3]\nversion = \"1.26.18\"\nsource = \"PyPI\"\n", @@ -374,18 +382,23 @@ mod equivalence_tests { "[[tool.poetry.dependencies.urllib3]]\nversion = \"1.26.18\"\npython = \">=3.8\"\nsource = \"pypi\"\n\n[[tool.poetry.dependencies.urllib3]]\nversion = \"1.26.17\"\npython = \"<3.8\"\n", ]; for version in VERSIONS.iter().filter(|version| !version.starts_with("0.")) { - let broken = matches!( - *version, - "1.3.2" - | "1.4.2" - | "1.5.1" - | "1.6.1" - | "1.7.1" - | "1.8.5" - | "2.0.1" - | "2.1.4" - | "2.2.1" - ); + // Every lock-version 1.0 / 1.1 lock (Poetry 1.0 – 1.2) is + // re-solved by any Poetry 1.3+ on install, which then + // refuses the constraint like 1.3 – 2.2 do (#1503). + let old_format = matches!(*version, "1.0.10" | "1.1.15" | "1.2.2"); + let broken = old_format + || matches!( + *version, + "1.3.2" + | "1.4.2" + | "1.5.1" + | "1.6.1" + | "1.7.1" + | "1.8.5" + | "2.0.1" + | "2.1.4" + | "2.2.1" + ); for pyproject in PYPROJECTS { let lock = fixture(version); let files = BTreeMap::from([ @@ -425,6 +438,11 @@ mod equivalence_tests { ); assert_eq!(codes, ["redirect_poetry_lock_unsupported"], "{what}"); assert!(got.warnings[0].detail.contains("source ="), "{what}"); + assert_eq!( + got.warnings[0].detail.contains("lock-version"), + old_format, + "{what}" + ); // Nothing is wired yet, so re-locking is the whole remedy. assert!(!got.warnings[0].detail.contains("rollback"), "{what}"); } else { diff --git a/crates/socket-patch-core/src/patch/redirect/pypi_takeover.rs b/crates/socket-patch-core/src/patch/redirect/pypi_takeover.rs index 271bf09e5..a7595a3f4 100644 --- a/crates/socket-patch-core/src/patch/redirect/pypi_takeover.rs +++ b/crates/socket-patch-core/src/patch/redirect/pypi_takeover.rs @@ -17,8 +17,8 @@ use super::RewriteWarning; use crate::crawlers::python_crawler::canonicalize_pypi_name; use crate::utils::fs::read_regular_to_string; use crate::utils::poetry_lock::{ - lock_version, pyproject_source_constraints, source_constraint_breaks_writer, - source_constraint_refusal, Wired, + lock_version, pyproject_source_constraints, source_constraint_break, source_constraint_refusal, + Wired, }; use crate::vendor::state::VendorEntry; @@ -119,8 +119,8 @@ fn recorded_unit_version(unit: &str) -> Option { /// Hosted mode refuses every Poetry 0.x lock (Poetry 0.12 ignores URL /// sources), which vendored mode supports (#945), and a package the -/// pyproject pins to a named source over a lock Poetry 1.3 – 2.2 wrote -/// (#1282). The revert never changes the lock's format, stamp or the +/// pyproject pins to a named source over a lock Poetry 1.3 – 2.2 wrote or +/// a lock-version 1.0 / 1.1 one (#1282, #1503). The revert never changes the lock's format, stamp or the /// pyproject, so the files on disk decide. async fn preflight_poetry_takeover(root: &Path, entry: &VendorEntry) -> Result<(), RewriteWarning> { let lock_file = entry @@ -152,16 +152,16 @@ async fn preflight_poetry_takeover(root: &Path, entry: &VendorEntry) -> Result<( } /// The hosted rewriter's source-pin refusal (#1282), predicted from the -/// pyproject beside the lock and the lock's writer stamp. +/// pyproject beside the lock and the lock's writer stamp and format. async fn preflight_poetry_source_constraint( root: &Path, lock_file: &str, lock_text: &str, entry: &VendorEntry, ) -> Result<(), RewriteWarning> { - if !source_constraint_breaks_writer(lock_text) { + let Some(why) = source_constraint_break(lock_text) else { return Ok(()); - } + }; let pyproject_file = format!( "{}pyproject.toml", lock_file.strip_suffix("poetry.lock").unwrap_or("") @@ -191,7 +191,8 @@ async fn preflight_poetry_source_constraint( version, &source, "URL", - Wired::Vendored + Wired::Vendored, + &why, ), ), }) @@ -357,7 +358,8 @@ mod tests { } /// #1282: hosted mode refuses a source-pinned package over a lock - /// Poetry 1.3 – 2.2 wrote, so the takeover must keep it vendored rather + /// Poetry 1.3 – 2.2 wrote, or a lock-version 1.0 / 1.1 one (#1503), so the + /// takeover must keep it vendored rather /// than revert it into neither mode. Other writers are admitted. #[tokio::test] async fn poetry_source_pinned_package_follows_the_writer() { @@ -375,7 +377,11 @@ mod tests { by hand.\n\n{}", POETRY_2.replace("lock-version = \"2.1\"", "lock-version = \"2.0\"") ); + // Poetry 1.1 / 1.2 stamp nothing and write lock 1.1, which every + // Poetry 1.3+ re-solves and refuses (#1503). + let poetry_1_2 = POETRY_2.replace("lock-version = \"2.1\"", "lock-version = \"1.1\""); for (stamp, lock, refused) in [ + ("1.2.2", poetry_1_2, true), ("1.3.2", poetry_1_3, true), ("1.8.5", stamp("1.8.5"), true), ("2.2.1", stamp("2.2.1"), true), diff --git a/crates/socket-patch-core/src/utils/poetry_lock.rs b/crates/socket-patch-core/src/utils/poetry_lock.rs index 86a6b59fa..487db9c3c 100644 --- a/crates/socket-patch-core/src/utils/poetry_lock.rs +++ b/crates/socket-patch-core/src/utils/poetry_lock.rs @@ -284,11 +284,13 @@ pub fn lock_wiring(lock_text: &str, name: &str, version: &str) -> Option } /// The refusal detail for a source-pinned package over a lock that -/// [`source_constraint_breaks_writer`] flags; `kind` is how the rewrite +/// [`source_constraint_break`] flags (`why`); `kind` is how the rewrite /// would source it (`URL` hosted, `file` vendored). Dropping the key or -/// upgrading Poetry both mean re-locking, which discards whatever the lock -/// already wires: `wired` (the lock still carries a vendored or hosted pin -/// for the package) puts `socket-patch vendor --revert` or `rollback` first. +/// re-locking with Poetry >= 2.3 both mean re-locking, which discards +/// whatever the lock already wires: `wired` (the lock still carries a +/// vendored or hosted pin for the package) puts `socket-patch vendor +/// --revert` or `rollback` first. +#[allow(clippy::too_many_arguments)] pub fn source_constraint_refusal( pyproject_path: &str, lock_path: &str, @@ -297,6 +299,7 @@ pub fn source_constraint_refusal( source: &str, kind: &str, wired: Wired, + why: &SourcePinBreak, ) -> String { let first = match wired { Wired::No => "", @@ -309,12 +312,22 @@ pub fn source_constraint_refusal( hosted pin; a bare `rollback` undoes every patch in the project), then " ), }; + let lock = match why { + SourcePinBreak::Writer => format!( + "{lock_path} was written by Poetry 1.3 – 2.2, whose installer refuses that \ + constraint on a package the lock sources from a {kind}: every `poetry install` \ + would fail" + ), + SourcePinBreak::Format(format) => format!( + "{lock_path} has lock-version {format:?}, which every Poetry from 1.3 on \ + re-solves on install, and that solve refuses the constraint on a package the \ + lock sources from a {kind}: `poetry install` would fail on any of them" + ), + }; format!( "{pyproject_path} pins {name} to the {source:?} source (`source = {source:?}`), and \ - {lock_path} was written by Poetry 1.3 – 2.2, whose installer refuses that constraint \ - on a package the lock sources from a {kind}: every `poetry install` would fail. To \ - patch it, {first}drop the `source` key or upgrade to Poetry >= 2.3, re-lock, and \ - re-run" + {lock}. To patch it, {first}drop the `source` key or upgrade to Poetry >= 2.3, \ + re-lock, and re-run" ) } @@ -330,8 +343,10 @@ pub enum Wired { Hosted, } -/// The advisory for a source-pinned package rewired over a lock that a -/// Poetry outside 1.3 – 2.2 wrote. +/// The advisory for a source-pinned package rewired over a lock that +/// [`source_constraint_break`] admits: lock format 2.1+, not written by +/// Poetry 1.3 – 2.2. Poetry 1.3 – 1.8 also read a 2.1 lock (with a +/// compatibility warning) and refuse the constraint like 2.0 – 2.2 do. pub fn source_constraint_advisory( pyproject_path: &str, lock_path: &str, @@ -340,8 +355,8 @@ pub fn source_constraint_advisory( ) -> String { format!( "{pyproject_path} pins {name} to the {source:?} source (`source = {source:?}`): \ - Poetry 1.3 – 2.2 refuse to install {lock_path} with that constraint (1.1 – 1.2 and \ - 2.3+ install it); drop the `source` key if this project is installed with one of them" + Poetry 1.3 – 2.2 refuse to install {lock_path} with that constraint (2.3+ install \ + it); drop the `source` key if this project is installed with one of them" ) } @@ -360,15 +375,47 @@ pub fn lock_writer_version(lock_text: &str) -> Option<(u64, u64)> { (lock_version(&lock).ok()? == "2.0").then_some((1, 3)) } -/// Whether the lock's writer is a Poetry release (1.3 – 2.2) whose installer -/// refuses a pyproject `source = "…"` constraint on a package the lock -/// sources from a URL or file: every `poetry install` fails ("Repository -/// \"pypi\" does not exist."; 1.3 and 1.4 report a solver failure). 1.1 – -/// 1.2 and 2.3+ install it. Keyed on the lock's header -/// ([`lock_writer_version`]), so it judges the writer, which is usually but -/// not always the installer. -pub fn source_constraint_breaks_writer(lock_text: &str) -> bool { - matches!(lock_writer_version(lock_text), Some(v) if ((1, 3)..(2, 3)).contains(&v)) +/// Why a lock can't source a pyproject-source-pinned package from a URL or +/// file: see [`source_constraint_break`]. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum SourcePinBreak { + /// Poetry 1.3 – 2.2 wrote it, and their installer refuses the + /// constraint. + Writer, + /// The lock's `lock-version` is 1.0 or 1.1 (Poetry 1.0 – 1.2), which + /// every Poetry from 1.3 on re-solves on install (they trust the lock + /// as is only from format 2.1); the solve refuses the constraint. Only + /// the old Poetry that wrote such a lock installs it (#1503). The + /// legacy 0.x layout is not judged here: hosted refuses it outright, + /// and vendored mode keeps admitting it. + Format(String), +} + +/// Whether a lock that sources a pyproject `source = "…"`-pinned package +/// from a URL or file breaks `poetry install` ("Repository \"pypi\" does +/// not exist."; 1.3 and 1.4 report a solver failure), and why: its writer +/// is Poetry 1.3 – 2.2 ([`lock_writer_version`]), or its format is older +/// than 2.1. Poetry 2.3+'s own locks install it. The writer is usually but +/// not always the installer, so a 2.1 lock is admitted (with the +/// [`source_constraint_advisory`]); a 1.0 / 1.1 lock is refused whichever +/// Poetry installs it, since only Poetry 1.0 – 1.2 would. +/// +/// A versioned stamp decides without parsing (only 1.4+ write one, and +/// none of them writes a format older than 2.0); otherwise the lock is +/// parsed once for its `lock-version`, which also names Poetry 1.3 (the +/// only unstamped writer of 2.0, as [`lock_writer_version`] reads it). +pub fn source_constraint_break(lock_text: &str) -> Option { + if let Some(writer) = generated_by_version(lock_text) { + return ((1, 3)..(2, 3)) + .contains(&writer) + .then_some(SourcePinBreak::Writer); + } + let lock = lock_text.parse::().ok()?; + match lock_version(&lock).ok()? { + "2.0" => Some(SourcePinBreak::Writer), + format @ ("1.0" | "1.1") => Some(SourcePinBreak::Format(format.into())), + _ => None, + } } /// Rewrite the `[[package]]` for `name`@`version` to install the wheel at @@ -901,30 +948,44 @@ grouped = {version = "*", source = "pypi"} } #[test] - fn source_constraint_breaks_only_poetry_1_3_to_2_2_writers() { + fn source_constraint_breaks_1_3_to_2_2_writers_and_pre_2_1_locks() { + use SourcePinBreak::{Format, Writer}; let stamp = |v: &str| { format!("# This file is automatically @generated by Poetry {v} and should not be changed by hand.\n") }; for (v, want) in [ - ("1.2.2", false), - ("1.3.0", true), - ("1.4.0", true), - ("1.8.5", true), - ("2.2.1", true), - ("2.3.0", false), - ("2.4.3", false), + ("1.2.2", None), + ("1.3.0", Some(Writer)), + ("1.4.0", Some(Writer)), + ("1.8.5", Some(Writer)), + ("2.2.1", Some(Writer)), + ("2.3.0", None), + ("2.4.3", None), ] { - assert_eq!(source_constraint_breaks_writer(&stamp(v)), want, "{v}"); + assert_eq!(source_constraint_break(&stamp(v)), want, "{v}"); } - assert!(!source_constraint_breaks_writer("[[package]]\n")); + assert_eq!(source_constraint_break("[[package]]\n"), None); // Poetry 1.3 writes the stamp without a version (#1447). Its lock is // still recognizable: only 1.3 – 1.8 write `lock-version = "2.0"`, - // and 1.4 – 1.8 stamp their version. - assert!(source_constraint_breaks_writer(&fixture("1.3.2"))); - for version in ["0.12.17", "1.0.10", "1.1.15", "1.2.2", "2.3.4", "2.4.3"] { - assert!( - !source_constraint_breaks_writer(&fixture(version)), + // and 1.4 – 1.8 stamp their version. Lock-version 1.0 / 1.1 locks + // break on every Poetry 1.3+, which re-solves them (#1503). The 0.x + // layout is left to the callers (hosted refuses it outright). + let format = |f: &str| Some(Format(f.into())); + for (version, want) in [ + ("0.12.17", None), + ("1.0.10", format("1.0")), + ("1.1.15", format("1.1")), + ("1.2.2", format("1.1")), + ("1.3.2", Some(Writer)), + ("1.8.5", Some(Writer)), + ("2.2.1", Some(Writer)), + ("2.3.4", None), + ("2.4.3", None), + ] { + assert_eq!( + source_constraint_break(&fixture(version)), + want, "{version}" ); } @@ -934,28 +995,66 @@ grouped = {version = "*", source = "pypi"} format!("[metadata]\nlock-version = \"{lock_version}\"\npython-versions = \"*\"\ncontent-hash = \"x\"\n") }; for (header, lock_version, want) in [ - (versionless, "2.0", true), + (versionless, "2.0", Some(Writer)), // A 2.0 lock whose stamp was edited away is still 1.3 – 1.8's. - ("", "2.0", true), - ("# a comment above the stamp\n", "2.0", true), - // The versionless stamp judges nothing over other formats. - (versionless, "1.1", false), - (versionless, "2.1", false), - ("", "1.1", false), - ("", "2.1", false), + ("", "2.0", Some(Writer)), + ("# a comment above the stamp\n", "2.0", Some(Writer)), + (versionless, "1.1", format("1.1")), + ("", "1.1", format("1.1")), + ("", "1.0", format("1.0")), + (versionless, "2.1", None), + ("", "2.1", None), + ("", "2.2", None), ] { let lock = format!("{header}{}", lock_body(lock_version)); assert_eq!( - source_constraint_breaks_writer(&lock), + source_constraint_break(&lock), want, "{header:?} {lock_version}" ); assert_eq!( - source_constraint_breaks_writer(&lock.replace('\n', "\r\n")), + source_constraint_break(&lock.replace('\n', "\r\n")), want, "CRLF {header:?} {lock_version}" ); } + // The legacy layout (no lock-version, `[metadata.hashes]`). + assert_eq!( + source_constraint_break("[metadata]\ncontent-hash = \"x\"\n\n[metadata.hashes]\n"), + None + ); + } + + /// The refusal names why: the 1.3 – 2.2 writer, or the old format. + #[test] + fn source_constraint_refusal_names_the_writer_or_the_format() { + let refusal = |why| { + source_constraint_refusal( + "pyproject.toml", + "poetry.lock", + "six", + "1.16.0", + "pypi", + "URL", + Wired::No, + &why, + ) + }; + let writer = refusal(SourcePinBreak::Writer); + assert!(writer.contains("written by Poetry 1.3 – 2.2"), "{writer}"); + let format = refusal(SourcePinBreak::Format("1.1".into())); + assert!(format.contains("lock-version \"1.1\""), "{format}"); + assert!(format.contains("every Poetry from 1.3 on"), "{format}"); + for detail in [writer, format] { + assert!(detail.contains("`source = \"pypi\"`"), "{detail}"); + assert!( + detail.contains("upgrade to Poetry >= 2.3, re-lock"), + "{detail}" + ); + } + let advisory = source_constraint_advisory("pyproject.toml", "poetry.lock", "six", "pypi"); + assert!(advisory.contains("Poetry 1.3 – 2.2 refuse"), "{advisory}"); + assert!(!advisory.contains("1.1 – 1.2"), "{advisory}"); } const WHEEL: &str = "urllib3-1.26.18-py2.py3-none-any.whl"; diff --git a/crates/socket-patch-core/src/vendor/pypi_poetry.rs b/crates/socket-patch-core/src/vendor/pypi_poetry.rs index c8fa8bd0e..76553ab16 100644 --- a/crates/socket-patch-core/src/vendor/pypi_poetry.rs +++ b/crates/socket-patch-core/src/vendor/pypi_poetry.rs @@ -196,10 +196,10 @@ fn source_constraint(p: &PoetryProject, canon_name: &str) -> Option { poetry_lock::pyproject_source_constraints(p.pyproject_text.as_deref()?).remove(canon_name) } -/// The advisory for a source-constrained package wired over a lock that a -/// Poetry outside 1.3 – 2.2 wrote: the writer installs it, but a 1.3 – 2.2 -/// installer would refuse it. [`check_target_guards`] refuses the 1.3 – 2.2 -/// writers outright. +/// The advisory for a source-constrained package wired over a lock Poetry +/// 2.3+ wrote: the writer installs it, but a 1.3 – 2.2 installer would +/// refuse it. [`check_target_guards`] refuses the locks every 1.3+ (or the +/// 1.3 – 2.2 writer) would refuse outright. pub(super) fn source_constraint_warning( p: &PoetryProject, canon_name: &str, @@ -212,16 +212,20 @@ pub(super) fn source_constraint_warning( } /// Refuse a source-constrained package over a lock written by Poetry -/// 1.3 – 2.2: its installer fails every install once the lock sources the -/// package from the vendored wheel (#1282). +/// 1.3 – 2.2, or a lock-version 1.0 / 1.1 one: a Poetry 1.3+ installer fails +/// every install once the lock sources the package from the vendored wheel +/// (#1282, #1503). fn source_constraint_guard( p: &PoetryProject, canon_name: &str, version: &str, wired: poetry_lock::Wired, ) -> Result<(), (&'static str, String)> { - match source_constraint(p, canon_name) { - Some(source) if poetry_lock::source_constraint_breaks_writer(&p.lock_text) => Err(( + let Some(source) = source_constraint(p, canon_name) else { + return Ok(()); + }; + match poetry_lock::source_constraint_break(&p.lock_text) { + Some(why) => Err(( "pypi_poetry_source_constraint", poetry_lock::source_constraint_refusal( "pyproject.toml", @@ -231,9 +235,10 @@ fn source_constraint_guard( &source, "file", wired, + &why, ), )), - _ => Ok(()), + None => Ok(()), } } @@ -319,7 +324,8 @@ pub(super) fn check_target_guards( )); } // A pyproject `source = "…"` pin breaks every install of the wired lock - // on Poetry 1.3 – 2.2; refuse when one of those wrote the lock. + // on Poetry 1.3 – 2.2, and on every 1.3+ when the lock is format 1.0 / + // 1.1; refuse those locks. source_constraint_guard(p, canon_name, version, poetry_lock::Wired::No)?; Ok(PoetryTarget::Fresh) } @@ -892,18 +898,23 @@ content-hash = "4b42a89b7ff7b26511b06acdc458dbd85312e5083db8f212b017482bc68cdd01 /// A pyproject `source = "pypi"` constraint on the vendored package has /// no trace in the lock. Rewiring it to a file source breaks every - /// `poetry install` on Poetry 1.3 – 2.2, so a lock one of those wrote is - /// refused before the wheel is built; other writers wire it with a - /// warning (#1282, #1447). + /// `poetry install` on Poetry 1.3 – 2.2, and on every Poetry 1.3+ over a + /// lock-version 1.0 / 1.1 lock, so such a lock is refused before the + /// wheel is built; Poetry 2.3+'s (and 0.x's) locks wire it with a warning + /// (#1282, #1447, #1503). #[tokio::test] - async fn source_constrained_dependency_is_refused_for_poetry_1_3_to_2_2() { + async fn source_constrained_dependency_is_refused_unless_poetry_2_3_installs_it() { const PYPROJECT: &str = "[tool.poetry.dependencies]\n\ python = \">=3.8\"\n\ urllib3 = {version = \"1.26.18\", source = \"pypi\"}\n"; for (version, broken) in [ + // Lock 1.0 / 1.1: every Poetry 1.3+ re-solves them on install + // and then refuses the constraint (#1503). The 0.x layout is + // still admitted, as before. ("0.12.17", false), - ("1.1.15", false), - ("1.2.2", false), + ("1.0.10", true), + ("1.1.15", true), + ("1.2.2", true), ("1.3.2", true), ("1.4.2", true), ("1.6.1", true), diff --git a/docs/testing/poetry-compatibility.md b/docs/testing/poetry-compatibility.md index d57f1dec2..7152c4a33 100644 --- a/docs/testing/poetry-compatibility.md +++ b/docs/testing/poetry-compatibility.md @@ -40,12 +40,19 @@ A dependency the pyproject pins to a named source (`six = {version = "1.16.0", source = "pypi"}`, in the main, legacy dev or any group table) leaves no trace in the lock. Poetry 1.3 – 2.2 refuse to install a lock that then sources it from a URL or file (`Repository "pypi" does not exist.`; 1.3 -and 1.4 report a solver failure); 1.1 – 1.2 and 2.3+ install it (measured, -#1282, #1447). So when the lock names a 1.3 – 2.2 writer (its `@generated -by Poetry X.Y` stamp, or, with no versioned stamp, `lock-version = "2.0"`, -which only 1.3 – 1.8 write; 1.3 stamps no version), both modes refuse that -package before any write (`redirect_poetry_lock_unsupported` hosted, -`pypi_poetry_source_constraint` vendored); otherwise they rewire it and warn +and 1.4 report a solver failure); 2.3+ install it (measured, #1282, #1447), +but only from a lock in format 2.1 or later. Every Poetry from 1.3 on +re-solves an older lock (`lock-version` 1.0 / 1.1, written by Poetry 1.0 – +1.2) on install instead of trusting it, and that solve refuses the +constraint the same way, so only Poetry 1.0 – 1.2 install such a lock +(measured on 1.3.2, 2.0.1, 2.2.1, 2.3.4, 2.4.3 and 2.5.1, #1503). So both +modes refuse that package before any write +(`redirect_poetry_lock_unsupported` hosted, `pypi_poetry_source_constraint` +vendored) when the lock names a 1.3 – 2.2 writer (its `@generated by Poetry +X.Y` stamp, or, with no versioned stamp, `lock-version = "2.0"`, which only +1.3 – 1.8 write; 1.3 stamps no version) or when its `lock-version` is 1.0 +or 1.1; the detail says which. Over a 2.3+ lock (or vendored, a 0.x one) +they rewire it and warn (`redirect_poetry_source_constraint` / `pypi_poetry_source_constraint`) that a 1.3 – 2.2 installer would refuse the lock. The header names the writer, which is usually but not always the installer.