Skip to content

Fix vendored planning ignoring PyPI refusals (#1908) - #1930

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-scan-vendor-preview-planning
Nov 2, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-scan-vendor-preview-planning

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1908

Summary

scan --mode vendored picks which new patches may take a --max-new-patches slot using a "would the wet run refuse this?" planning pass. That pass was a hand-written copy of the dry-run preview's preflight set, and the copy never ran the PyPI wiring guards (pypi_refusals). So a PyPI patch the wet run refuses before downloading (pypi_no_requirements, pypi_uv_no_sources) still took a rollout slot and pushed an installable patch to the next run, while --dry-run showed the same patch as would_refuse.

Root cause

Two functions answered the same question: preview_vendor (dry-run rows) and preflight_refused_purls (planning). They drifted. The planning copy dropped pypi_refusals.

Change

  • preview_vendor(common, selected) is now the single entry point. It resolves the --patch-server-url origins and the gem takeover refusals itself, where scan JSON, scan human and get each used to build them by hand. Unit tests drive the inner preview_vendor_with.
  • The planning pass reads VendorPreview::refused_purls(). preflight_refused_purls is deleted.
  • scan computes the preview once per run. The dry-run rendering reuses it, trimmed to the post-plan selection with VendorPreview::retain_selected, instead of running every preflight a second time. This is safe because each verdict is per patch: Bun's exemptions, vlt, PyPI and gem are all judged purl by purl.
  • get no longer imports vendor, so its edge is removed from the layering allowlist.

Not done from the issue's list: replacing the test-only preview_vendor_json helper with assertions on PreviewVerdict. It's test scaffolding and isn't needed for the fix.

Tests

Issue Test
#1908 scan::vendor_flow::preview_tests::planning_pass_excludes_a_pypi_refusal
  • Red on main (test against the old preflight_refused_purls): refused was {}, so the test FAILED. Green on this branch, together with its control case (the same project without no-sources keeps its slot).
  • cargo fmt --all -- --check: ok. cargo clippy --workspace --all-features -- -D warnings: ok.
  • cargo test --workspace --all-features --no-fail-fast: everything passes, including scan_rollout_e2e, scan_vendor_e2e, scan_vendor_requirements_unwired, all 26 vendor_flow unit tests and command_module_layering. 11 tests fail only in this sandbox, which runs as root and has no access to the live patch API. They are chmod-based write-failure tests (copy_tree, vlt_heal, pypi_poetry/pypi_requirements wire failures, repair_*, in_process_redirect write failures) and two e2e_bun_lockb tests that reach the live blob API. None of them touch the code in this PR, so CI is the judge.

Code review (/code-review high)

Fixed: an unreachable recompute fallback in scan's dry-run branch, and a confusing test import alias.

Not fixed, with reasons:

  • The preview doesn't predict every engine refusal. The yarn, yarn-berry, Cargo and non-hosted pnpm lock-text gates in lock_text_refusals aren't covered, so in those ecosystems a refused new patch can still take a slot. This gap predates the PR and is documented on preview_vendor ("engine refusals outside the preflights are not predicted"). Adding those refusals would add new would_refuse rows, which the issue's acceptance criteria rule out ("preview rows are unchanged"). Follow-up: route preview_vendor_with through lock_text_refusals too.
  • Wet runs now run the PyPI guards and the symlink probes during planning. Running the PyPI guards there is the fix itself. The symlink probes are a few metadata reads per row.
  • hosted_claimed_purls is read once in pypi and once in pnpm. This predates the PR and is unchanged here.
  • The test asserts on refused_purls rather than Stage::plan. Both planning call sites now pass refused_purls() to stage.plan directly, with no other logic in between.

https://claude.ai/code/session_011f9aeQujB4vD43SMXRBSVZ


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
scan --mode vendored chose which new patches may take a
--max-new-patches slot through its own copy of the preflight checks,
and that copy never ran the PyPI wiring guards. A PyPI patch the wet
run refuses before downloading (pypi_no_requirements,
pypi_uv_no_sources) still took a rollout slot and pushed an
installable patch to the next run, while --dry-run showed the same
patch as would_refuse.

The planning pass now reads the dry-run preview's would-refuse rows,
so the two can no longer disagree. preview_vendor takes the run's
GlobalArgs and resolves the patch-server origins and gem takeover
refusals itself, and scan computes it once per run, reusing it for
the dry-run rendering instead of running every preflight twice.

Fixes #1908

Assisted-by: Claude Code:claude-opus-5-5
The dry-run rendering now reuses the planning pass's preview directly
instead of keeping an unreachable recompute path, the preview tests
call the inner helper by its own name, and get's layering allowlist
drops its import of vendor, which it no longer needs.

Refs #1908

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review November 1, 2026 12:57
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

1 similar comment
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 2a26a39. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 1, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review (burn-down agent).

  • Head: 2a26a39b735e361f1a955b37c6ef4ffd7fecdff8
  • CI: 82/82 checks green (32 success, 50 skipped), no main-wide failures
  • Bugbot: reviewed 2a26a39b73, no findings; no unresolved review threads
  • Mergeable, no CHANGELOG.md change
  • Reviewer focus: scan now reuses one VendorPreview per run (retain_selected) instead of rerunning preflights for the dry-run rendering.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does: scan --mode vendored decided which new patches get a --max-new-patches slot by calling preflight_refused_purls, a hand-kept copy of the dry-run preview's preflights. That copy had dropped the PyPI wiring guards, so a PyPI patch the wet run refuses still took a slot. This PR deletes the copy: planning now reads VendorPreview::refused_purls() from the one preview_vendor, which also resolves the origins and gem takeover refusals itself. scan reuses the preview it computed for planning in the dry-run rendering, trimmed to the post-plan selection.

Risk: Low to medium. The planning logic is simpler, but wet vendored runs now also run the PyPI guards during planning (that is the fix, and it can probe pipenv --version and Hatch envs). The dry-run preview is now computed over the pre-plan selection and then trimmed, which is only correct because every verdict is per purl. I checked this: Bun's refusal is project-level with a per-purl applies_to and ledger exemption, and the PyPI, pnpm and gem refusals are keyed by purl.

Look here:

  • scan/mod.rs:3308: the planning pass builds the preview once, and scan/mod.rs:3434 reuses it (expect is safe because vendor and report_only are mutually exclusive mode values)
  • vendor_flow.rs:116 and :128:`` refused_purls and `retain_selected` (keyed by `(purl, uuid)`)
  • vendor_flow.rs:220:``` the new preview_vendor(common, selected) entry point
  • vendor_flow.rs:777:```` the same change in the JSON path
  • vendor_flow.rs:1926:````` the regression test plus its control case

Verified: Read the full diff. Confirmed selected isn't changed between planning and the dry-run render in run_scan, and that get no longer needs the vendor layering exception. Ran cargo test -p socket-patch-cli --lib -- vendor_flow (26 passed) and --test command_module_layering (3 passed). CHANGELOG.md is untouched. CI 82/82 green, Bugbot ✅ (no new issues).

Changes I made: none.

Open questions: none blocking. A known gap remains and is documented: the yarn, Cargo and non-hosted pnpm lock-text engine refusals still aren't predicted by the preview, so a refused new patch in those ecosystems can still take a slot. The follow-up is in the PR description.

Auto-merge (squash) is armed, so approving sends it straight to the merge queue.


Generated by Claude Code

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[agent] Reviewed 2a26a39b735e361f1a955b37c6ef4ffd7fecdff8. Checked shared vendor preview refusals through rollout eligibility and dry-run selection retention. No major correctness issues found. Validation: source and regression-test review plus the available exact-head CI checks; no local test pass is claimed.

@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 01e40cb Nov 2, 2026
82 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-scan-vendor-preview-planning branch November 2, 2026 16:30
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Nov 2, 2026
#1930 made the rollout plan read the dry-run preview, so the
lock-text refusal now lives only in the preview and the separate
planning pass is gone. Also corrects the contract: only a detached
ledger entry already holding this uuid's record is left out.

Assisted-by: Claude Code:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scan's vendored planning and preview run the refusal preflights twice, and the planning copy skips the PyPI refusals

3 participants