Fix vendored npm-family tarballs dropped by .gitignore (#831)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #831

Summary

Vendored mode in a yarn classic, yarn berry, npm, pnpm or bun project wrote .socket/vendor/npm/<uuid>/<name>-<ver>.tgz without checking whether git would commit it. GitHub's stock Node.gitignore ships *.tgz, and polyglot repos often ignore vendor/ or .socket/. Under any of these rules the scan exited 0 with no warning. The commit then kept the rewired lockfile and dropped the tarball, so every fresh checkout's frozen install failed. Under vendor/ or .socket/, vendor --check also passed, because it had no ledger left to check.

Root cause

All npm-family tarball backends stage their artifact through npm_common::stage_patch_pack. The vlt and directory paths already guard this with npm_dir::gitignore_probe, refusing with vendor_artifact_gitignored and writing a <uuid>/.gitignore that re-includes the payload. The shared tarball pipeline had neither, so the gap covered every tarball flavor, not just yarn classic (the yarn berry matrix in the issue comment is the same gap).

Changes

• stage_patch_pack (crates/socket-patch-core/src/vendor/npm_common.rs)
  • Before any write, dry run included: git check-ignore --no-index on <uuid>/. A rule that ignores the directory itself (vendor/, .socket/, .socket/vendor/) refuses with vendor_artifact_gitignored, naming the rule.
  • After the tarball is in place (fresh or reused): writes <uuid>/.gitignore (!*) and <uuid>/.gitattributes (-text), the same files and content as vlt. This overrides file rules such as *.tgz.
  • It then re-probes the tarball and both metadata files. If they are still ignored, it refuses and removes a uuid dir this run created. If git can't answer, it warns with vendor_artifact_gitignore_unchecked. Outside a git work tree nothing changes.
• vendor --check (commands/vendor.rs): reads the lockfile references (repair::scan_vendor_references, the scan repair and the orphan sweeps use). Each .socket/vendor/<eco>/<uuid>/ that no ledger entry owns fails with vendor_ledger_missing, using the same artifact-level shape repair emits (uuid + details.{ecosystem,path}). A reference whose manifest key was already reported as unledgered is not reported twice.
• CLI_CONTRACT.md: the vendor_artifact_gitignored / _unchecked rows now cover the tarball flavors. The vendoring table names the uuid .gitignore/.gitattributes. The vendor --check section documents the lock-reference check.

The npm/PyPI/gem wrappers only dispatch to the binary, so they need no parallel change.

Test evidence (red without the fix, green with it)

Issue scenario	Test	Without fix	With fix
#831 *.tgz (shared pipeline)	npm_common::tests::a_tgz_ignore_rule_is_overridden_by_the_uuid_gitignore	FAIL	pass
#831 vendor/, .socket/, .socket/vendor/ (wet and dry)	npm_common::tests::a_directory_ignore_rule_refuses_before_any_write	FAIL	pass
control: no work tree	npm_common::tests::no_work_tree_stages_without_a_refusal	pass	pass
#831 yarn classic *.tgz: real yarn 1.22.22, git commit then git clone, --frozen-lockfile --offline installs the patched bytes	e2e_vendor_yarn_classic_build::yarn_classic_vendored_tarball_survives_a_tgz_gitignore_rule	FAIL	pass
#831 vendor --check on a clone without ledger/manifest fails vendor_ledger_missing	same test, final block (red with only the vendor.rs change reverted)	FAIL	pass
#831 yarn classic vendor/ and .socket/: refused, yarn.lock untouched, nothing written	e2e_vendor_yarn_classic_build::yarn_classic_vendor_refuses_a_gitignored_vendor_dir	FAIL	pass
#831 yarn berry *.tgz: real yarn 4.12.0, clone then yarn install --immutable with an empty global folder installs the patched bytes	e2e_vendor_yarn_berry_build::yarn_berry_vendored_tarball_survives_a_tgz_gitignore_rule	FAIL	pass
#831 yarn berry .socket/: refused, yarn.lock and package.json untouched	e2e_vendor_yarn_berry_build::yarn_berry_vendor_refuses_a_gitignored_socket_dir	FAIL	pass

Local runs (Linux, as root):

• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test -p socket-patch-core --all-features --lib: 4845 passed, 4 failed. The 4 failures are permission tests that rely on chmod read-only dirs, which root bypasses (copy_tree, vlt_heal, pypi_poetry, pypi_requirements). This PR doesn't touch those modules.
• cargo test -p socket-patch-cli --all-features --lib: 834 passed.
• CLI suites passed: in_process_vendor 104, e2e_vendor_yarn_classic_build 13, e2e_vendor_yarn_berry_build 16, e2e_vendor_npm_build 17, e2e_vendor_pnpm_build 16, e2e_vendor_bun_build 14, e2e_bun_lockb 12, in_process_vendor_bun_takeover 20, scan_vendor_e2e 33, in_process_rollback_vendored 6, e2e_vex_vendor 23, e2e_vendor_yarn_classic_dev_flow 11, e2e_yarn4_workspaces_build 13, e2e_yarn4_pnpm_linker_build 15, cli_parse_vendor 32, e2e_vendor_vlt_build 25, coverage_fix_vendor_silent_mute_exit 3.
• Root-only failures:
  • covgap_commands_vendor: 41 passed, 3 failed (*_state_write_failure_*, which chmod 0o555 the vendor dir). The same 3 fail on main as root.
  • repair: 114 passed, 2 failed (unremovable-file tests, same class).
• A full cargo test --workspace doesn't fit this sandbox's disk allowance, so the CI matrix covers the rest.
• cargo fmt --all -- --check isn't clean on main with the pinned toolchain's rustfmt, and CI doesn't run it. The new code follows the surrounding style and no unrelated files are reformatted.

Notes

• An earlier push (002c870) briefly carried an accidental repo-wide rustfmt reformat that touched PDM files, which triggered the path-filtered PDM compatibility workflow. Its extras agent cell failed on that head. That head is superseded: the current branch touches only the six files above, and PDM compatibility doesn't run on it.
• Follow-up, out of scope: Vendored Gradle exits 0 when the project's .gitignore excludes *.jar, so the commit silently drops the patched jar and every fresh checkout fails to build #620 (vendored Gradle under *.jar) is the same symptom in a different backend.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FMZyKmgYNAridSR5eqv999

---

Note

Medium Risk
Changes core vendoring and pre-takeover ordering for npm-family lockfiles; mistakes could block vendoring or leave hosted wiring, but the design is fail-closed rather than silently dropping tarballs from commits.

Overview
Vendored npm-family tarball flows (npm, yarn classic/berry, pnpm, bun) now treat git ignore rules like the existing vlt/directory path: refuse before writing when the uuid directory itself would be ignored, write <uuid>/.gitignore and .gitattributes to re-include *.tgz (and similar file rules), then re-probe and roll back if the artifact still would not be committed. A new npm_tarball_gitignore_preflight runs on hosted→vendored takeover before restoring the registry lock entry, so a .socket/ ignore does not leave the package wired only to hosted mode.

vendor --check also scans lockfiles for .socket/vendor/<eco>/<uuid>/ references with no matching ledger entry and fails with vendor_ledger_missing (artifact-level event, aligned with repair), including cases the manifest alone cannot see.

Docs in CLI_CONTRACT.md are updated for tarball flavors and the check behavior. JVM/Gradle code paths switch to shared sha1_hex_of / sha256_hex_of helpers (no behavior change). E2E and in-process tests cover *.tgz, vendor/, .socket/, clone/install, and check-on-missing-ledger.

^{Reviewed by Cursor Bugbot for commit d07dda2. Configure here.}

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

Ready for review (burn-down agent).

• Head: 5d7c6e4be3
• CI: 410/410 check runs green (6 skipped, 1 neutral, 0 failing)
• Cursor Bugbot: reviewed 5d7c6e4be3, no open findings. The single earlier thread on vendor.rs is resolved and outdated.
• Reviewer focus: the new git check-ignore --no-index preflight in npm_common::stage_patch_pack and the <uuid>/.gitignore re-include written after staging.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

coverage and test-release failed on 6494750 because of main, not this PR. Two tests failed: commands::vex_consumed::tests::hosted_expands_alias_only_copies and hosted_reuses_expanded_npm_copies_and_merges_alias_variants. Both also fail on main at 4646693: #605 changed the npm resolver, and these tests still assume the old behavior. #851 has the fix. I merged main and ported #851's test change as 9334fe9. Locally, socket-patch-cli --lib passes (840 tests) and workspace clippy is clean. Once #851 lands, the ported change has no further effect.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

[agent] Head bf356dd is green: 455 checks passed, 6 skipped by matrix rule, 0 failed. Cursor Bugbot reviewed this head and posted no new findings, and no review threads are open. The approval is on this head and the PR is mergeable. It is waiting on a maintainer to merge.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Burn-down agent: labeled Ready for review at bf356dd.

• CI: 455 success + 6 skipped, 0 failing; mergeable clean against main.
• Bugbot: Cursor Bugbot check passed on bf356dd; 0 unresolved review threads.
• Linked issue(s) still open and not fixed on main.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

coverage failed on 508a598 because of main, not this PR. The failing test is utils::digest::tests::production_digests_go_through_the_helpers. It flags inline sha1/sha256 calls in crawlers/gradle_cache.rs, patch/jvm_jar.rs and patch/sidecars/maven.rs, and it fails the same way on main at 9c43dfc. #878 has the fix. I merged main and ported #878's change as d07dda2. Locally, the core lib tests pass except 4 that fail only because this sandbox runs as root. The CLI lib tests pass (847) and workspace clippy is clean. Once #878 lands, the ported change has no further effect.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

gradle 6.9.4 / jdk 11 / vendor / windows-latest failed on 508a598, but not because of this PR. In gradle_multi_project_fake_central_mirror_smoke_both_dsls (Kotlin DSL), the vendor step succeeded, then Gradle itself failed to configure :app with a java.util.concurrent.TimeoutException in its Kotlin DSL workspace cache lock. This PR doesn't change the Maven/Gradle backends. The check runs again on d07dda2, the commit that ports #878. If it fails the same way there, I'll treat it as a real failure.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

lock-diff (vlt patch compatibility) failed on d07dda2 because no tests ran, not because of a code failure. The plan, build (ubuntu-latest) and build (windows-latest) jobs were cancelled while still queued and never got a runner. That skipped native, so lock-diff found no vlt-results-* artifacts ("locks from no OS"). I've re-run the failed jobs once.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

The e2e (ubuntu-latest, e2e_gradle_discovery_build e2e_gradle_agent_build e2e_redirect_gradle_build, …) job on d07dda2 failed because its runner was shut down partway through the job, not because a test failed. The log shows "The runner has received a shutdown signal". Up to that point, every test that finished had passed (1 + 11 + 5 of 43). GitHub won't re-run failed jobs while the rest of the CI run is still going, so I'll re-run it once when the run finishes.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d07dda2. Configure here.}