Fix gem pair model ignoring custom lockfile and Bundler 1 twins (#749, #751)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #749
Fixes #751

Summary

Hosted gem mode wired the wrong files in two Bundler layouts. In both, the scan reported success (and --vex attested the patch) while Bundler installed the unpatched gem or every frozen install failed. Both now wire the pair Bundler actually loads, or refuse before writing anything.

Root cause

formats::gem::manifest::LoadedManifest is the single model of "which manifest/lock pair does Bundler load", and hosted mode (keep_bundler_loaded_gem_files) and vendored mode (gem_manifest_refusal) both rely on it. It modelled BUNDLE_GEMFILE, but:

• Hosted gem redirect ignores Bundler 4's custom lockfile (lockfile setting / BUNDLE_LOCKFILE), so it never pins the lock Bundler uses and frozen installs fail with no warning #749: it derived the lock only from the manifest name, so it never saw Bundler 4's custom lockfile (BUNDLE_LOCKFILE env, or lockfile in .bundle/config). The lock Bundler reads was never pinned. A leftover Gemfile.lock was rewritten instead, even though Bundler ignores it.
• Hosted gem redirect wires gems.rb in a Gemfile/gems.rb twin locked by Bundler 1.17, which loads Gemfile, so the install stays unpatched while the in-run VEX attests it #751: default discovery for a Gemfile + gems.rb twin always followed Bundler ≥ 2 (gems.rb first). Bundler 1.x loads Gemfile first.

Fix

• LoadedManifest::with_lockfile resolves the configured lockfile the way Bundler::CLI does (env first, then app config; BUNDLE_IGNORE_CONFIG honoured; relative to the project root). If it names the loaded pair's own lock, nothing changes. Anything else is the new UnsupportedLockfile:
  • hosted mode refuses with redirect_gem_bundle_lockfile_unsupported (nothing written, nothing attested);
  • vendored mode refuses with gemfile_not_loaded.
• default_twin_manifest reads both twin locks' BUNDLED WITH (new shared formats::gem::bundled_with_major):
  • every recorded major is 1.x → wire Gemfile + Gemfile.lock;
  • none is 1.x → gems.rb (unchanged);
  • the locks disagree → refuse with redirect_gem_twin_bundler_versions_diverge.
• Docs: CLI_CONTRACT.md (new additive warning codes), docs/ecosystems.md (RubyGems row), CHANGELOG.

Tests (red → green)

Issue	Test	Without fix	With fix
#749	hosted_memory_engine::a_bundler4_custom_lockfile_is_refused	FAILED (rewrote the leftover lock)	ok
#749	e2e_redirect_gem_build::gem_hosted_bundler4_custom_lockfile_redirects_nothing (real Bundler 4.0.17)	FAILED: redirected: 1, rewrittenFiles: [Gemfile, Gemfile.lock], vex statements: 1	ok; frozen bundle install of the untouched project succeeds
#749	ruby_crawler::loaded_manifest_reads_the_lockfile_setting (disk, no leftover lock; env vs config priority; BUNDLE_IGNORE_CONFIG)	new API	ok
#749	vendor::gem::a_bundler4_custom_lockfile_is_refused	new	ok
#749	hosted_memory_engine::a_lockfile_setting_naming_the_default_lock_is_wired (control)	ok	ok
#751	hosted_memory_engine::a_bundler1_twin_wires_the_gemfile_pair	FAILED (wired gems.rb)	ok
#751	hosted_memory_engine::a_twin_with_diverging_bundler_majors_is_refused	FAILED	ok
#751	hosted_memory_engine::a_bundler2_twin_still_wires_gems_rb (control)	ok	ok
#751	e2e_redirect_gem_build::gem_hosted_bundler1_twin_wires_the_gemfile_and_installs	runs on the CI bundler 1.17.3 leg; skips on ≥ 2	skipped locally (Bundler 4)
both	manifest.rs unit tests (with_lockfile_*, default_twin_manifest_*, config_lockfile_*), bundled_with_major_reads_the_version_line	new	ok

Local results

• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo fmt --all -- --check is not clean on main itself (≈500 diffs, and CI has no fmt gate), so I formatted only the hunks I touched.
• cargo test --workspace --all-features --lib --bins: the core lib has 4848 passed and 4 failed. All four (copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_*, pypi_requirements::wire_failure_rolls_back_*) are chmod-based write-failure tests that can't fail writes when run as root (uid 0) in this sandbox. They don't touch gem code.
• Integration tests: hosted_memory_engine 33/33, hosted_memory_parity, in_process_vendor and e2e_redirect_gem_stale_install all ok. in_process_redirect has 104 passed and 3 failed, again only the chmod-based write-failure tests (root sandbox).
• e2e_redirect_gem_build --ignored (real Bundler 4.0.17): new custom-lockfile, dual-boot and gems.rb arms pass.
• I couldn't run the full cargo test --workspace locally because the sandbox's disk allowance runs out building every integration-test binary. CI runs the full suite.

Notes / follow-ups

• PR Fix gem lock readers ignoring gems.locked (#736) #750 (Lock inventory reads only Gemfile.lock, so a gems.rb project's gems.locked is invisible and a stale Gemfile.lock is read instead #736) routes the lock readers (VEX discovery, lock inventory) through LoadedManifest::pair. Once both land, a Bundler 1.x twin's post-install VEX should use default_twin_manifest too. This PR doesn't touch those readers, to avoid conflicting with Fix gem lock readers ignoring gems.locked (#736) #750.
• The Gemfile lockfile "custom.lock" DSL (issue matrix row 4) is Ruby code the model can't read. Bundler 4.0.17 itself fails frozen installs on that shape before any scan.
• A BUNDLE_LOCKFILE setting is refused even on Bundler < 4 (which ignores it). That's a conservative, fail-closed choice.

---

Note

Medium Risk
Changes which gem manifest/lock files get rewritten in hosted and vendored modes; incorrect behavior previously caused silent unpatched installs or broken frozen bundle install, but the fix still touches lockfile mutation paths.

Overview
Gem hosted/vendored wiring now matches Bundler’s real manifest and lock pair, fixing false success and bad VEX when Bundler 4 uses a custom lockfile (#749) or when a Gemfile/gems.rb twin was last locked with Bundler 1.x (#751).

LoadedManifest gains with_lockfile (env/config BUNDLE_LOCKFILE, same priority as Bundler). If the configured lock is not the pair’s default, the run is refused before any write with redirect_gem_bundle_lockfile_unsupported (hosted) or gemfile_not_loaded (vendored), instead of rewriting a leftover Gemfile.lock Bundler ignores.

For Gemfile + gems.rb twins, default_twin_manifest uses each lock’s BUNDLED WITH major: all 1.x → wire Gemfile/Gemfile.lock; otherwise keep Bundler ≥2’s gems.rb path. Conflicting majors refuse with redirect_gem_twin_bundler_versions_diverge. The hosted engine’s candidate filtering and ruby_crawler::bundler_loaded_manifest (BundlerEnv) apply the same rules.

Docs and tests (memory engine, e2e with real Bundler, manifest unit tests) cover custom lock refusal, Bundler 1 twin wiring, and divergent-twin refusal.

^{Reviewed by Cursor Bugbot for commit 731f489. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 731f489. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Ready for review at 731f489.

• CI: 402/402 non-skipped checks green on the current head (9/9 workflows, path-filtered).
• Bugbot: reviewed 731f489 and found no issues. There are no unresolved review threads.
• Mergeable: yes, no conflicts with main. The diff is 10 files, all in the gem lane, plus docs and changelog.
• For the reviewer: look at the LoadedManifest pair model in formats/gem/manifest.rs (the configured BUNDLE_LOCKFILE and the BUNDLED WITH major of a Gemfile/gems.rb twin) and the matching ruby_crawler.rs changes.

---

Generated by Claude Code