sbt, Mill and scala-cli support in agent, hosted and vendored modes - #690
Mikola Lysenko (mikolalysenko) wants to merge 96 commits into
Conversation
Introduce crawlers/jvm_cache: one list of JVM project markers, a layout-tagged cache root (Maven2 / GradleModules2 / Coursier / Ivy) that MavenCrawler crawls and resolves PURLs through, and a per-build-tool project_dependency_set provider list. Behavior is unchanged: only the Maven2 root is populated and no provider is registered yet. Gradle and sbt support each plug into this seam from their own modules. Also counts build.gradle.kts and settings.gradle(.kts) as manifest markers in scan policy, which previously listed only build.gradle. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Gradle detect/run/skip helpers, the classpath and lockfile readers, the project writer and the Windows verbatim-path strip lived inside e2e_vendor_jvm_build.rs, so every new Gradle suite would have had to copy them. They move to tests/gradle_build_common/ unchanged in behaviour, and the module gains what the agent/hosted/vendored suites need: the Gradle major/minor and JDK banner, Isolated Projects runs, per-DSL project writers, a configuration-cache-safe printRuntimeClasspath task plus an assertion on the bytes Gradle actually consumed, a test-only mirror init script for the fake origins, an autocrlf clone and per-cell probe reports. The launcher scrub now also drops GRADLE_RO_DEP_CACHE and GRADLE_HOME. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Gradle hosted suites drive the same Socket API and suffixed maven2 repository as the real-Maven hosted capstone. The wiremock Server, the API mounts and the suffixed-pom rewrite move to tests/hosted_maven_common/ behind a Hosted descriptor of the patched GAV and grant, so they can be reused for other coordinates. e2e_redirect_maven_build keeps its constants and thin wrappers and behaves exactly as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Once the crawlers learn Gradle's user home they resolve it from GRADLE_OPTS / JAVA_OPTS (-Dgradle.user.home), GRADLE_USER_HOME and ~/.gradle, plus the read-only GRADLE_RO_DEP_CACHE, and m2 from ~/.m2. Inherited as-is, a developer's warm caches would leak into every test that does not pin them. The common and prebuilt harnesses now scrub those variables by default and pin HOME / USERPROFILE to an empty stand-in (carrying version-manager roots over); a test passes a cache explicitly when it wants one, and prebuilt_common's fixture server serves explicit GRADLE_USER_HOME / GRADLE_RO_DEP_CACHE trees as maven2 repositories (with a slot for sbt's COURSIER_CACHE). Install detection learns the files-2.1/<sha1>/ layout (the jar and the pom live in different hash dirs), and fabricate_files21 lays out a Gradle cache under the real sha1 names, padded or with leading zeros dropped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Gradle 9 prints `Launcher JVM:` / `Daemon JVM:` instead of `JVM:`, so the JDK the harness logged and probed was empty on 9.x. The new SOCKET_PATCH_GRADLE_E2E_ARGS knob appends arguments to every Gradle run, which is how the compatibility grid's configuration-cache and Isolated Projects cells reach every suite without per-test plumbing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Gradle suites need artifacts real Central cannot give them on demand: a victim at two versions with Gradle module metadata and classifier jars, a transitive range consumer, a parent pom, a BOM, a platform .module that requires the victim, a buildscript-classpath library whose class prints a marker from build logic, artifact-level maven-metadata.xml, checksum sidecars, PGP signatures, and a jar whose sha1 starts with 0 (Gradle drops that zero from the files-2.1 hash dir). tests/jvm_fixture_repo/ generates all of it byte-for-byte reproducibly (stored zip entries with fixed timestamps and permissions, hand-assembled Java 8 class files, fixed-order JSON/XML, a tabulated MD5) and serves it from wiremock as FakeCentral, with overlays and a patched-jar route for the member-keyed swap. Only the signatures of a committed THROWAWAY key, the key itself and SHA256SUMS are committed; the stability self-test regenerates the repository on every OS and compares it with SHA256SUMS, and SOCKET_PATCH_JVM_FIXTURES_REGENERATE=1 re-signs it reproducibly. gradle_multi_project_fake_central_mirror_smoke_both_dsls resolves the victim through the test-only mirror init script in both DSLs under FAIL_ON_PROJECT_REPOS, through a pom range and from the settings buildscript classpath, and records the hash-dir naming in a probe report. Locally Gradle 6.9.4, 7.6.6, 8.14.3 and 9.8.0 all name the dir with the leading zero dropped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Gradle campaign lands its suites package by package, so CI needs the
rows before the tests exist, without letting an empty leg pass forever.
ci.yml: JVM legs carry `jvm_tool` (gradle | maven | sbt). One step picks
the JDK from the runner image (JAVA_HOME_<N>_X64 / _arm64), falling back
to setup-java, and decides whether Maven is needed: Maven legs, and Gradle
legs whose filter selects a Maven-seeded test (gradle_vendor_*,
multi-project); agent / hosted Gradle legs run without it. The PR tier
is the lean table: ubuntu x {6.9.4/11, 7.6.6/17, 8.14.3/21, 9.8.0/21} x
{agent + hosted, vendor + multi-project}, plus the existing windows
8.14.3 multi-project leg. A row's `suite` may list several binaries;
`allow_empty` skips suites that have not landed and tolerates zero tests,
and a Gradle leg without it that runs nothing fails. Probe reports are
uploaded.
gradle-compatibility.yml runs the full grid (3 OSes x 4 lines x 3 modes,
fail-fast off, 60 min) plus JDK-ceiling, configuration-cache, Isolated
Projects (recording only) and real-Central rows, path-filtered on PRs,
nightly and on dispatch. It compiles its own binaries once per OS and
documents the JDK ceilings per Gradle line; 9.8.0 is still current.
ci-e2e-bundle.py learns multi-suite rows, `--suites` and a per-suite
prefix guard: every #[ignore] test of a Gradle suite must start with
gradle_agent_ / gradle_hosted_ / gradle_vendor_ / gradle_multi_project,
the prefixes the rows filter on, or the bundle (and `--check`) fails.
test_ci_gradle_prefixes.py covers the guard (including a stray name) and
forces `allow_empty` off once every suite of a row has landed;
test_ci_e2e_tiers.py pins the PR table, the jvm_tool steps and the grid
expansion.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two format!-of-a-constant pom heads and a cloned single-element slice in the files-2.1 self-test; behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A developer's global commit signing or hooks would break the fixture commit, and a global autocrlf would change what the clone checks out. The helper now runs git against an empty global config, and a self-test pins the result: LF as committed, CRLF in the clone, -text files untouched. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Gradle support in agent, hosted and vendored mode needs one shared,
filesystem-free model of a Gradle build so every mode reads scripts,
versions and locks the same way and can be tested in memory on every OS.
This starts crate::gradle with the contract the later packages code
against: TextReadFn / ListFn / Env / Os.
- dsl: a comment- and string-aware Groovy/Kotlin tokenizer (copied from
the vendored planner's lexer and extended: Kotlin raw strings and
nested comments, `${}` templates with nested strings, decoded escapes,
BOM handling, strict UTF-8 decode) plus call-site parsing for both
parenthesised and Groovy command-expression calls.
- eol: CRLF sniffing, re-spelling and line-ending-blind comparison for
files a core.autocrlf clone checks out with CRLF.
- selector: Gradle's version ordering and selector scheme. Checked
against real Gradle 6.9.4, 7.6.6, 8.14.3 and 9.8.0, which showed two
behaviours changed in Gradle 7 (the special-qualifier set, and an
exclusive upper bound also rejecting qualified versions of the bound,
so `[1.9,1.10.0)` admits `1.10.0-socket.<hex>` only on 6.x), so the
comparator and admits take the Gradle major. The golden tables are
exported for the hosted script's Groovy port, and
tests/gradle_selector_golden.rs asks real Gradle for every row when
SOCKET_PATCH_GRADLE_E2E_GRADLE is set.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Discovery and agent mode must find the same Gradle caches Gradle itself uses, and hosted mode must rewrite locked versions without disturbing anything else in a lock file. - home: GradleHome::resolve over an explicit Env (the process-env adapter lives with the crawler): -Dgradle.user.home from GRADLE_OPTS then JAVA_OPTS (quote-aware per OS, last wins), a non-empty GRADLE_USER_HOME, then <home>/.gradle with USERPROFILE first on Windows; the files-2.1 cache, the read-only GRADLE_RO_DEP_CACHE copy, GRADLE_HOME, and the init-script locations (init.gradle(.kts), both init.d directories, sorted as Gradle runs them). - locks: every gradle.lockfile / buildscript- / settings- lock file and legacy gradle/dependency-locks/*.lockfile under a root (pruning build output, .gradle, node_modules, .socket and .git, eight levels deep), a parser for both formats with empty= and CRLF, and a one-entry rewrite that keeps each line's ending and configuration tail and merges into an already-locked target version. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every mode has to reason about the whole build, not only the root scripts: #461 (exclusiveContent and Android checks that only read the root build), #428 (vendoring from a subproject), #511 / #533 (range, rich, catalog and classifier declarations the root-only scan missed) and #551 (mavenLocal declared in an init script or convention plugin). ScriptGraph::collect follows, statically and with caps (8 levels of apply-from / included-build nesting, 512 files, 1 MiB each): the root settings, literal include forms with implied parents, projectDir and buildFileName overrides, each project's build script, buildSrc and literal includeBuild roots with their subprojects and precompiled convention plugins, literal apply-from targets (including rootProject.file, file(), new File(rootDir, ..) and "$rootDir/.." spellings, with a visited set), each build's libs.versions.toml and versionCatalogs files(..) catalogs, and the caller's init scripts. Anything it cannot follow (computed paths, URLs, escapes, missing, oversized or malformed files, caps) lands in `unresolved`, so callers that must fail safe can. Queries: settings_includes / project_dirs, subproject_owner for an ancestor settings file, declarations_of (string, map, Kotlin named and positional, rich version blocks, `!!`, classifier in all four forms, catalog entries with version refs), exclusive_content_filters with filter_claims_group (non-literal or uncompilable rules claim), android_or_kmp, settings_classpath_has, maven_local (Declared / NotDeclared / Undetermined), custom_lock_file and wrapper_version. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
print_cp_task's Kotlin branch read a script-level `val`, so the doLast lambda dragged the script object into the configuration cache and every --configuration-cache run failed; GRADLE_APP captured the configuration provider, which the cache serializes as a fixed file collection that `.get()` then rejects. Both now capture a task-local FileCollection. A cache reuse also skips the settings script, so the multi-project capstone only asserts the settings marker when configuration ran (configuration_reused). A new gradle_multi_project test runs the print task twice under --configuration-cache in both DSLs on Gradle >= 8.1, so the "configuration-cache safe" claim is exercised on every vendor leg. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard admitted every Gradle-campaign prefix in every suite, but each suite only runs under the prefixes its own rows select: a gradle_vendor_ test in the discovery suite, or a gradle_hosted_ test in the agent suite, passed --check and then ran in no row (or only on the ubuntu PR tier). GRADLE_SUITE_PREFIXES now maps each suite to its rows' prefixes, and a test pins that every admitted (suite, prefix) pair is selected by a ci.yml row and a gradle-compatibility.yml mode. Both workflows summed passed tests across a leg's suites, so one suite's tests hid another whose filter selected nothing, and the real-Central row (filter gradle_vendor_511/487) counted the always-landed e2e_vendor_jvm_build and failed on every run until WP3 lands. Each landed suite must now run a test on its own; compat rows take a `suites` override, and the real-Central row names only e2e_vendor_gradle_build. The SOCKET_PATCH_GRADLE_E2E_REAL_CENTRAL knob gains its reader, gradle_build_common::real_central(). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review found several shapes where the graph gave a confident answer
(mavenLocal NotDeclared, no declarations, "not owned") for builds Gradle
actually configures differently, which would drop ~/.m2 as a root or
miss the vulnerable declaration:
- `mavenLocal { content { … } }` (the Action form, the usual way to
scope it) now counts as a declaration, not just `mavenLocal(`.
- Scripts a settings script applies are parsed as settings too, spliced
in at the `apply from` as Gradle runs them, for both the graph and
subproject_owner. Inside them `file()`, nested `apply from` and catalog
`files()` resolve against the applied script's own directory while a
bare `includeBuild 'x'` stays settings-relative (measured on Gradle
6.9.4, 7.6.6 and 9.8.0).
- include / projectDir statements are applied in source order, and an
implied child is created under its parent's directory as it stands at
the include (Gradle puts 🅰️ b at modules/a/b after relocating :a).
- Binary plugin sources (.kt/.java/.groovy) of buildSrc and of plugin
projects in included builds are read as ScriptKind::PluginSource, so a
`repositories.mavenLocal()` or plugin id in a Plugin<Project> class is
seen. Product sources of ordinary included builds are not read.
- android_or_kmp also reads catalog `[plugins]` ids, which is the only
place the id appears for `alias(libs.plugins.android.application)`.
- ScriptGraph::lockfile_paths / locks::lockfile_paths_in list only the
lock files of the build's own projects, so a hosted rewrite cannot
touch a nested sample or fixture build the checkout does not include.
locks::lockfile_paths stays as the whole-tree inventory.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On Unix the JVM's user.home is the passwd entry's home, not $HOME, and Gradle derives its user home from it; in container CI jobs the two differ (HOME=/github/home, pw_dir=/root), so the CLI would scan and patch a cache Gradle never reads. GradleHome::resolve now prefers the caller's home_dir (which must be the passwd home) over $HOME on Unix. A wrapper build runs the init.d of the distribution it unpacked under <user home>/wrapper/dists, not $GRADLE_HOME's, and custom corporate distributions ship mavenLocal/mirror scripts there. init_scripts_with now includes every unpacked wrapper distribution's init.d, and init_scripts_for / wrapper_init_dirs narrow that to the build's distributionUrl. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Disassembling VersionRangeSelector / DefaultVersionSelectorScheme (the same on 6.9.4, 7.6.6 and 9.8.0) showed a range bound cannot hold whitespace, the single-value form allows whitespace only after its `[`, and both `[a]` and `[a,a]` become ExactVersionSelector (string equality). The port's lazy bound pattern accepted `[ 1.1 ]` and inner spaces, and `[a,a]` compared instead of matching exactly. New golden rows pin each case, including `[1.01]` NOT admitting `1.1`, and pass against real Gradle on all four majors. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s, home, eol, selectors) Brings in the pure, I/O-light Gradle layer (DSL scanning, script graph, lockfiles, Gradle home resolution, EOL handling, version selectors) that the later work packages build on. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ted fixtures, CI Gradle tiers, prefix guard) WP5a lands the shared real-Gradle test harness, deterministic fake Maven Central and hosted-API fixtures, the ci.yml jvm_tool scaffold with lean Gradle rows, and the path-filtered gradle-compatibility.yml grid that the later mode packages build on. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
aca8b1c routed every cache root through a per-layout match whose GradleModules2 arms were empty, so `scan --global-prefix ~/.gradle/caches/modules-2/files-2.1` found nothing (before, the plain .pom walk happened to read the poms there). Add crawlers/gradle_cache with the files-2.1 layout: a walk over exactly three literal levels (group keeps its dots, then artifact, version) and the 1-40 hex digit hash dirs below them, skipping bookkeeping and unsafe coordinates. Each version dir with `<a>-<v>.{jar,pom,module}` in some hash dir is one package whose path is the version dir; find_by_purls resolves the same dirs. hash_eq / pristine compare hash dir names as 40-digit numbers, since some Gradle releases drop the sha1's leading zeros. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Gradle package path is the version dir, but its files live one level down, each in the hash dir its sha1 names, and the same jar can sit in two hash dirs after a re-download. Every join site (apply, rollback, verify, VEX, select_installed_variants) needs the real file locations. Add gradle_cache::installed_copies, the one layout-agnostic resolver: a Gradle version dir maps each key's file name to every hash dir holding it (keys found nowhere stay on the version dir so they verify as not found); any other path is returned unchanged. installed_copies_detailed reports the missing keys apart. jvm_cache::locate_artifact lists every copy of one artifact file per cache layout, and gradle_cache::stale_derived_copies finds the instrumented and transformed copies Gradle keeps outside files-2.1, for agent mode to refuse on. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Gradle build resolves from its user home's files-2.1 (and the read-only GRADLE_RO_DEP_CACHE), not ~/.m2, unless something declares mavenLocal(). Scanning only m2 missed every Gradle-cached package (#349) and reported m2 contents a Gradle-only build never uses (#551). Cache roots now come from a JvmEnv (process env by default, an explicit Env in tests): the Gradle user home resolved like Gradle does it (the gradle.user.home property, GRADLE_USER_HOME, then the passwd home on Unix) and the Maven local repository. A Gradle build, or a global scan, crawls files-2.1 and the read-only cache. m2 stays a scan root for a pom.xml, a non-Gradle cwd, a global scan, or a Gradle build where the script graph plus the init scripts that apply (user home, GRADLE_HOME, and the wrapper's own distribution, wherever distributionBase/Path unpack it) declare mavenLocal() or cannot rule it out. A custom wrapper distribution that is not unpacked yet, an unreadable init script or an unfollowable script reference keeps m2 (undetermined). PURL lookups keep m2 regardless, since its bytes still serve vendoring and apply. --global-prefix accepts a Gradle user home, caches/modules-2 or a read-only modules-2 for the files-2.1 inside them; a Maven repository named `caches` is left alone. jvm_cache gains all_local_roots for byte sourcing, gradle_cache the fs/env adapters (fs_text_read, fs_list, home_from_process_env, init-script reads) the other Gradle packages share, and locked_gavs for the lock-membership annotation. The CLI test harness now also pins GRADLE_USER_HOME to the stand-in home: with Gradle's home taken from the passwd entry, pinning HOME alone would let a developer's real ~/.gradle into every test. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
scan now says when the Gradle side of discovery is not what the user may expect, on the run-level warnings[] channel: - gradle_build_ignores_m2: a Gradle-only build declares no mavenLocal(), and modules its locks or patch records name exist only in ~/.m2, which the build never resolves from and the scan leaves out (#551). - gradle_maven_local_undetermined: m2 stays a root because a script or init script could not be read literally. - gradle_user_home_differs: Gradle's home follows the passwd entry, not $HOME. Each Gradle-cached package in packages[] carries an additive inLock flag from the build's graph-scoped lock files. It only annotates: an unlocked buildscript or plugin dependency is still reported. e2e_gradle_discovery_build runs the real binary against fabricated caches everywhere, and gradle_agent_349_scan_finds_gradle_cache lets real Gradle fill a fresh user home from the fake Central, then checks the scan reports the module and that the crawled version dir expands to the hash dir whose jar Gradle consumed. On 6.9.4, 7.6.6, 8.14.3 and 9.8.0 that dir drops the jar sha1's leading zero. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
get_maven_repo_paths now lists the Maven local repository ahead of the Gradle caches. Callers that still take the first copy (agent apply's Maven arm, hosted VEX copies) then keep resolving where they always did; the Gradle copies remain reachable for the all-copies fan-out that follows. e2e_maven's Gradle-marker scan now declares mavenLocal(): a Gradle-only build without it does not read ~/.m2, so the scan no longer counts the m2 artifact for it (#551). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
stale_derived_copies counted any file named after the jar as stale and stopped silently after 200,000 entries. A copy Gradle rebuilt from the patched jar has the same name, so the warning could never clear, and a truncated walk looked exactly like "no stale copies". It now returns DerivedCopies: copies proven pristine-derived (identical bytes or the pristine sha1 in a dir or stem) are stale, other same-named files are unknown, and an entry cap or unreadable entry sets incomplete. The walk is sorted so it is deterministic. gradle-wrapper.properties is now read the way java.util.Properties reads it: ISO-8859-1, whitespace as a separator, escapes and continuations. A wrapper file that names no distribution, or cannot be read, marks mavenLocal() undetermined instead of falling back to "no wrapper", so a custom distribution's init.d cannot drop ~/.m2 silently. get_maven_repo_paths no longer returns Gradle files-2.1 roots. Its callers (apply, rollback, vendor, VEX) join file keys onto the package path. A Gradle version dir holds no files directly, so a Gradle-only GAV failed with NotFound instead of being skipped as not installed. The Gradle roots move to the new get_maven_copy_paths for callers that expand version dirs through installed_copies. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The vendor baseline pre-check ran verify_file_patch on the crawled package path, and for a Gradle version dir that always returns NotFound. It now goes through installed_copies, so any hash-dir copy that differs from the baseline flags the patch. inLock is now written only when the cwd's Gradle locks were actually read. A global run inside a Gradle build reads them too, and a run outside one has no inLock at all, where it used to report false for every package. gradle_user_home_differs is skipped under --global-prefix, because the user home is not used then. The Gradle warning codes now carry a level in the JSON warnings[]: info for gradle_maven_local_undetermined and gradle_user_home_differs, warn for gradle_build_ignores_m2. Human mode prints the info ones as "Note:" and leaves them out under --silent. The real-Gradle capstone now takes the package path from the crawler, over the roots this build scans, and expands that path rather than one it built itself. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Agent mode could only patch files that sit directly in a version directory, so member-keyed Maven records (#264) were unpatchable and a Gradle files-2.1 version dir verified nothing. patch/jvm_jar classifies a record as leaf- or member-keyed, verifies jar members against an explicit jar name (so hosted copies check their suffixed jar), swaps in the patch service's build of the whole jar after checking every unpatched member is upstream's, and keeps the original under .socket/jvm-originals/ where blob cleanup never looks. Rollback restores that backup byte for byte, or re-downloads a Gradle copy's jar and accepts it only when it hashes to the copy's hash directory. Variant selection, verify_patch_record, judge_installed_record and the hosted copy check now expand Gradle version dirs through installed_copies, and VEX takes every installed copy of a Maven purl and reports the ones that do not verify. A Gradle cache copy no longer counts against a vendored entry, since the vendored build never reads it. Maven ~/.m2 .sha1/.md5 files are rewritten only when they matched the pre-patch bytes, and Gradle cache writes carry Info advisories about refreshes, daemons and the shared user home. The registry fetch helpers in vendor/maven_repo.rs become pub(crate) for the upstream fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vendored Gradle reported success on builds it did not actually patch: a pom.xml next to a Gradle build skipped the Gradle side (#395), a subproject was wired as its own root (#428), an autocrlf checkout failed --check and left the script behind on revert (#429), exclusiveContent and Android checks only looked at the root build script (#461), pgp-only verification entries broke the build (#487), ranges downgraded to an unpatched release because the tree listed no versions (#511), and a declared classifier stopped resolving (#533). - detect() reports every build of the root (Detected{maven, gradle}); a mixed root is planned through both planners in one transaction, and a refusal of either writes nothing. Revert, --check, VEX liveness and repair run both halves. - not_build_root refuses vendor and repair from a Gradle subproject, from includes that cannot be read, and from a project configured by an ancestor settings file. - Owned text (script, index, .gitattributes, derived metadata, .mvn/maven.config) is compared line-ending blind; new owned .socket/gradle/.gitattributes and .socket/vendor/.gitattributes keep them out of EOL conversion. A vendor-created settings file is deleted once only whitespace is left. - The planner builds crate::gradle::graph::ScriptGraph and refuses a conflicting exclusiveContent or an Android/KMP plugin anywhere it can follow (subprojects, convention plugins, apply from, catalogs), naming the file; what it cannot follow is degraded. - A pgp-only metadata entry gets a sha256 beside its <pgp>; --check and the parent-chain warning require a checksum when metadata verification is on. - Each vendored GA gets a derived maven-metadata.xml (Gradle version order, no lastUpdated), recomputed on revert and deleted with the GA's last row. A range is noted; one admitting no vendored version refuses. - JvmPatch.extra_artifacts serves declared classifiers (and sources when found) from the tree; a declared one that cannot be sourced refuses. - Upstream files come from the crawler's directory and every local JVM cache (jvm_cache::locate_artifact over all_local_roots), Gradle copies authenticated by their hash directory. - The settings helpers WP4 needs are pub(crate) and parameterized by WiringTarget; the vendored defaults keep the output byte-identical. The lexer is crate::gradle::dsl. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Agent mode patched only the first copy of a Maven purl, and Gradle version directories were joined like ~/.m2 ones, so a GAV cached by Gradle stayed vulnerable while apply reported success (#551). The Maven lookup now returns every cache holding a copy (get_maven_copy_paths), and a JvmScope sorts them into the copies a build consumes, the read-only cache, and an ~/.m2 a Gradle-only build never reads. apply patches each consumed copy, expanding Gradle version dirs into the hash dirs holding the record's files, and swaps the whole jar for member-keyed records (#264). Each Gradle hazard has its own code: gradle_verification_metadata_present refuses with nothing written, gradle_build_ignores_m2 fails an ~/.m2-only GAV, gradle_ro_cache_shadows fails a run with a read-only copy, gradle_copy_unexpected_bytes leaves a pristine download of other bytes alone, and gradle_transform_copy_stale fails a copy whose derived transforms still hold the pristine jar. rollback groups by (base purl, copy), restores each hash dir and checks the restored bytes hash to their directory (gradle_rollback_hash_mismatch otherwise), restores whole jars from their backups, and puts ~/.m2 checksum files back. get narrows release variants over every copy. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pre-write pristine check is read-only, so it now runs on --dry-run too and the preview shows the gradle_rollback_hash_mismatch refusal the wet run gives. It also only reads a before-blob named by a valid 64-hex hash that is a regular file (not a symlink), the same guards verify_file_rollback applies; anything else is left to the engine, which refuses it without reading through it. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018KHkK9FFA1HRxPF21XndGW
|
bugbot run Generated by Claude Code |
Hosted confirmation already treats both as unverified; standalone vex now withholds the lockfile basis for them too, so it cannot attest a pin whose repository was dropped or whose lock would reject the suffix. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018KHkK9FFA1HRxPF21XndGW
|
bugbot run Generated by Claude Code |
|
Burn-down agent: labeled Ready for review.
Generated by Claude Code |
Release notes are written when a release is cut, from the merged PR log and the code, so PRs no longer edit CHANGELOG.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Release notes are written when a release is cut, from the merged PR log and the code, so PRs no longer edit CHANGELOG.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Resolve conflicts by keeping both sides: Gradle takeover refusals alongside main's requirements.txt takeover preflight, the eject snapshot's new written-only restore with the Gradle wiring files added to its planned scope (and parent dirs recreated on put-back), the artifact_metadata rename next to gradle_unreadable, and both contract additions. Co-Authored-By: Claude <noreply@anthropic.com>
Bring in the main merge on the base branch. Resolve by keeping both sides: the npm-manifest filter next to sbt's synthetic-key filter, the shared VENDOR_TREES list now also naming the Coursier tree (so the wiring check and the vendor-dir symlink guard cover it), sbt modules next to the requirements.txt preflight export, and the eject snapshot's new written-only restore with the Coursier captured files in its planned scope, still read FIFO-safely; both eject snapshot test sets are kept. Co-Authored-By: Claude <noreply@anthropic.com>
|
bugbot run Generated by Claude Code |
The merge in 46c9d4b took this branch's stale vex_consumed tests over feat/gradle-support's #605 updates: since #605 the name-keyed resolver already returns npm aliases and nested-store peers, so the old assertions that it returns none failed. Restore the gradle-side tests, keeping this branch's sbt hosted repo lookup in maven_copies. #664 (via main) refuses a linked `.socket/vendor` up front for every ecosystem with `vendor_dir_symlink_unsupported`, before the sbt shape check runs; expect that code for the linked vendor dir in the sbt vendor e2e, as the Maven CLI test already does. Co-Authored-By: Claude <noreply@anthropic.com>
|
bugbot run Generated by Claude Code |
# Conflicts: # .gitattributes # .github/workflows/ci.yml # crates/socket-patch-cli/CLI_CONTRACT.md # crates/socket-patch-cli/src/commands/apply.rs # crates/socket-patch-cli/src/commands/rollback.rs # crates/socket-patch-cli/src/commands/scan/hosted.rs # crates/socket-patch-cli/src/commands/vendor.rs # crates/socket-patch-cli/src/commands/vex.rs # crates/socket-patch-cli/src/commands/vex_consumed.rs # crates/socket-patch-cli/src/ecosystem_dispatch.rs # crates/socket-patch-cli/tests/common/jvm_env.rs # crates/socket-patch-cli/tests/common/mod.rs # crates/socket-patch-cli/tests/e2e_gradle_agent_build.rs # crates/socket-patch-cli/tests/e2e_gradle_discovery_build.rs # crates/socket-patch-cli/tests/e2e_redirect_gradle_build.rs # crates/socket-patch-cli/tests/e2e_vendor_gradle_build.rs # crates/socket-patch-cli/tests/gradle_agent_cli.rs # crates/socket-patch-cli/tests/maven_sidecar_cli.rs # crates/socket-patch-core/src/crawlers/gradle_cache.rs # crates/socket-patch-core/src/crawlers/jvm_cache.rs # crates/socket-patch-core/src/crawlers/maven_crawler.rs # crates/socket-patch-core/src/crawlers/mod.rs # crates/socket-patch-core/src/formats/registry.rs # crates/socket-patch-core/src/hosted/engine.rs # crates/socket-patch-core/src/patch/jvm_jar.rs # crates/socket-patch-core/src/patch/redirect/mod.rs # crates/socket-patch-core/src/patch/redirect/upstream/mod.rs # crates/socket-patch-core/src/patch/sidecars/mod.rs # crates/socket-patch-core/src/vendor/jvm/apply.rs # crates/socket-patch-core/src/vendor/jvm/mod.rs # crates/socket-patch-core/src/vendor/maven_repo.rs # crates/socket-patch-core/src/vex/discover/mod.rs # crates/socket-patch-core/tests/crawler_gradle_e2e.rs # crates/socket-patch-core/tests/crawler_maven_e2e.rs # docs/design/maven-vendoring.md # docs/ecosystems.md # docs/testing/README.md # scripts/tests/test_ci_e2e_tiers.py
|
bugbot run Generated by Claude Code |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Maven scans Coursier and Ivy
- Restricted Coursier/Ivy cache crawling to scala-tool projects only, preventing Maven and Gradle builds from scanning them
- ✅ Fixed: Unreadable vendor file can be replaced
- Added filesystem checks to detect and adopt unreadable vendored files instead of replacing them
Or push these changes by commenting:
@cursor push 14924aee3d
Preview (14924aee3d)
diff --git a/crates/socket-patch-core/src/crawlers/maven_crawler.rs b/crates/socket-patch-core/src/crawlers/maven_crawler.rs
--- a/crates/socket-patch-core/src/crawlers/maven_crawler.rs
+++ b/crates/socket-patch-core/src/crawlers/maven_crawler.rs
@@ -872,10 +872,10 @@
}
// sbt / Mill / scala-cli: Coursier's per-repository roots, the Ivy
// caches, then the caches local sbt evidence points into (the
- // directory walks are blocking). The first root holding a PURL wins
- // the crawl dedup. Locally only for a Scala-tool project: a Maven or
+ // directory walks are blocking). Locally only for a Scala-tool project: a Maven or
// Gradle build never reads those caches.
- if !options.global && !scala_tool_project(&options.cwd).await {
+ let is_scala_tool = scala_tool_project(&options.cwd).await;
+ if !is_scala_tool {
return roots;
}
let cwd = options.cwd.clone();
diff --git a/crates/socket-patch-core/src/vendor/jvm/coursier_tree.rs b/crates/socket-patch-core/src/vendor/jvm/coursier_tree.rs
--- a/crates/socket-patch-core/src/vendor/jvm/coursier_tree.rs
+++ b/crates/socket-patch-core/src/vendor/jvm/coursier_tree.rs
@@ -160,8 +160,13 @@
bytes: render_index_like(&all, crlf).into_bytes(),
tree: false,
});
+ // ReadFn returns None for both missing AND unreadable files. Check if
+ // the index exists on disk; if it does but was unreadable, adopt it
+ // rather than replacing it (matching the owned_file_with guard).
+ let index_exists_on_disk = existing_index.is_some()
+ || std::path::Path::new(INDEX_REL).exists();
Ok(vec
This adds sbt, Mill and scala-cli support for
pkg:mavenpatches in agent, hosted and vendored modes. It is stacked on #646 (Gradle), so review and merge #646 first; this PR's diff is againstfeat/gradle-support.In every mode the user's own build files (
build.sbtand friends) are left untouched. socket-patch only writes files it owns.How each mode works
useCoursier := false). Coursier's checksum sidecars are resynced.socket-patch.sbtsocket-patch-vendor.sbtand a committed.socket/vendor/maven2treesocket-patch.scalaplus a same-GAV Coursier tree (directory builds)Hosted and vendored sbt. Both modes use a generated file with these properties, all verified by probe on every sbt line:
inThisBuild, so it reaches every subproject and transitive dependency, and never adds a dependency.<v>-socket.<hex8>version withdependencyOverrides. This is needed because Ivy and Coursier both rank the plain release above the suffix.Scoping. sbt has no lockfile, and
dependencyOverrideswould downgrade a project that resolves a newer version. Hosted and vendored modes therefore pin a package only when sbt's own resolution records undertarget/show the project resolves exactly the patched base version.Background:
docs/design/sbt-support.md,docs/design/sbt-template-probe.mdanddocs/design/sbt-evidence-probe.md. New codes are listed inCLI_CONTRACT.md.Test coverage
These are the real-tool docker runs (
scripts/sbt-compat-matrix.sh):useCoursier := falseBefore this rebase, the full matrix also passed with JDK 21 legs for 1.13.0 and 2.0.9.
The hermetic suites (
e2e_sbt,e2e_sbt_hosted,e2e_sbt_vendor,e2e_scala_cli_vendor), the golden harnessredirect_sbt_goldenand the vendored VEX cells all pass.CI changes:
sbt-compatibility.yml: a docker matrix, scala-tools legs, and native macOS and Windows legs.ci.yml.Known limits
installed_copies.mode_migration_npm::berry_vendored_then_hosted_takeover_leaves_pure_hostedfails locally on main as well. Fix yarn berry hosted pin leaking npm auth (#404) #465 changed the hosted Berry pin and the test was never updated. It only runs when real yarn is installed.🤖 Generated with Claude Code
Note
Medium Risk
Large new JVM integration touching apply, hosted redirect, vendoring, and VEX across multiple cache layouts; risk is mitigated by extensive CI matrices but native OS legs and offline resolution gating add operational complexity.
Overview
Adds sbt, Mill, and scala-cli as first-class JVM build targets for
pkg:mavenpatches across agent (in-place patches in Coursier/Ivy caches with checksum sidecar resync), hosted (generatedsocket-patch.sbtwith resolution-evidence gating), and vendored (socket-patch-vendor.sbt/ scala-cli Coursier tree) modes—without editing userbuild.sbtfiles.Agent / rollback / VEX paths are extended so Maven apply treats Coursier and Ivy cache copies like Gradle’s expanded dirs (
gradle_cache::expands), fixes Gradle-onlymavenLocal()warnings when only~/.m2is patched, and crawls hosted sbt download dirs for VEX. Vendor/eject runs sbt/scala-cli preflight gates before hosted takeover, tightens commit/reinstall hints for generated root files, and hardens eject snapshots (FIFO-safe reads,eject_incompletebackstop).CI and testing: new
sbt-compatibility.yml(Docker matrix across sbt versions/JDKs × agent/hosted/vendored, Mill/scala-cli, macOS/Windows native legs),sbtecosystem incoverage-docker/e2e-dockerwith version filters and longer timeouts, e2e ubuntu legs for hosted/vendored 1.13.0, warm-seed scripts,docker_e2e_sbt, hermetice2e_sbt/ vendored VEX cells, and.gitattributes/.gitignorefor byte-exact sbt fixtures. Docs/README/CLI contract document hosted sbt rules, resolution evidence, and new warning codes.Reviewed by Cursor Bugbot for commit d836a59. Configure here.
Generated by Claude Code