Skip to content

CI: vendor suite asserts codes, not sentences (#1089) - #1544

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-vendor-envelope-asserts
Oct 14, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
agent/fix-ci-vendor-envelope-asserts

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #1089 (vendor slice of child 2; the tracking issue stays open for children 3-6).

Summary

covgap_commands_vendor.rs pinned English sentences in its --json tests. A copy edit to a vendor message broke them even when the exit code, error code, events and bytes on disk were unchanged. This PR moves the file into the vendor suite binary, split three ways:

  • vendor/vendor_failure_paths.rs: the --json tests, asserting codes, events and disk state. A message is read only for what it must name: the lockfile, the git checkout remedy, the ecosystem, the link name, or the --offline flag. 4+-word .contains sentence asserts: 0.
  • vendor/vendor_render.rs: the human-mode render checks, one per message family. 25 sentence asserts, all in this render suite as Tracking: assert stable codes instead of human sentences in CLI tests, and fold the covgap suites #1089's target design allows.
  • vendor/npm_fixture.rs: the fixture and runners both files share.

The --json tests also got these fixes:

  • Skipped and --check events carry their message under reason, not error. The tests now read the right field, and the yarn --check test finds the event by its code instead of stringifying it.
  • The tests use names and docstrings that match what they assert. For example, the corrupt-artifact runs fail the redownload on the --offline conflict and carry no repair hint, and the legacy-ledger and no-lockfile runs exit 1 rather than skip.
  • patch_target_absent_from_the_install_still_vendors (formerly human_patch_failure_prints_failed_to_vendor, which checked no output at all) now asserts that the service-built tarball carries the patched package/absent.js.
  • The three read-only-directory tests skip under root, as repair_vendor_failure_paths.rs does. Before, they failed in any root container, because chmod 0o555 can't stop root writing.

coverage_fix_vendor_silent_mute_exit.rs stays where it is. It has no sentence asserts, and #1089 puts it in child 3 with its apply twin.

CI cost

Test-only change. A lean PR run keeps the same ~28 Linux jobs and ~8 min. There is one fewer integration test binary to link in the test shards, and scripts/ci-test-durations.json folds the old target's 3.75 s into vendor (7.77 → 11.52 s) so the Windows shard balance holds.

Test evidence

  • cargo test -p socket-patch-cli --all-features --test vendor: 148 passed. As root, the three chmod tests skip; run as nobody, they pass (3/3).
  • cargo test -p socket-patch-cli --all-features --test cli: 113 passed, including the envelope_helper_copies ratchet.
  • cargo fmt --all -- --check: clean.
  • python3 -m pytest scripts/tests -q: 342 passed, 6 skipped.
  • cargo clippy -p socket-patch-cli --all-features --test vendor: 26 warnings, against 35 across the two old targets, and none of a new kind. CI's clippy step doesn't lint test targets.
  • /code-review high raised 10 findings and 9 are fixed. Not fixed: switching the runners to common::envelope::parse_json_envelope. These runners put the child's stderr in the panic message, and the shared helper takes stdout only.

🤖 Generated with Claude Code


Note

Low Risk
Test-only refactor and assertion updates; no production vendor CLI behavior changes in this diff.

Overview
Refactors the vendor integration tests (#1089) so --json contracts (exit codes, error codes, events, on-disk state) live separately from human CLI output checks.

Shared npm fixture/runners move to npm_fixture.rs. vendor_failure_paths.rs drops long English .contains asserts in favor of structured envelope fields (reason vs error, specific event codes). Several scenarios are retargeted to current behavior (e.g. corrupt committed artifacts → vendor_redownload_failed / --offline conflict; legacy ledger and missing lockfile → exit 1, not calm skips). Adds patch_target_absent_from_the_install_still_vendors (service tarball includes patched absent file). Unix fault-injection tests skip as root.

Human summary/stderr tests move to vendor_render.rs. The vendor test binary in main.rs wires the new modules; scripts/ci-test-durations.json removes covgap_commands_vendor and bumps vendor duration (~7.77s → ~11.52s).

Reviewed by Cursor Bugbot for commit cdd3949. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
The vendor covgap tests pinned English sentences in --json mode, so a
copy edit to a vendor message broke them even when the exit code,
error code, events and bytes on disk were unchanged.

The --json tests now read the takeover, symlink and revert messages
only for what they must name (the lockfile, the git checkout remedy,
the ecosystem), and the patch-target test checks its applied event
instead of human output. The human-only twin of the stale-ledger test
is dropped; its --json twin covers the same path.

The file moves into the vendor suite binary, one fewer test binary to
link: vendor_failure_paths.rs (--json), vendor_render.rs (human
output) and npm_fixture.rs (their shared fixture). The CI shard
timings fold its time into the vendor target.

Refs #1089.

Assisted-by: Claude Code:claude-opus-5-5
The read-only-directory tests skip themselves under root, as the
repair suite's do: chmod cannot stop root writing, so they failed the
whole vendor binary in root containers.

Test names and docs now say what the tests check. The corrupt
artifact runs fail the redownload on the --offline conflict (there is
no repair hint), the legacy-ledger and no-lockfile runs fail rather
than skip, and the missing-patch-target run checks that the service's
artifact carries the patched file. The yarn check reads the failed
event by its code, not by stringifying it. The human stale-ledger
render check is back.

Refs #1089.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 14, 2026 19:13
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit cdd3949. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-perf CI / merge-queue performance finding (profiler routine)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants