Skip to content

CI: scan hosted tests assert codes, not sentences (#1089) - #1556

Merged
Mikola Lysenko (mikolalysenko) merged 6 commits into
mainfrom
agent/fix-ci-scan-hosted-envelope-asserts
Oct 15, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 6 commits into
mainfrom
agent/fix-ci-scan-hosted-envelope-asserts

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #1089 (child 5, scan_hosted slice). covgap_commands_scan_mod.rs remains for a later slice.

What changes

  • tests/covgap_commands_scan_hosted.rs (2,884 lines, 36 tests, its own test binary) moves into the scan suite binary as three modules:
    • scan/hosted_failure_paths.rs (26 tests) checks exit code, status, error.code, hosted skipped events and their errorCodes, warnings[] codes and disk state. Messages are read only for what they must name (a purl, a path, a remedy). It has 0 .contains("…") asserts of 4+ words; the old file had 20.
    • scan/hosted_render.rs (14 tests) is the human output, one check per message family (9 sentence asserts). Four of its tests are the human halves of mixed tests (takeover refusal, lock held, lock I/O, supersedes warning). They were split out so the --json file reads no stderr sentences, and each one keeps its disk-state checks.
    • scan/hosted_fixture.rs holds the shared mocks, project writers and runners. hosted_pinned is replaced by the existing common::envelope::hosted_pins.
  • The ledger-save failure now asserts error.code == "lockfile_write_failed" (the code json_error_envelope emits for a failed commit_hosted_writes) instead of the phrase "nothing was changed".
  • The lock_held test no longer compares the contention sentence. That wording is pinned by lock_cli.rs's unit tests (format_lock_error_matches_acquire_or_emit, human_error_is_capitalized_without_forced_period).
  • Fixed a stale doc on the lock_io test: it claimed the envelope keeps a redirect payload, but the test asserts there is none.
  • scripts/ci-test-durations.json folds the old target's 1.46 s into scan. Two doc pointers to the old path are updated (cli/telemetry_e2e.rs, docs/testing/bun-compatibility.md).
  • No production change.

CI cost

No workflow change. A lean PR run stays at about 28 Linux jobs, ~8 min (~110 job-min) before and after. The test shards link one fewer integration-test binary (scan absorbs it), so per-shard link time drops slightly.

Test evidence

  • cargo fmt --all -- --check: clean.
  • cargo clippy --locked --workspace --all-features -- -D warnings (the CI command): clean. cargo clippy -p socket-patch-cli --all-features --test scan -- -D warnings reports nothing in the three new files. The remaining hits in that target (prebuilt_common, hosted_yarn_classic_sha1.rs, module-loaded-twice) are already on main.
  • cargo test -p socket-patch-cli --all-features --test scan: 181 passed. The 40 hosted tests (36 moved + 4 render twins) also pass as a non-root user, so the root-skipping permission tests ran for real: ledger_save_failure_after_successful_revert_fails_closed, unreadable_bun_lockb_is_preserved_and_reports_the_io_error and unreadable_pnpm_workspace_gets_warning_only_guidance_in_a_live_run.
  • python3 -m pytest scripts/tests -q: 342 passed, 6 skipped.
  • The moved code includes cfg(unix) tests, so one full-scope ci.yml run was dispatched on c69692a to cover the Windows/macOS builds.

/code-review high

Findings fixed in 3998ee0 (and 279bf6a, which repoints the two get-suite doc comments that named the old file once #1553 had merged): the FIFO test again requires the "regular file" refusal, lock_held requires a non-empty message, the shrinkwrap needle is back to "no package-lock.json", the dry-run render checks pin the package count, the human twins check disk state, and hosted_pinned was deduplicated.
Not fixed, and why:

🤖 Generated with Claude Code

https://claude.ai/code/session_01FFRqDFywToxBViqhDGXs6d


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
covgap_commands_scan_hosted.rs was its own 2,884-line test binary
whose checks pinned English sentences (20 four-plus-word .contains
asserts), so a wording change broke tests that were about behaviour.

Its 36 tests move into the scan suite binary as three modules:

- hosted_failure_paths.rs (26 tests) checks exit code, status,
  error.code, hosted skipped events, warnings[] codes and disk state.
  Messages are read only for what they must name (a purl, a path, a
  remedy). It has no sentence asserts left.
- hosted_render.rs (14 tests) keeps the human output checks, one per
  message family. Four of them are the human halves of mixed tests,
  split out so the --json file reads no stderr sentences.
- hosted_fixture.rs holds the shared mocks, project writers and
  runners.

The ledger-save failure now asserts lockfile_write_failed instead of
the "nothing was changed" phrase. A stale doc on the lock_io test
(it claimed a redirect payload the test asserts is absent) is fixed.
No production change; CI links one fewer test binary.

Refs #1089

Assisted-by: Claude Code:claude-opus-5-5
Review of the split found checks that had become too loose:

- the FIFO bun.lockb test again requires the "regular file" refusal,
  so a generic format error can no longer pass it;
- the lock_held test requires a non-empty contention message;
- the shrinkwrap warning must still say "no package-lock.json";
- the dry-run render checks pin the package count again;
- the human takeover, lock_held and lock_io tests check the ledger,
  the lockfile and the squatting .socket file are left untouched.

hosted_pinned is dropped for the shared common::envelope::hosted_pins.

Refs #1089

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 15, 2026 01:20
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-perf CI / merge-queue performance finding (profiler routine) label Oct 15, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit c69692a. Configure here.

Resolves the ci-test-durations.json and bun-compatibility.md conflicts
with #1552 (rollback split): both old covgap targets are gone and the
doc lists the new suite files.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) force-pushed the agent/fix-ci-scan-hosted-envelope-asserts branch from 4970bd2 to a6363e7 Compare October 15, 2026 01:51
Resolves the ci-test-durations.json conflict with #1553 (get split)
and repoints the two get-suite doc comments that named the deleted
covgap_commands_scan_hosted file.

Assisted-by: Claude Code:claude-opus-5-5
Merged via the queue into main with commit 7b6371c Oct 15, 2026
53 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-ci-scan-hosted-envelope-asserts branch October 15, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-perf CI / merge-queue performance finding (profiler routine)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants