Skip to content

CI: get tests assert codes, not sentences (#1089) - #1553

Merged
Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
agent/fix-ci-get-envelope-asserts
Oct 15, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
agent/fix-ci-get-envelope-asserts

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #1089 (child 5, get slice). get --json already emits a {code, message} error, so this slice is unblocked. covgap_commands_scan_mod.rs and covgap_commands_scan_hosted.rs remain for later slices.

What changes

  • tests/covgap_commands_get.rs (3,424 lines, 65 tests, its own binary) is split into the get suite binary:
    • get/get_failure_paths.rs (44 tests) checks exit code, status, error.code, event errorCodes, warnings[] codes and disk state through common::envelope. It has 0 .contains("…") asserts of 4+ words; the old file had 22.
    • get/get_render.rs (29 tests) holds the human and --silent output checks, one per message family. This includes the human halves of the old both-modes tests (blob-write failure, noMatch, vendored and hosted lock-held, forced identifier type, proxy 403), which are now split into a JSON test and a render test. 18 sentence asserts remain, all in the render suite, as the tracker's target design calls for.
    • get/get_fixture.rs holds the shared fixtures once. It adds traversal_files, run_get_via_proxy and hosted_project_with_held_lock, which used to be inlined.
  • Sentence matches replaced by codes: patch_no_applicable_files (uuid guardrail), blob_write_failed (exact message match dropped), manifest_write_failed (×3, was "Failed to write manifest"), lock_held (hosted exact-message match dropped) and identifier_invalid (forced --id/--cve/--ghsa; the message is read only for the kind it names). The engine tests stop comparing events[].error to "patch has no applicable files" and "Blob decode or write failed".
  • One exact reason string remains in a JSON test: engine_view_404_is_download_failed keeps error == "could not fetch details". A view 404 (Ok(None)) and a fetch Err both record download_failed, so that fixed text is the only thing telling the two arms apart (found in /code-review).
  • New --json test: global_package_search_empty_prefix_is_no_packages. Before, only the human message covered the noPackages exit for an empty --global-prefix.
  • get/coverage_fix_get_double_json.rs is renamed get/get_single_json_document.rs, unchanged.
  • get_edge_cases_e2e.rs and get_modes_e2e.rs drop their private received_paths / requests_containing for the fixture's copies. received_paths now panics when wiremock recording is off, so a "no request" check can't pass on an empty list.
  • scripts/ci-test-durations.json (Windows shard timings only): the removed binary's entry is dropped, and get goes from 8.01s to 9.82s (the two runtimes summed).
  • A comment in scan/covgap_commands_scan_vendor_flow.rs points at the new test location.

CI cost

Workflows are unchanged. A lean PR run has the same ~28 Linux jobs before and after, at roughly the same job-minutes (~8 min wall). The Linux test shards link one fewer integration-test binary, which saves a few seconds per shard. That is too small for a ci-perf label.

The moved tests include cfg(unix) bodies (read-only .socket, first-party symlink), and prestage_lock_file is now cfg(unix), since only unix tests use it. Locally, I compiled the suite with every cfg(unix) item compiled out and the Windows venv layout selected (approximating Windows); it built with no warnings in the new files. The one full-scope run on 576b8e9 passed every job, including the macOS and Windows test shards: https://github.com/SocketDev/socket-patch/actions/runs/38658389722.

Test evidence

  • cargo test -p socket-patch-cli --all-features --test get: 167 passed. The 72 new or moved tests (get_failure_paths, get_render) passed 3 runs in a row. This container runs as root, so the read-only-directory tests take their documented skip here; they run for real on CI's non-root runners.
  • cargo test -p socket-patch-cli --all-features --test cli envelope_helper: 11 passed (the private-reader ratchet).
  • cargo fmt --all -- --check: clean.
  • cargo clippy --locked --workspace --all-features -- -D warnings (CI's invocation): clean. With --test get, clippy reports nothing in the new files.
  • CI on 576b8e9: ci-ok and clippy green, and every lean job passed. Cursor Bugbot found no issues.
  • python3 -m unittest discover -s scripts/tests: 347 OK, 6 skipped. This includes the shard planner test that reads ci-test-durations.json.

/code-review high

Fixed: the 404 test's branch check (above), the duplicated mock-request helpers in the same binary, and a mis-wrapped doc comment.

Not taken:

  • In-process tests setting SOCKET_* vars could leak into subprocess tests in the same binary. This is already handled: hermetic::command removes every flag-bound SOCKET_* var by name (GLOBAL_ARG_ENV_VARS / LOCAL_ARG_ENV_VARS, plus the SOCKET_CLI_* peers), not only the ones present when the sweep runs (common/hermetic.rs, pinned by spawn_env_hygiene::command_drops_flag_bound_vars_set_after_it_is_built). Every subprocess get here goes through common::run_with_env, which uses hermetic::command.
  • Use run_get_via_proxy in get_invariants.rs too. Those two tests sit outside this slice, and each documents its proxy setup inline; I left them as is.

Still open in #1089

Child 5's scan files (covgap_commands_scan_mod.rs, covgap_commands_scan_hosted.rs) and child 6 (render suites and the remaining non-covgap files).

🤖 Generated with Claude Code

https://claude.ai/code/session_017icypW7qeEf7tGLJUewAZh


Note

Low Risk
Test-only refactor with no production code changes; CI may need the updated shard timings for the consolidated get binary.

Overview
Reorganizes get integration tests (#1089) by removing the standalone covgap_commands_get.rs binary and folding its coverage into the get test suite.

get_failure_paths.rs keeps --json contracts: exit codes, status, error.code, event errorCodes, warnings[], and disk state via common::envelope helpers instead of long English string matches. get_render.rs holds human/--silent output pins for the same flows. Shared mocks, runners, and fixtures live in get_fixture.rs (including received_paths that fails if wiremock recording is off).

Adds a JSON test for the empty --global-prefix noPackages path, renames coverage_fix_get_double_json → get_single_json_document, wires main.rs for the new modules, and updates a scan vendor-flow comment to the new test location.

Reviewed by Cursor Bugbot for commit 576b8e9. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
The get covgap file pinned whole English sentences, so a copy edit to
get's output broke tests that were really about exit codes and disk
state (#1089, child 5, get slice).

covgap_commands_get.rs (its own test binary) moves into the get suite:
- get_failure_paths.rs asserts exit, status, error.code, event
  errorCodes, warning codes and disk state through common::envelope.
  It has no 4+-word sentence asserts; the old file had 22.
- get_render.rs keeps the human-output checks, one per message
  family, including the human halves of the old both-modes tests.
- get_fixture.rs holds the shared fixtures once.

A new --json test covers the global empty-prefix noPackages exit.
coverage_fix_get_double_json.rs is renamed get_single_json_document.rs.
The CI shard timings fold the removed binary into get.

Assisted-by: Claude Code:claude-opus-5-5
- The 404 view test again checks the fixed reason text: it is the
  only thing that tells the Ok(None) arm from a fetch Err, since both
  record download_failed.
- get_edge_cases_e2e and get_modes_e2e use the suite's shared
  received_paths / requests_containing instead of private copies.
  received_paths now panics when wiremock recording is off, so a "no
  request" check can't pass on an empty list.
- Doc comment rewrapped.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 15, 2026 00:09
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

…lope-asserts

# Conflicts:
#	scripts/ci-test-durations.json
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 0d4e7cf. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 15, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at 0d4e7cf (merge of main at 75e49ae).

  • Conflict fixed: scripts/ci-test-durations.json. main (CI: rollback tests assert codes, not sentences (#1089) #1552) removed the covgap_commands_rollback entry and this PR removes covgap_commands_get. Both suites are gone, so I dropped both lines. No code conflicts. cargo check -p socket-patch-cli --all-targets passes and cargo test -p socket-patch-cli --test get passes 167/167 locally.
  • CI: 54/54 checks passed (35 succeeded, 19 skipped). No failures that also fail on main.
  • Bugbot: reviewed 0d4e7cf and found no new issues. No unresolved review threads.

For reviewers: this is a test-only change. It moves covgap_commands_get.rs into the get/ suite (get_failure_paths.rs, get_render.rs, get_fixture.rs), and the tests now check error codes instead of English sentences (#1089).


Generated by Claude Code

Merged via the queue into main with commit 15db5c4 Oct 15, 2026
54 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-ci-get-envelope-asserts branch October 15, 2026 01:58
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 15, 2026
Resolves the ci-test-durations.json conflict with #1553 (get split)
and repoints the two get-suite doc comments that named the deleted
covgap_commands_scan_hosted file.

Assisted-by: Claude Code:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants