Skip to content

Spawn every CLI test child through one hermetic Command builder; 10 test files inherit ambient SOCKET_* today #823

Description

[agent] Filed by the scheduled architecture audit routine (CLI and core). Register: register comment.

Kind: refactor (test structure). Source: review Part 8.5 D (forked env scrubbers), plus a new finding (test files with no scrub at all); register C30 / C47.

Problem

The CLI test tree already has a hermetic spawner: common::run_bin_with_env seeds and scrubs the high-risk vars, prefix-sweeps the remaining SOCKET_* (keeping the telemetry opt-outs, SOCKET_NO_CONFIG and SOCKET_NO_UPDATE_CHECK), and forces SOCKET_NO_CONFIG=1 / SOCKET_NO_UPDATE_CHECK=1. But it only returns (code, stdout, stderr), so tests that need a Command (stdin, PTY, extra env removal, output() bytes) hand-roll their own:

  • 15 private scrub_socket_env copies with 14 different bodies. Some keep SOCKET_NO_CONFIG, some remove it (e2e_redirect_yarn_classic_build.rs#L98`` strips every SOCKET_*, including the `.cargo/config.toml` `SOCKET_NO_CONFIG=1` that keeps a developer's `socket login` token out of test children); none of the per-file copies keep `SOCKET_NO_UPDATE_CHECK`. Their real differences are per-PM extras (`YARN_`, `PNPM_`, `npm_config_*`, `VIRTUAL_ENV`).
  • 10 files that spawn the binary with no scrub at all (only SOCKET_TELEMETRY_DISABLED or a token removal): repair/repair_vendor_e2e.rs (L201-L212),`` scan/scan_invariants.rs (L53), `scan/scan_sync_e2e.rs`, `cli/api_client_errors_e2e.rs`, `cli_parse_remove.rs`, `apply/in_process_npm_multicopy.rs`, `self_update_e2e.rs`, `self_update_failures_e2e.rs`, `update/covgap_update_swap.rs`, `update/covgap_update_download.rs`. (Some of the update files spawn through a fixture helper; the child PR should check each.)

Proof (on 045d7ec, debug build, run twice, as root in a cloud container):

ambient var repair_vendor_e2e (unscrubbed, 19 tests) the other 97 tests in the repair target (scrubbed)
none 19 pass 95 pass, 2 fail*
SOCKET_DRY_RUN=true 1 pass, 18 fail ("setup must vendor the tarball") 95 pass, 2 fail*
SOCKET_OFFLINE=true 1 pass, 18 fail 95 pass, 2 fail*

*The two baseline failures are the "lock file unremovable" tests, which can't simulate an unremovable file as root; they are unrelated.

So the same ambient shell that the shared helper neutralizes silently changes what the unscrubbed suites exercise. Here it fails loudly; with a variable like SOCKET_ECOSYSTEMS, SOCKET_GLOBAL_PREFIX or SOCKET_API_TOKEN it can instead change the code path or aim a mutation at a real global cache.

Symptoms

None filed; contributors hit this as "passes in CI, fails locally".

Impact

Test hermeticity and maintenance: each new SOCKET_* variable has to be remembered in up to 16 places. Size: test-only, no production change.

Proposed change

  1. Split run_bin_with_env into pub fn hermetic_command(bin: &Path) -> Command (seed, scrub, force the two opt-outs) and the existing runner, which becomes hermetic_command(bin) + args + caller env + output().
  2. Give it the per-PM extras as an explicit opt-in, e.g. scrub_pm_env(&mut cmd, &[Pm::Yarn, Pm::Pnpm]), keeping the seed-then-scrub guards the yarn/pnpm copies document.
  3. Route the 10 unscrubbed files and the 15 scrub_socket_env copies through it, and delete the 15 copies.

Size and scope

~25 test files, net negative (~−250 lines). Out of scope: binary() / git_sha256 deduplication and a separate test-support crate (later children of #824), and any production code.

Acceptance criteria

  • common::hermetic_command exists, and run_bin_with_env is built on it.
  • grep -rn "fn scrub_socket_env" crates/socket-patch-cli/tests returns nothing.
  • Every test file that spawns CARGO_BIN_EXE_socket-patch (or a staged copy of it) does so through hermetic_command or the run* helpers; a guard test (or a grep in an existing lint test) keeps it that way.
  • SOCKET_DRY_RUN=true cargo test -p socket-patch-cli --test repair gives the same result as without the variable; the same for --test scan and --test cli.
  • All existing tests stay green in CI; the yarn/pnpm seed-then-scrub guards still fail red when the scrub is removed.

Dependencies

Blocked by nothing. First child of tracking issue #824. Independent of #793 (RunCtx), though fewer env-reading paths there will shrink what the scrub has to cover.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:claimedagent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)priority:p3refactorStructural change: duplicated code or logic, missing abstraction, layering, dead code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions