Packaging smoke harness reports exitCode: 0 for timed-out commands — a hang is indistinguishable from a pass #1795

Found while building .squad/e2e/windows-test-baseline.md (EECOM, baseline commit 369bba8f).

Summary

The runCommand helper in test/cli-packaging-smoke.test.ts catches an execFileSync timeout and returns exitCode: 0. A command that hung is therefore indistinguishable from a command that succeeded. This is the same "reports nothing → mistaken for passing" shape as #1788, in a suite that is supposed to be our packaging safety net.

Mechanism

test/cli-packaging-smoke.test.ts:158-188:

const stdout = execFileSync(process.execPath, [cli.cliEntryPath, ...args], {
  cwd: cli.tempDir, encoding: 'utf8', timeout: 2000, env: NO_COLOR_ENV,
});
return { stdout, stderr: '', exitCode: 0 };
} catch (err: any) {
  // Timeout means the command started running (routed) but hung
  // waiting for infrastructure — that's a pass for routing verification
  if (err.killed || err.signal === 'SIGTERM') {
    return { stdout: ..., stderr: ..., exitCode: 0, timedOut: true };   // <-- hang becomes success
  }

The intent is defensible for expectCommandRouted (a hang does prove routing). But exitCode: 0 leaks to every caller, including assertions that are not about routing.

Blast radius: 6 call sites use expectCommandRouted, and the remaining assertions in the file consume exitCode directly with no knowledge that 0 may mean "timed out".

Controlled proof

On Windows at 369bba8f:

FAIL  test/cli-packaging-smoke.test.ts > squad --version exits 0 and outputs semver
AssertionError: expected '' to match /^\d+\.\d+\.\d+/
  245|     const result = runCommand(['--version']);
  246|     expect(result.exitCode).toBe(0);      <-- PASSED
  247|     expect(result.stdout).toMatch(/^\d+\.\d+\.\d+/);   <-- failed, stdout was ''

exitCode: 0 passed while stdout was empty. A working CLI prints a version on --version; the try branch returns real stdout. Empty stdout with exit 0 can only come from the timeout branch. So squad --version exceeded the 2000 ms budget on Windows cold start and the harness reported it as a clean exit.

Had this test asserted only exitCode, it would have passed while the CLI hung.

Suggested fix

1. Never return exitCode: 0 for a timeout. Return a distinct sentinel (e.g. exitCode: null, keep timedOut: true) and let expectCommandRouted explicitly accept timedOut. Then a hang can only pass a test that has opted into accepting a hang.
2. Raise the timeout for Windows cold start (2000 ms is tight with Defender + cold FS), or make it env-tunable. This is secondary — the masking is the defect, the slowness is the environment.
3. Consider asserting a non-empty stdout in expectCommandRouted so "routed" means "produced output", not "did not crash within 2s".

Success criteria

• A deliberately-hanging command causes any test asserting exitCode === 0 to fail, not pass.
• expectCommandRouted still tolerates a genuine infrastructure hang, but does so explicitly via timedOut.
• squad --version returns a semver on Windows within the configured budget.
• A regression test that fails against current dev by simulating a timeout and asserting the harness does not report success.

Related

• .squad/e2e/windows-test-baseline.md row 5.
• check-changeset-drift.test.ts silently runs zero tests on Windows (shebang + CRLF) #1788 — same class: a gate that reports nothing and is read as passing.