[tujack39]

Season 4 CLI Improvement Bonus

• I'm entering this PR for the Season 4 CLI Improvement Bonus

Discord username: johnlu1976

Closes #398

Problem

When a --wait fan-out (test run --all / test rerun / testlist run)
fails because a member poll hit an auth error, resolveWaitFailure()
(src/lib/wait-exit.ts) builds the process-level error as
new CLIError(message, 3) without passing a code. CLIError.code therefore
defaults to the generic, out-of-catalog 'CLI_ERROR' bucket, so:

• --output json prints error.code: "CLI_ERROR" even though the process
  exits 3 (auth, per the documented exit-code table) and the message itself
  says auth error (AUTH_...): ...;
• telemetry's errorCode (via classifyCliError, which reads err.code)
  is misclassified the same way.

Agents and CI that branch on error.code to decide whether to re-auth
cannot do so for this path — the one failure where re-auth is exactly the
right response. Every other branch of resolveWaitFailure already surfaces
a faithful code (typed operational errors are routed through
ApiError.fromEnvelope); the auth branch was the odd one out.

Fix

Pass the member's real catalog code through CLIError's existing code
constructor parameter:

return new CLIError(
  `${notPassed} run... failed — auth error (${authErr.error.code}): ${authErr.error.message}`,
  3,
  authErr.error.code,
);

authErr is selected via isAuthCode(), which is derived from the single
exitCodeFor mapping, so authErr.error.code is guaranteed to be one of
AUTH_REQUIRED / AUTH_INVALID / AUTH_FORBIDDEN — the exit-3 family,
consistent with the hardcoded exit code and with the code already named in
the message. Envelope error.code, telemetry errorCode, exit code, and
message now all agree.

Tests

src/lib/wait-exit.test.ts:

• Extended "auth error wins over a concurrent timeout" to assert
  f.code === 'AUTH_INVALID'.
• New test "an auth failure carries the member auth code, not the generic
  CLI_ERROR": asserts the returned CLIError carries
  code === 'AUTH_REQUIRED' and that classifyCliError(f).errorCode
  (the telemetry path) reports the same.

Both assertions fail on the pre-fix code with
expected 'CLI_ERROR' to be 'AUTH_INVALID' / 'AUTH_REQUIRED'
(verified by reverting the fix locally).

Verification (local, Node v24.20.0)

• npm test — 396 files passed, 7056 tests passed, 26 skipped, 0 failed.
• npm run typecheck — clean.
• npm run lint — 0 errors (215 pre-existing warnings, none in the
  changed files); check-ticket-refs clean.
• npm run format:check — all files pass Prettier.
• npm run test:coverage — the changed file src/lib/wait-exit.ts is at
  100% statements / 100% lines / 100% functions (84.5% branches) under its
  test file. Note: the full coverage run in this sandbox shows 6 failing
  tests in doctor.test.ts / test.test.ts / secret-file.test.ts — all
  POSIX chmod/EACCES cases that fail because the sandbox runs as root
  (root bypasses file-permission checks). Verified pre-existing: the same
  tests fail identically on the pristine tree with this change stashed,
  and none of them touch the --wait path.

Hackathon S4

• I am participating in TestSprite Hackathon Season 4
• Discord: johnlu1976

Summary by CodeRabbit

• Bug Fixes

  • Authentication failures now retain their specific error classification in CLI output and telemetry, improving error reporting.
  • Concurrent authentication and timeout failures now resolve consistently with the appropriate authentication error when applicable.

• Tests

  • Added coverage for authentication error codes, telemetry classification, and concurrent failure handling.

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

• Source mode: Auto
• Repository: TestSprite/testsprite-cli
• Branch: main
• File: .coderabbit.yaml
• Commit: 2ff4960
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3365bda3-65b7-4f53-96f7-53854cfaeab5

Commits

Reviewing files that changed from the base of the PR and between ecf10ed and e2fc728.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The wait failure path now preserves specific authentication error codes in CLIError and telemetry classification. Tests cover AUTH_INVALID precedence and AUTH_REQUIRED classification.

Changes

Wait authentication error codes

Layer / File(s)	Summary
Preserve authentication error codes src/lib/wait-exit.ts, src/lib/wait-exit.test.ts	resolveWaitFailure passes the member’s authentication code to CLIError. Tests verify that AUTH_INVALID and AUTH_REQUIRED remain classified as authentication errors.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix · Severity of issue fixed: Low

Merge Risk: Minimal · previous review ecf10

The authentication error-code change is ready to merge based on the supplied review evidence.

Pre-merge checks: 5

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: reporting the specific authentication error code for --wait failures.
Linked Issues check		Issue #398 requires the batch --wait auth branch to preserve the member auth code, exit with code 3, and keep the auth message and telemetry classification consistent. src/lib/wait-exit.ts now pas…
Out of Scope Changes check		The reviewed changes are limited to the resolveWaitFailure() auth error construction and focused tests in src/lib/wait-exit.test.ts. The tests directly verify the behavior required by issue #398. …
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 2 files.

Full details: Linked Issues check

Explanation

Issue #398 requires the batch --wait auth branch to preserve the member auth code, exit with code 3, and keep the auth message and telemetry classification consistent. src/lib/wait-exit.ts now passes authErr.error.code to CLIError while retaining exit code 3 and the auth code in the message. src/lib/wait-exit.test.ts checks AUTH_INVALID, AUTH_REQUIRED, exit code 3, the auth message, and telemetry classification.

Full details: Out of Scope Changes check

Explanation

The reviewed changes are limited to the resolveWaitFailure() auth error construction and focused tests in src/lib/wait-exit.test.ts. The tests directly verify the behavior required by issue #398. No unrelated product behavior or unrelated files are identified.

Finishing Touches

Fix failing CI checks

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: TestSprite/testsprite-cli/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: de069264-7ab5-417a-8b91-58fd4f7e2e77

📥 Commits

Reviewing files that changed from the base of the PR and between d61e6df and ecf10ed.

📒 Files selected for processing: 2

• src/lib/wait-exit.test.ts
• src/lib/wait-exit.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

[github-actions]

🏆 Season 4 CLI Improvement Bonus: entry recorded. Thanks, @tujack39!

To be counted, this PR needs the following by Oct 29, 11:59 PM PT:

• ✅ A linked issue assigned to you.
• ✅ Out of draft.
• ✅ A Discord username in the description.

Rewards go to merged PRs judged a genuine improvement, and up to three bonus-eligible PRs count per person. Untick the box to withdraw the entry. This comment refreshes when you edit the PR. Full rules.