[Hackathon] --wait polling turns into a tight request loop when a long-poll read comes back without being held (run poll + plan generation) #481

Verified against (commit SHA or released version)

commit 988c035 (main, v0.14.0)

Discord Username / User ID

949225590181994556

What does this improvement do?

What's broken

pollRunUntilTerminal (src/lib/poll.ts), which every --wait path goes through (test run --wait, test run --all --wait, test rerun --wait, test wait, testlist run, test flaky, test create --run --wait), re-issues the GET /runs/{id}?waitSeconds=N read immediately after every non-terminal answer:

// Long-poll path: server already waited up to waitSeconds for us;
// loop immediately (no extra client-side sleep).
if (!useBackoff) {
  if (Date.now() >= deadlineMs) throw new TimeoutError(...);
  continue;
}

That is only safe if the server really held the read. The loop never checks. When a non-terminal answer comes back early, nothing paces the next request, and the CLI sends back-to-back GETs for the whole --timeout (up to 3600 s). This happens when a server or proxy accepts but ignores waitSeconds, when an intermediary answers early, or when the backend only holds some statuses. The code only switches to the paced backoff mode on a 400 VALIDATION_ERROR, so a server that silently ignores the parameter never triggers the fallback.

The plan-generation ladder (src/lib/plan-poll.ts, test plan generate) does the same in its active-status branch (if (!useLongPoll) await paceBeforeNextRead(); continue;). Its own comment says the server only enters the waitSeconds hold for some statuses. So the "server already waited" assumption already depends on server behaviour the client doesn't check.

Impact:

• A request flood against the API from every waiting CLI. A --all --wait batch multiplies it by --max-concurrency.
• The flood trips the per-minute throttle. poll.ts rethrows RATE_LIMITED as soon as http.ts's 3 attempts are used up, so a healthy run's wait ends with exit 11 even though the test is fine.

Repro (no API key, no network)

// node repro.mjs (after npm run build)
import { pollRunUntilTerminal } from './dist/lib/poll.js';
let calls = 0;
const client = { async getRun(id) { calls++; return { runId: id, testId: 't', status: 'running' }; } };
try { await pollRunUntilTerminal(client, 'r1', { timeoutSeconds: 2 }); } catch (e) { console.log(e.name); }
console.log('getRun calls in 2s:', calls);

Actual (main @ 988c035): TimeoutError / getRun calls in 2s: 6728387. Against a real server, that is one request per round trip for the whole timeout.

Expected: a non-terminal answer that came back without being held is followed by the same pacing the fallback mode already uses (2 s), so at most one read per interval.

Details / implementation notes

Measure how long each long-poll read was held:

• A read held for at least the fallback pace (BACKOFF_SCHEDULE_MS[0] = 2 s in poll.ts, SETTLED_POLL_DELAY_MS = 2 s in plan-poll.ts) is re-issued at once, as today.
• A faster read sleeps only the remainder, clamped to the --timeout deadline.

Behaviour against a server that holds reads is unchanged: no extra sleep, no extra latency. No new dependencies and no new requests. Vitest coverage uses a fake Date so the test is deterministic.

I have a fix with tests ready on a branch and will open the PR once this is accepted.

Confirmations

• I have searched existing issues and this is not a duplicate.
• I have provided my Discord identity above for reward coordination.