Skip to content

fix(e2e): cancel a case's interrupt once it has been accounted for - #82

Open
aniruddhaadak80 wants to merge 1 commit into
CopilotKit:mainfrom
aniruddhaadak80:fix/e2e-clear-the-interrupt-timer
Open

aniruddhaadak80 wants to merge 1 commit into
CopilotKit:mainfrom
aniruddhaadak80:fix/e2e-clear-the-interrupt-timer

Conversation

@aniruddhaadak80

Copy link
Copy Markdown

What this changes

The E2E harness schedules a mid-stream interrupt for a case and then never cancels it:

let interruptTimer: NodeJS.Timeout | undefined;
if (spec.interrupt && parentTs) {
  interruptTimer = setTimeout(() => {
    postAsUser(TEST_CHANNEL, spec.interrupt!.prompt, { threadTs: parentTs })
      .catch((e: Error) => errors.push(`interrupt send failed: ${e.message}`));
  }, spec.interrupt.afterMs);
}

and 23 lines later, this:

if (interruptTimer === undefined) {
  /* no-op */
}

That block is the tell. The handle was read only to stop the compiler complaining about an unused variable, and nothing in e2e/ ever calls clearTimeout — grep for it across e2e/*.ts returns nothing, and the only other setTimeout calls are all awaited.

afterMs is deliberately longer than a lot of replies take, so this fires whenever the bot answers before the delay:

  1. Case A schedules the interrupt at afterMs: 15000, the bot replies in 3s, runCase returns with the timer still armed.
  2. main() starts case B.
  3. At t=15s the timer from case A fires and posts A's interrupt prompt into A's parentTs thread — a message the runner never sent and no case is watching. That starts an unplanned bot turn.
  4. If the send fails (thread_not_found, an archived thread), it pushes interrupt send failed: … into case A's errors array — but A's CaseResult was already returned and written to report.json at run.ts:386-389.

So a fast reply leaks a real message into a real thread, and can append a failure to a result that has already been reported.

The scheduling moves into e2e/interrupt.ts and hands back a cancel(), called as soon as the interrupt phase has read its result — the point after which the interrupt can no longer belong to this case. The dead no-op goes with it.

Why a new module

e2e/run.ts calls main() at import (run.ts:397-400), so a test importing it would execute the whole harness against real Slack and then process.exit. The scheduling is therefore extracted, which is what makes the behaviour testable at all. vitest.config.ts only includes app/**/*.test.ts, so the test lives at app/e2e-interrupt.test.ts and imports from ../e2e/interrupt.js, matching the existing app/e2e-cases.test.ts which already reaches into e2e/.

Verification

Commands run from the repository root, per AGENTS.md:

  • pnpm check-types — exit 0.
  • pnpm test — 30 files, 440 tests, all passing. The four new ones are in app/e2e-interrupt.test.ts, using fake timers:
    • is sent once the delay has passed — the behaviour that already worked, pinned.
    • is not sent at all once the case has cancelled it — the regression. Cancels, then advances 120s and asserts nothing was sent.
    • reports a send that failed, and still only once — the interrupt send failed: … message is preserved, so the change does not swallow a real failure.
    • can be cancelled after it has already fired without throwing — makes cancel() idempotent, since it is called on a path that may run after the timer fired.
    • I confirmed red first: before e2e/interrupt.ts existed the suite failed to resolve its import.
  • node node_modules/railway/dist/iac/bin.js — exit 0, "diagnostics": [].

Two things I did not run, stated plainly rather than implied:

  • uv run pytest in agent/ (AGENTS.md lists it). This change is TypeScript-only and touches no Python, but I did not execute it.
  • CI's pnpm --dir deployment/aws install/build/test. My diff is confined to e2e/ and app/, so it cannot affect the CDK deployment, but I did not run those steps either.

I also have not run pnpm e2e itself — it needs SLACK_USER_TOKEN and a live bot, so the fix is verified at the scheduling layer rather than against a real interrupted thread.

The mid-stream interrupt was scheduled and never cancelled:

  let interruptTimer: NodeJS.Timeout | undefined;
  if (spec.interrupt && parentTs) {
    interruptTimer = setTimeout(() => { postAsUser(...) }, spec.interrupt.afterMs);
  }
  ...
  if (interruptTimer === undefined) {
    /* no-op */
  }

That trailing block is the tell: the handle was read only to keep the
compiler quiet about an unused variable, and nothing ever cleared the timer.
afterMs is longer than a lot of replies take, so a case whose bot answers
first returns with the timer still armed and the runner moves on to the next
case. It then fires into the finished case's thread: a message the runner
never sent, a bot turn nobody asked for, and an "interrupt send failed" line
appended to an errors array whose CaseResult was already returned and written
to the report.

The scheduling moves to e2e/interrupt.ts and hands back a cancel(), called as
soon as the interrupt phase has read its result - the point after which the
interrupt can no longer belong to this case. run.ts calls main() at import, so
this is also what makes the behaviour testable at all; the new test drives the
fake timers directly.

The dead no-op goes with it.
Copilot AI balanced review requested due to automatic review settings October 4, 2026 11:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants