Skip to content

[bug] The idle reap releases a process on the user's own machine and notifies nobody #698

Closed

Description

@serge-ivo

A run finishes, the owner walks away, six hours later the engine process on their machine is released. The only record is a row written into the session's own timeline, which is visible only to someone who has already come back and opened that repo.

The absence

grep -c 'notifyUser\|createNotification' workers/api/src/lib/coding-session-sweeper.ts → 0.

The sweeper writes the reap into coding_timeline as the run's outcome (via IDLE_REAP_PREFIX, :109) and updates the session row. It sends no notification, so nothing reaches /v1/notifications, the console bell, or push.

Why that is the wrong shape

The reap is the platform releasing a resource on the user's own machine — IDLE_SESSION_MS is 6h (:63) and the header is explicit that the number is chosen so an abandoned session does not hold a child process on somebody's laptop overnight. That is a decision about their computer, taken while they are not looking, and it is exactly the kind of thing a notification exists for.

Compare the second branch of the reap message (:123): "the engine could not be stopped on your machine — it may still be running. Check it there if you need the process gone." That branch is asking the user to go do something on their own hardware, and there is no channel that carries the ask. It is written into a timeline the user has no reason to open.

Observed, 2026-08-17

Instance bd43f4de (Chess coder 2). Session csess_21b8d1b9 ran 2026-08-16 09:41→17:07 UTC; real work ended 11:11 UTC and the reap fired at 17:07 UTC, 03:07 Melbourne time. The owner found out the next morning by opening the repo and reading a dead pane, then asked what had happened. Nothing had told them.

Fix

Notify on reap, and make the notification carry the thing that is actually useful — not that a session ended, but until when the work is still resumable:

"Chess coder 2 went to sleep on apps/chess-academy. Your conversation is kept until 20 Aug."

The date is computable today: RESUME_WINDOW_MS (lib/coding-session-continuity.ts:47, 4 days) applied to the session's last_activity_at — the same column the reaper already measures idleness against (coding-store.ts:410 notes the two clocks and their relationship).

Deliberately NOT session vocabulary — see #695. The user does not need to know a process was released; they need to know their work is safe and for how long.

Route it through notifyUser rather than createNotification directly, so the owner's notification preferences apply. Worth checking against #656, which found the subscribe path bypassing notifyUser and producing an unmutable ping — do not add a second one.

One judgement call for whoever takes this: a user with many coding instances could get several of these a night. Either suppress when the repo had no activity that day beyond the run that already reported its own outcome, or batch per instance. A nightly "3 agents went to sleep" is fine; three separate 3am pings are not.

Activity

  1. added
    bugSomething isn't working
    backendBackend / Worker / API work
    on Aug 17, 2026
  2. serge-ivo commented on Aug 17, 2026

    @serge-ivo
    ContributorAuthor

    Implemented in 05d5250f, with a follow-up in 849dcda5. API Worker deployed, CI green on the follow-up. Leaving this open on one point — see the bottom.

    The judgement call you left me: BATCHED PER OWNER PER SWEEP.

    People stop working all at once, so one evening's repos share a last-activity window and idle out in the same per-minute tick — batching there is what turns three separate 3am pings into one. Rejected:

    • Suppress a repo whose run already reported its own outcome. It answers a different question — whether the run was reported, not whether the machine was touched — needs a query nobody has, and still lets N repos produce N pings.
    • Batch per instance. Same failure one level up: a user with five coding agents still gets five.

    The residual — repos that idle out a few ticks apart — is caught by the mechanism that already exists for it rather than a second timer: every sleep notification carries the same stable event key, so notifyUser's ten-minute duplicate window collapses the interruption while both batches still write their row. The bell keeps the full record; the phone buzzes once.

    update, never alert. An alert is documented as "a human is blocked on this" and is unmutable; nothing is blocked here, and a 3am ping people cannot turn off is how they learn to stop reading these — the same reasoning coding-pause.ts applies to a usage-limit park. Routed through notifyUser, so preferences apply — the trap you flagged from #656.

    What it says: 😴 Chess coder 2 went to sleep / apps/chess-academy. Your conversation is kept until Thu 20 Aug, 11:11.

    The time is in there deliberately, and it is a correction to the issue's sketch. "Kept until 20 Aug" is a promise the platform breaks: the window expires at an instant — 11:11 Melbourne in your own example — so a user returning that evening would find the conversation gone having been told it would be there. Five characters buy a sentence we can keep. No stored zone → UTC, labelled UTC, per accountTimeZone's "nobody has told us is a first-class state".

    It also refuses to promise what it cannot. A raw engine (Codex, Grok, a custom command) has no conversation, so those get no "kept until" clause at all. resumeDeadlineFor asks resolveSessionContinuity the same question the next open will ask rather than carrying a second copy of "claude is the engine with a resume protocol" — that copy would be right today and silently wrong the moment another engine grows one. Multi-repo batches state the earliest deadline, since "kept until X" has to be true of everything the sentence covers.

    The stray-process branch keeps its ask and now has a channel: "One engine could not be stopped on your machine and may still be running — check there if you need the process gone."


    A miss worth recording. The first push went red on CI: listIdleSessions gained three LEFT joins and a documented row shape, taking coding-store.ts from 798 to 844 and tripping the #302 ratchet. I had run the gate — as node scripts/check-file-size.mjs | tail -1 under set -e, and a pipeline's status is its last command's, so tail succeeded and a failing gate passed silently. Fixed forward in 849dcda5, and the fix was an extraction, not a pin: scripts/check-file-size.mjs carries a note from #676 recording the last time this exact file crossed 800, ending "a pin was the easy answer and the wrong one." listIdleSessions and listInstancesWithQuietSessions have exactly one caller each — the sweeper — and answer the cron's question rather than the registry's, so they moved to lib/coding-sweep-queries.ts with the coverage the move made worth writing (the joins are LEFT so a deleted repo still gets its engine released; the instance label prefers the owner's own rename; an unrecognised client_type normalises rather than passing through).

    Why this stays open: the mechanism is tested (15 new cases) and deployed, but I have not observed it fire in production — that needs a real six-hour idle reap, which cannot be forced from here. What to look for after the next one: a single coding notification per owner per sweep, titled 😴 … went to sleep, with a kept until <date>, <time> clause and no occurrence of the word "session". Close it once that lands.

  3. serge-ivo commented on Aug 18, 2026

    @serge-ivo
    ContributorAuthor

    Audit verdict: delivered — closing. 05d5250f (+ 849dcda5, which moved two queries out of coding-store.ts when this pushed it past the 800-line ratchet).

    The judgement call you left open was taken, and taken the way you framed it as acceptable. :242 — the sweeper batches per user within one per-minute tick, so "three separate 3am pings" become one; :267 renders 😴 3 agents went to sleep / 😴 <agent> went to sleep on N repos. The rejected alternative is recorded at :243: suppressing a repo whose run already reported its own outcome. :272-276 takes the earliest deadline across the batch, because "Kept until X" has to be true of everything in it. SLEEP_NOTIFY_KEY is constant per user so notifyUser's duplicate window collapses two batches a few minutes apart (:390).

    One residual the fix names itself rather than hides (:287): there is no page that shows every sleeping repo across agents, so a batch spanning several agents has no single link to point at. That is a surface gap, not this bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend / Worker / API workbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions