Skip to content

Fix idle Console CPU and IO with change-driven updates - #1160

Merged
JacobStephens2 merged 8 commits into
mainfrom
issue-1153
Oct 23, 2026
Merged

JacobStephens2 merged 8 commits into
mainfrom
issue-1153

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

filesystem changes ─┐
process exits      ├→ shared notices → Inbox / log streams
elapsed boundaries ─┘

unchanged retained files → bounded parse/text caches
60s recovery → metadata changes + account expiry checks
session last-use writes → at most once per minute

Replace all four fast polling loops. Preserve updates across atomic saves, replaced log roots, macOS canonical paths and external Session traces. Share process waits and elapsed-time deadlines while viewers are connected. Preserve loaded Theme fonts and unsubmitted Lanes filters during live updates.

Closes #1153

Evidence

60s measurements on stephens.page, using stable snapshots of its existing 6.4 GB store and isolated signed-in Consoles. Three Inbox clients consume SSE and refetch HTML on each Command notice, alongside one live log reader. The fixed binary is an unoptimized build. The service was untouched; scratch artifacts were removed.

Idle case Before CPU (one core) After CPU Before rchar After rchar
No viewers 28.45% 0.75% 428,430,723 B 588,467 B
3 refetching Inbox clients + 1 live log reader 261.18% 2.07% 1,809,106,605 B 4,222,722 B

All three acceptance limits pass. Both fixed samples had zero write_bytes; the viewer sample made zero idle refetches. Earlier socket-only viewer numbers are superseded by this stronger measurement. It exposed unnecessary recovery broadcasts, then parent-directory metadata notices; both were fixed before this final run.

The extra IO came from each log reader's 250ms appended → catalog::find → load → Snapshot::load inventory scan, plus the unconditional render_page → status::inbox loop without viewers. Parsing caches alone did not remove that polling work.

  • Before: idle IO regression read 2,098,898 B in 3s; an unchanged request read 1,049,817 B; session persistence rewrote the account store before a minute. After: native notices, retained-file caches and throttled persistence pass those regressions.
  • Before: the Spec review's exact process-death command timed out despite a fresh read seeing completion. After: the same command passes; its tracked Rust regression verifies final unterminated output and completion within 2s without a record write.
  • Before: the reviewers' exact Inbox clock-boundary and session-expiry commands failed on 428a5b6. After: both exact commands pass. Retained tests cover Done → Resolved, retention expiry and an open session expiring after a child-only clock shift.
  • Before: browser stress exposed font unloading during Theme refresh and a live push resetting an edited Lanes filter. After: ten Theme stress passes, a retained filter regression, and the complete 60-case Console browser suite pass.
  • Focused final checks: both notification unit tests, all three watching HTTP tests and 38 logs unit tests pass. Final PR CI passes the full Linux Rust suite, formatting, clippy, the expanded macOS gate, 49 site browser tests and 60 Console browser tests. Linux cargo test took 571s including 28.17s compilation. A separate full cross-platform CI run also passes, including all four macOS shards.

/usr/bin/time -p scripts/affected-tests main passed the site selection. Its redundant local Rust run was interrupted after 708.91s (including the site run and 0.55s Rust compilation) once exact-head Linux CI passed, as docs/testing.md directs. This is recorded as an interrupted local run, not a local full-suite pass.

Test seams: Console binary/HTTP/SSE, fixture files, native process lifetime and Linux process IO; durable accounts with a supplied clock; child-only wall-clock injection; browser Theme editing and Lanes filter state.

Unaddressed findings

Standards

The other five advisory findings and the reproduced P2 session-expiry finding are fixed.

Spec

None. Both P2 findings are fixed and their regressions are retained.

Security

None. Guidance-mode review of the 3c53d52...HEAD diff (attack classes: auth/session, resource/file handling, availability, data isolation; no SECURITY.md or threat model in repo) found no introduced Security findings and fixed none. The one candidate considered — SSE/log streams rechecking still_allowed on account notices plus a 60s fallback instead of every 250ms — is not a finding: notices carry no data, every new request still re-authenticates in auth::guard, allowlist/config edits raise an account notice immediately, and the revocation and expiry tests pass (console::accounts::tests, console::changes::tests, watching HTTP tests).

Security review outcome

No unaddressed introduced findings.

Review coverage

Independent Standards and Spec reviews compared the branch with main, pinned to 3c53d52 after merging updated main. Both axes inspected all 31 changed files, including all final fixes. Unread changed files: none. Generated-output exclusions: none. Their complete reports, earlier findings, dispositions, exact reproductions and each axis's combined files-read list are saved to .thirdshift-review-ECulfg/standards.md and spec.md in this worktree.

Merge Danger

Door: two-way

No persisted format or log-store migration. Reverting restores the previous reader behavior.

Blast Radius: Console

Linux uses pidfds and macOS uses kqueue, with identity-only fallback when native process waits are unavailable. The macOS gate now includes filesystem notifications, account revocation, binary replacement and elapsed-time HTTP regressions.

Built with codex · gpt-6.1-sol · xhigh

@JacobStephens2
JacobStephens2 merged commit f197dcb into main Oct 23, 2026
18 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-1153 branch October 23, 2026 16:22
@JacobStephens2 JacobStephens2 mentioned this pull request Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

thirdshift console uses 50-74% of a core while idle

1 participant