Skip to content

perf(chat): patch the open transcript window instead of re-reading it - #1235

Open
bmc08gt wants to merge 2 commits into
mainfrom
perf/chat-window-per-conversation-revision
Open

bmc08gt wants to merge 2 commits into
mainfrom
perf/chat-window-per-conversation-revision

Conversation

@bmc08gt

@bmc08gt bmc08gt commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

messageRevision was one counter per controller and part of the confirmed-window cache key. So an arrival in any chat, a reaction (ConversationReactions.didPersist), and each catch-up batch re-decoded the open chat's whole window on the main actor, at about 20 µs per row.

Writes now report what they touched, as a MessageWriteScope: every chat, one chat, or specific message rows in one chat. noteMessageWrite still bumps messageRevision, so the views that observe it re-evaluate as before. Then it handles the cached window by scope:

  • A write to another chat leaves the window alone. The revision is no longer part of the cache key.
  • Rows written in the open chat are re-read by id and patched into the window by binary search: replaced in place, inserted in id order, or dropped when the row is gone. For the newest-N window, rows that an arrival pushes past the limit are dropped too, using newestWindowFloor (one indexed OFFSET read). When more than a quarter of the window was written, it re-reads the window instead.
  • Writes whose rows aren't known drop the window as before: reset resync, the initial load, and the notification-extension reload in reloadFromDatabase. So does a write that removed a row from the newest-N window, since a fresh read would pull an older row in to fill the limit.

A catch-up collects the rows from each batch and reports them once, in a defer, so the error and reset paths report too. ConversationReactions.didPersist now passes the chat and message ids.

Trade-off: during a multi-batch catch-up, unreadBelowCount, lastConfirmedMessage, hasMessages and the jump badge read the DB directly but observe messageRevision, so they now update when the run ends rather than after each batch.

Numbers

These come from the chat-load benchmark harness (ChatLoadBenchmarkTests) on an iPhone 17 simulator (iOS 27.0), warm, with origin/main 641184f against this branch.

hostedRereads(), open window of 1000 rows in a 2000-row chat. Each event is one write:

Event Window misses before → after
reaction in the open chat 1 → 0
arrival in the open chat 1 → 0
arrival in another chat 1 → 0
notification-extension write 1 → 1

The same holds for the 60-row window.

reactionFlow(), the main-actor cost of one reaction after the DB write. Before, that was a full window re-read. After, it's the patch inside didPersist, and the re-read is a cache hit:

Window rows Before (re-read) After (patch)
60 1.27 ms 0.12 ms
1000 20.33 ms 0.20 ms
5000 106.66 ms 0.62 ms
10000 211.57 ms 1.08 ms

The before and after columns are separate runs a few minutes apart, on a machine with a load average of 80 to 150. A separate earlier origin/main run measured 20.46, 99.90 and 212.07 ms at 1000, 5000 and 10000 rows. The cost left after the patch grows with the window, likely from copying the cached array (inferred, not profiled).

The extension write still misses, because the Darwin notification doesn't say which chats it wrote (see #1234).

Tests: ConfirmedWindowPatchTests checks the patched window against a fresh messages(conversationID:from:) / messagesWindow read on a temp database, after:

  • appends, an in-place edit, and a write below the anchor
  • arrivals that push rows out of the newest-N window
  • a written row that is no longer stored

At the controller level, it covers:

  • a write to another chat keeping the open chat's window
  • an arrival and a reaction patching only the reported row
  • a write that can drop rows forcing a re-read
  • a multi-batch catch-up reporting its writes once

`messageRevision` was one counter per controller and part of the
confirmed-window cache key. A write to any chat, a reaction, and every
catch-up batch re-decoded the open chat's whole window on the main
actor, at about 20 µs per row.

Writes now report which chat and which message rows they touched.
`noteMessageWrite` still bumps `messageRevision` for the views that
observe it, and then:

- leaves the cached window alone for a write to another chat
- patches the written rows into the window for a write to the open
  chat, by binary search, and drops rows an arrival pushes out of the
  newest-N window
- drops the window when a write's rows aren't known (reset resync,
  initial load, the notification-extension reload)

A catch-up reports its writes once, when the run ends, not once per
batch. `ConversationReactions.didPersist` now passes the chat and
message ids.
@bmc08gt bmc08gt self-assigned this Nov 6, 2026
A fresh read pulls an older row in to fill the limit, which the patch
can't see.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant