Skip to content

fix(chat): keep stored history when a newest page misses it - #1237

Open
bmc08gt wants to merge 1 commit into
mainfrom
fix/chat-history-gap-keep-rows
Open

bmc08gt wants to merge 1 commit into
mainfrom
fix/chat-history-gap-keep-rows

Conversation

@bmc08gt

@bmc08gt bmc08gt commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

Opening a chat after missing 50 or more messages deleted its whole stored history. dropStaleEpochIfNeeded ran whenever the newest page (50 messages) didn't overlap the stored rows, so a 10,000-message chat was left with 50 rows and re-paged from the network on scroll: 99 pages to get back to 5,000 rows.

flows1000x10000(), rows in the opened chat afterwards:

Missed Before (641184ff0) After (37a16862d)
49 10000 → 10049 10000 → 10049
50 10000 → 50 10000 → 10050
500 10000 → 50 10000 → 10050
2000 10000 → 50 10000 → 10050

The open's load step also gets cheaper, since it no longer deletes 10,000 rows: median 24.5 / 24.5 / 25.9 ms before and 11.8 / 10.4 / 10.8 ms after at 50 / 500 / 2,000 missed. The open total at 50 missed goes from 34.4 to 22.3 ms. These are medians of 3 warm runs on simulator "Flipcash Tests chat-history-gap-keep-rows" (6E26A273-1904-4B92-A599-D77BAFAA9287, iPhone 17, iOS 27.0), dropping the first reading after each seed. The harness is the orchestrator's benchmarks/chat-load, uninstalled before commit.

Re-fetch cost is derived, not timed: filling a gap of N missed messages takes ceil(N/50) + 1 pages (the last one overlaps stored rows), so 11 pages at 500 missed, against 99 pages to rebuild 5,000 rows before. repageReadExtension() wipes rows with raw SQL, so it only shows the per-page cost didn't move: persist total over 99 pages 914 ms before, 867 ms after, one run each.

What changed

  • A non-overlapping newest page records a gap (conversation_history_gap: the newest stored id below it, the oldest id of the page) instead of deleting rows. Schema 65 → 66.
  • Every id-ranged transcript read stops at the gap's upper bound: messagesWindow, messages(from:), messages(after:through:), olderAnchor, messageIDs, oldestMessageID, newestMessageID(through:), inboundMessageCount, firstInboundMessageID. Without that, the 60-row opening window would stitch 10 rows from below the gap onto the 50-row page. The unread divider and badge give the same answers they gave after a wipe.
  • loadOlderMessages already pages before oldestMessageID, which is now the gap's upper bound, so scrolling up fills the gap. persistOlderPage writes each page and moves the bound in one transaction: the gap closes when a page reaches the stored rows, and an empty page drops the rows under the gap, since the server holds nothing older.
  • A second miss before the fill finishes drops the run between the two gaps and keeps the oldest rows.
  • MessageLoader.reveal returns false for a stored message below the gap, so revealPaging pages the gap closed before jumping to it. isEntireHistory is false while a gap exists, so the encryption marker doesn't sit on top of a partial history.

The fill uses GetMessages paging rather than GetDelta forward from the newest stored row. eventSequence is a message's current-state version (Schema.swift:438): an edit or reaction bumps it on an old id, so a stored row's sequence doesn't mark where the gap starts. And loadMessages has already moved the cursor to the head by then.

The gap lives in its own table and is read from the database on each query, not mirrored in ConversationStore. Every read it clamps is already a database read.

Known limits

  • A GetDelta catch-up in flight while a gap opens can store rows inside it. Those rows stay hidden until paging back reaches them, which re-fetches pages already stored. This costs extra requests but loses nothing.
  • Chat search can now return rows below a gap. Opening one pages up to revealPageLimit (20 pages, about 1,000 messages); a wider gap leaves the transcript where it was.

Conflicts with #1235

#1235 is still open. Expect conflicts at the two former dropStaleEpochIfNeeded call sites (resyncAfterReset, loadMessages) and at the load-older persist. That write moves the gap bound, so the rows a window can read change. Under #1235 it needs the whole-conversation scope, not a per-id didPersist.

Opening a chat after missing 50 or more messages deleted every stored
row, because the newest page did not overlap them and an unmarked gap
would have rendered as contiguous. A 10,000-message chat was left with
50 rows and re-paged from the network.

Record the gap instead (schema 66, conversation_history_gap): keep the
rows below it, stop every transcript read at the gap's upper bound,
and fill it by paging GetMessages back from that bound until a page
reaches the stored rows. A second miss before the fill drops only the
run between the two gaps. An empty page below the gap drops the rows
under it, since the server holds nothing older.

The fill pages GetMessages rather than streaming GetDelta forward:
event_sequence is a message's current-state version (an edit or
reaction bumps it on an old id), so a cursor taken from a stored row
does not mark where the gap starts.
@bmc08gt bmc08gt self-assigned this Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant