Skip to content

csnode: backward sync never asks for more than its buffer holds - #145

Merged
EnzinoBB merged 1 commit into
masterfrom
fix/backfill-buffer-window
Oct 31, 2026
Merged

EnzinoBB merged 1 commit into
masterfrom
fix/backfill-buffer-window

Conversation

@EnzinoBB

Copy link
Copy Markdown
Collaborator

Problem

Seen on the mainnet test node (full rocksdb from the snapshot) after it restarted on master 9752d06: the first stores were slow (15-20 ms a block), the 50 ms turn budget (#140) stored a few blocks per turn, and the buffer filled: BACKFILL: 800 blocks wait to be stored: the others of … are dropped. The request window allowed up to window + one batch of blocks asked for and not stored (next_ - nextToRequest_ >= window checked before adding a batch), so a reply could overrun the buffer. The dropped blocks stayed missing until the batch timed out, which counted as a failure of the peer that had sent them; within 10 minutes 11 peers were "marked for 1 h" and the backfill fell from ~544 to ~30 blocks/s (no neighbour can serve … waiting).

Change

  • A batch is requested only if it fits: next_ - nextToRequest_ + batch > window stops.
  • Blocks a reply brings when the buffer is full anyway are put back in the retry queue at once and removed from the batch's missing set, so the batch counts as answered and the peer is not blamed.
  • Test SlowStoresNeitherDropBlocksNorMarkPeers: slow stores (12 ms simulated) with a 30 ms budget, batch 20: no drop, no mark, done (fails on master: the window overran the buffer).

Test plan

  • CI build and unit tests (Linux, Windows, macOS)
  • backfill robustness rows
  • the mainnet test node on this build

🤖 Generated with Claude Code

The request window let the blocks asked for and not stored reach the
buffer plus one batch, so with slow stores a reply could meet a full
buffer. Its extra blocks were dropped and left missing until the batch
timed out, which counted against the peer that had sent them: on
mainnet, after a restart with slow first stores and the 50 ms turn
budget, most peers were left out for an hour and the backfill fell to
about 30 blocks/s. A batch is now asked for only when it fits, and
blocks without room are asked for again at once without blaming the
peer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EnzinoBB
EnzinoBB merged commit 374e661 into master Oct 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant