Repository navigation
snapshot: p2p fetcher, agreement and download (P4) - #104
Merged
Merged
Conversation
snapshot_agreement: netPrefix (/24, /48) and agree() over the collected manifests (design 6.3): groups by (seq, head) and digests, highest block first, trusted keys counted by key, shortfall for the waiting line. SnapshotFetcher: discovery of serving peers and configured keys, manifest requests every 30 s, agreement at each tick, download of the agreed snapshot's files piece by piece (2 per source, 8 in total, 30-s timeout, busy, marks, bad pieces, group switch, whole-file check, rate cap), fetch_wait with sync/wait, import failures, dry run, stop. Single-threaded, hooks for the network and the clock. Node forwards the two reply messages to snapshotFetcher_ when it exists; nothing constructs it yet (P5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rce table Agreement: among the groups that reach min_sources, a group with a configured key first, then the most independent sources, then the highest block (a tie-breaker only), so that min_sources forged sources at a made-up later block lose to a larger honest group; equal groups are a conflict. Every group not chosen carries the reason, logged with the minority. Source table: configured keys (sources=, sources_file) have their own room; discovered peers are limited to 4 per /24 (/48) and 64 in all; when full, the oldest entry of the most represented prefix makes room, never one with pieces outstanding or in the agreed group during a fetch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e manifest Source table: a full table always makes room, whatever the prefixes: the least useful discovered source goes first (not a neighbour, silent for a minute, no manifest after a minute, then the most represented prefix); discovered sources silent for a minute are let go, and an evicted key is not taken again for 10 min, so that returning keys cannot churn the table. Discovered sources are also limited to 16 per /16 (/32 IPv6). Addresses: an address not known yet is not capped at discovery; it is resolved again when the manifest arrives, and the per-prefix limits and the agreement count both use that address. A source whose address is unknown does not count towards agreement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o symlinks, mode 0600 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bans, logs) - Silence: counted from the first unanswered request (two requests over more than 35 s), not from the discovery; asked again after 10 s twice, then every 30 s; no proactive eviction: silence is only an eviction tier when room is needed; the cool-down after an eviction is 60 s. - Busy: delay floored at 500 ms (cap 10 s); busy for a minute without a piece leaves the source alone for a minute, without counting to a ban. - A fetch always ends: no new byte for 5 min returns to Collect, and every return to Collect counts as an attempt with the refused imports (3 -> Fallback). - A refused import bans every agreed source that declared the refused (name, sha256); without a file at fault, the source that signed the manifest. - A manifest reply over the prefix limits no longer erases a source in use. - Agreement recomputed only when its input changed; filesHash once per source when choosing the files. - Whole-file SHA-256 at most 64 MiB per tick. - Logs: the 30-s line names a conflict, the manifests not counted (address unknown) and the replies dropped; the fallback lines name CREDITS_SNAPSHOT=github; fetch_wait=0 documented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e a sync from 0 Piece timeout: 30 s plus the bytes outstanding to the source at 64 KB/s. A source that timed out but delivered a piece in the last 60 s is not marked: its piece is asked elsewhere and it gets one piece at a time until it delivers two in time; its late reply is still taken (hash-checked) while the piece is missing. A source that never delivered keeps the mark. Attempts: refused imports and failed fetches are counted apart. A fetch that ends with sources banned or marked in it goes back to Collect without counting; other failed fetches count, 3 -> Fallback with fetch_fallback=sync, never with wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d bootstrap A failed fetch is not counted only when a source was banned in it for bytes that do not match the signed manifest; timeouts, busy and not-serving answers no longer excuse it. Excused aborts have their own cap (10, then Fallback with fetch_fallback=sync). With sync the whole bootstrap ends in Fallback at fetch_wait + 30 min whatever the counters say; with wait it keeps going. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…view fixes - With fetch_fallback=sync the bootstrap cap grows by the agreed snapshot's download time: at fetch_rate_kbps when set, else at 64 KB/s for at most 2 h; a warning at the start of the fetch when the expected time exceeds what is left. - With fetch_fallback=wait refused imports go back to Collect instead of Fallback (the declaring sources are banned). - Late pieces: stored with their send time; for one more timeout they count towards their source's outstanding pieces; the source is not given the same chunk again while another is free; the oldest entry is evicted. - An in-flight piece filled meanwhile by another source's late reply is dropped at expiry without marking anyone. - Tests: the configured rate beyond 30 min, refused imports with wait, the re-assignment race, a symbolic link in the download folder. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EnzinoBB
marked this pull request as ready for review
October 25, 2026 16:02
EnzinoBB
added a commit
that referenced
this pull request
Oct 25, 2026
Step P5 rewritten against the code merged through #104 and PR #103 (official even nodes in CI): the init split into initCommon/initChain/initRunning, the gates (chooseMessageAction, onNeighbourAdded, onPingChecked, onRoundTimeElapsed), the worker thread for the import and initChain with the hand-over through CallsQueue, the fetcher hooks, the hash databases released before the import, the interrupted-import marker, the start-up verification with exit code 4, the run.sh variables and the robustness rows with official even nodes, and the risks with a test for each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step P4 of
docs/plans/2026-10-25-snapshot-p2p-plan.md. Based on #102 (P3): until #102 is merged the diff includes its commits.What it does
snapshot_agreement.hpp/.cpp:netPrefix(address)(/24 for IPv4, /48 for IPv6, IPv4-mapped counted as IPv4, port ignored) andagree(sources, minSources). Sources are grouped by(seq, head_hash, state_digest, settings_digest)and counted by address prefix; trusted keys (sources=) count once each by key; a source whose address is unknown does not count. Among the groups that reachmin_sources, the choice goes in this order: a group holding a configured key, then the most independent sources, then the highest block, which only breaks ties. A later block alone never wins over more support, somin_sourcesforged sources at a made-up later block lose to a larger honest group. Two groups equal on all three are a conflict, and nothing is chosen. Every group that is not chosen carries the reason, which is logged. This differs from the design's 6.3 rule "highest seq first" (a security review finding).snapshot_fetcher.hpp/.cpp:cs::SnapshotFetcher, single-threaded, everything external throughHooks(send manifest/piece request, add neighbours, address lookup, ms clock, optional log sink).sources=keys andsources_filekeys are passed toaddNeighbours(once per key, again after 60 s if the key is still not a neighbour). Configured keys have their own room in the source table. Discovered peers are limited to 4 per /24 (/48), 16 per /16 (/32) and 64 in all. A full table always makes room, removing the least useful entry first: not a neighbour, then silent (two requests unanswered over more than 35 s, counted from the first request; silence is a reason only when room is needed), then no manifest after a minute, then the most represented prefix. It never removes a configured source, one with pieces outstanding, or one in the agreed group during a fetch. An evicted key is not taken again for 1 min. A request without an answer is repeated after 10 s twice, then every 30 s (design 5.3a). An address that is not known yet is not capped at discovery: it is resolved again when the manifest arrives, and both the limits and the agreement count use that address; every serving neighbour is asked for its manifest every 30 s (no answer means ask again 30 s later); a manifest is kept only iffromJson,verify(),node_key == senderand the chunk size (power of two, at most 1 MiB) all pass;fetch_waitruns out,syncgoes toFallbackwith the design's warning line, andwaitkeeps waiting with a line every 5 min.retry_ms(at least 500 ms, at most 10 s), and busy for a minute without a piece leaves the source alone for a minute; not serving, unknown or timeout marks the source for 60 s, and a second mark keeps it out for the rest of the run; a bad chunk keeps the source out for the rest of the run and the piece is requested elsewhere; a reply that does not match an outstanding request is dropped and counted. Pieces are written at their offset into pre-sized files. When a file has no usable source left, the fetcher switches to another file group, and goes back to Collect if none is left. A fetch always ends: with nothing to ask for 2 min, or no new byte for 5 min, it goes back to Collect, and counts as a failed fetch unless a source was banned in it for bytes that do not match the signed manifest (timeouts, busy and not-serving answers do not excuse it). Withfetch_fallback=sync: 3 failed fetches, or 10 excused ones, →Fallback, and the whole bootstrap ends inFallbackatfetch_wait+ 30 min + the agreed snapshot's download time (atfetch_rate_kbpswhen set, else at 64 KB/s for at most 2 h) whatever the counters say; withwaitit keeps retrying. The whole-file SHA-256 is checked from the next tick, at most 64 MiB per tick; on a mismatch the file is reset and the fetcher goes back to Collect.fetch_rate_kbpslimits the bytes requested; there is a progress line every 10 % and every 30 s.Import:result()returns the manifest and the agreed sources.onImportFailed(file)refuses that file's(name, sha256)and bans every agreed source that declared it (with no file at fault: the source that signed the manifest), keeps the files already verified (they are re-checked) and goes back to Collect; after 3 refused imports it goes toFallbackwithsync, back to Collect withwait. Agreement is recomputed only when its input changed. The 30-s line names a conflict between equal groups, the manifests not counted (address unknown) and the replies dropped; the fallback lines nameCREDITS_SNAPSHOT=github.dryRunstops atDoneafter agreement or afterfetch_wait.stop()clears everything outstanding.Node::getSnapshotManifestReply/getSnapshotPieceReplyforward tosnapshotFetcher_when it exists, otherwise they drop and count as before. Nothing constructs the fetcher in this step (P5 will), so a running node behaves the same.Tests
snapshot_agreement_tests(16): netPrefix/netBlock forms, threshold,min_sources=1, same /24 counted once, trusted keys counted separately, IPv6 /48, unknown addresses do not count, duplicate key, 2 forged sources at S+1000 lose to 3 honest at S, equal support goes to the higher block, a configured key's group wins (but not belowmin_sources), only the lower block reaches the threshold, digest majority with the minority reported, tie as a conflict, different heads, shortfall.snapshot_fetcher_tests(36): realSnapshotProviderinstances as sources, serving five snapshots built bybuildSnapshotfrom one generated chain. A1 and A2 are identical; B and E have one or two trailing bytes in walletscache; D has another digest. They cover all 15 cases of the plan: (1) discovery, (2) bad signature, relayed manifest, oversized chunk and an unsolicited reply, (3) 3 of 4 agree and the minority is logged, (4) piece spread and the 2-per-source / 8-total limits, (5) a corrupt piece, (6) a stalled source, (7) busy, (8) a source that leaves the neighbours, (9) a group that runs dry, (10) the whole-file mismatch, (11)fetch_waitwithsyncand withwait, (12) import failures over three groups and then Fallback, (13) dry run, (14) stop in each phase, (15) the rate cap at 200 KB/s. Two more tests cover the source table: 100 discovered keys from one /24 push out neither a configured key nor the sources of other networks, a full table makes room for new networks, 64 silent keys over 64 networks do not lock a newcomer out, and unknown addresses are not capped at discovery, have no vote until they are known, and are capped by the address resolved with the manifest. Further tests: a late neighbour and a source silent for its first two requests are kept; busy with no delay is not asked again before 500 ms; sources busy forever end each fetch after 5 min and the bootstrap after 3 withsync, never withwait; three waves of sources serving bad pieces are banned without reaching Fallback and an honest group completes; sources that only time out cannot prevent Fallback withsync; ten bad-data aborts end withsync; a source keeping the fetch alive with one piece every 4 min hits the cap, while an honest join limited byfetch_rate_kbpspast 30 min completes; refused imports withwaitkeep looking; a slow and a fast source race without blame; a symbolic link in the download folder ends in Fallback; a slow downlink marks no source; slow sources go to one piece at a time and their late pieces are taken; a refused import bans the declaring sources, or the signer when no file is at fault; the conflict line.🤖 Generated with Claude Code