Skip to content

snapshots: node --survey-snapshots and the weekly snapshots.md / snapshot_sources.txt (P8) - #116

Merged
EnzinoBB merged 3 commits into
masterfrom
feat/snapshot-survey
Oct 27, 2026
Merged

EnzinoBB merged 3 commits into
masterfrom
feat/snapshot-survey

Conversation

@EnzinoBB

Copy link
Copy Markdown
Collaborator

Summary

Plan step P8 of the p2p snapshots: a way to see who serves which snapshot, and the weekly job that publishes it.

  • node --survey-snapshots [seconds] (default 600, 10 to 86400): the node starts without opening its chain (whatever its folder holds), asks the neighbours that announce snapshots for their manifests for that long, prints what they declared and exits 0. It is the bootstrap's discovery and manifest phases with the fetcher in dry-run; nothing is fetched or imported. Output, one line per snapshot (strongest first), then the serving peers that gave no manifest:
    SURVEY group seq=… head=… state=… settings=… sources=<key>@<address>@<created>,…, SURVEY silent <key>@<address>, SURVEY end groups=… serving=… silent=….
  • ci/known-hosts/snapshot_sources.py turns that output and known_peers.json into snapshots.md (the snapshots seen, their sources, how many independent networks serve: what the min_sources decision depends on) and snapshot_sources.txt (the serving peers that answered a verified manifest, at most 3 per /24 or /48). The output comes from a node that talked to untrusted peers, so every field is checked against its exact shape (keys, IP addresses, 64-hex hashes, ISO dates) before it reaches a file or a pull request; --self-test covers hostile input.
  • The weekly job (known-hosts.yml, run-weekly.sh) runs the survey after its collection, publishes both files in the known-hosts release and adds snapshot_sources.txt to the known-hosts pull request only when it found serving nodes. make-package.sh and the Docker entrypoint ship the file when the repository has it.
  • Normal starts are untouched: the survey is a separate mode behind the new option; startBootstrap only moves its hook set into a shared makeFetcher.

Test plan

  • Unit test of the report (SnapshotFetcher.SurveyTextListsTheGroupsAndTheSilentSources), snapshot_sources.py --self-test
  • Linux build and tests
  • Private network, row snapshot-p2p-survey (at the end of a p2p join the joiner is stopped and surveyed): SURVEY end groups=1 serving=3 silent=0, the three serving nodes in one group and in the generated sources file
  • This PR's CI

🤖 Generated with Claude Code

EnzinoBB and others added 3 commits October 27, 2026 09:40
…and snapshot_sources.txt

The node asks its neighbours for snapshot manifests for N seconds without opening its chain, prints
the snapshots they declared (one line per group with its sources) and the serving peers that gave no
manifest, and exits 0. The weekly known-hosts job runs it after the collection and
ci/known-hosts/snapshot_sources.py (strict parsing: the output comes from a node that talked to
untrusted peers) writes snapshots.md and snapshot_sources.txt, published in the known-hosts release
and proposed with the known hosts; the packages ship snapshot_sources.txt when the repository has it.

- SnapshotFetcher::surveyText, Node::startSurvey/finishSurvey (the bootstrap's discovery and manifest
  phases with the fetcher in dry-run), the --survey-snapshots option
- robustness row snapshot-p2p-survey (JOIN_SURVEY=1): at the end of a p2p join the joiner is stopped and
  surveyed; the three serving nodes must be in one group and in the sources file

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@EnzinoBB
EnzinoBB merged commit e309122 into master Oct 27, 2026
11 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant