Skip to content

csnode, client: snapshot format 2, builder and --export-snapshot - #89

Merged
EnzinoBB merged 8 commits into
masterfrom
feat/snapshot-builder
Oct 24, 2026
Merged

EnzinoBB merged 8 commits into
masterfrom
feat/snapshot-builder

Conversation

@EnzinoBB

@EnzinoBB EnzinoBB commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Step 3a of snapshots served by community nodes. It depends on #87 (checkpoint digest) and #88 ([snapshot] config), both merged into this branch. Review only the commit "snapshot format 2, builder and --export-snapshot".

  • SnapshotManifest (snapshot.json): checkpoint block, head hash, state and settings digests, first block, files with whole-file and per-chunk SHA-256, Ed25519 signature of the building node. fromJson accepts only a well-formed format-2 manifest within the limits (1 MB JSON, 32 files, 8 GB, chunks 64 KiB to 16 MiB) whose file names are in the fixed table. A path never comes from the network.
  • buildSnapshot: the qs files of the checkpoint, blocks F..S read through the storage and checked as a hash chain ending at the checkpoint's head, contract data sorted by key. It writes through <S>.tmp and leaves nothing behind on failure or stop.
  • Compression: lz4 HC level 9 in independent 4 MiB blocks with the lz4 already in the tree. Measured on the mainnet checkpoint files it is 10–30 % above zstd -3: walletscache 12.7x vs 16x, blocks 1.83x vs 1.90x. No new library.
  • --export-snapshot <dir>: on a stopped node, builds the snapshot of the latest periodic checkpoint that has state_digest.bin, then exits.
  • readCheckpointHead is now public.

Nothing is served or fetched yet.

Tests: build and verify (files, chunks, blocks byte for byte, decompression), first block bounded by the storage, broken chain, wrong head, missing digest, stop, manifest tampering of every signed field, JSON round trip and rejections, file-name table.

🤖 Generated with Claude Code

EnzinoBB and others added 7 commits October 24, 2026 10:00
A snapshot of checkpoint S is built by the node from its own state: the
files of qs/S, the blocks F..S read through the storage as a hash chain
that must end at the checkpoint's head (F = S - snapshot_blocks, bounded
by the first stored block), and the contract data. Each file is lz4 HC in
independent 4 MiB blocks (no new library; within 10-30 % of zstd -3 on
the mainnet files), hashed whole and per chunk. snapshot.json carries the
block, head hash, state and settings digests, the files, and an Ed25519
signature of the node's key; fromJson accepts only a well-formed
format-2 manifest within the limits and with known file names.

--export-snapshot <dir> builds the snapshot of the latest periodic
checkpoint that has state_digest.bin on a stopped node and exits.

Step 3a of snapshots served by community nodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-export-snapshot

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The builder replaces or removes only folders it built (they hold
  snapshot.json) and refuses an existing folder that is not one, so a
  misdirected output (qs, db, ...) loses nothing.
- --export-snapshot refuses next to a running node, when the folder
  holds no database of the configured backend, and when the output is
  the node's folder or inside qs, db or caches.
- The VERSION text is reduced to plain characters before it goes into
  the manifest.
- The signed line covers the chunk size and every chunk hash; fromJson
  refuses block numbers above 32 bits and nesting deeper than 8 before
  parsing.
- The CI check reads only the lines a checkpoint save logs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EnzinoBB
EnzinoBB marked this pull request as ready for review October 24, 2026 10:24
@EnzinoBB
EnzinoBB merged commit 41e1386 into master Oct 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant