Skip to content

[snapshot] Archive format version 4: commit to each child's metadata checksum - #495

Draft
duckhawk wants to merge 2 commits into
mainfrom
feat/snapshot-archive-format-v4
Draft

duckhawk wants to merge 2 commits into
mainfrom
feat/snapshot-archive-format-v4

Conversation

@duckhawk

Copy link
Copy Markdown
Member

Description

Adds snapshot archive format version 4 to d8, and pins d8's implementation of the format to the reference vectors of state-snapshotter.

  • Reference vectors.
    • internal/snapshot/archive/testdata/snapshotarchive/ is a byte-for-byte copy of the snapshotarchivetest vectors of the state-snapshotter api/snapshotarchive package. The package is the shared definition of the format, and d8 cannot import it because the module is not publicly fetchable.
    • A README there names the source commit and the rule: the files are refreshed only by copying them again.
    • vectors_test.go runs the version 3 and version 4 trees through the local and the upload verification paths. Every node's checksum, childrenChecksum and metadataChecksum must equal the reference, rewriting a descriptor must reproduce its bytes, and every tampered version-4 descriptor must fail from the root.
    • The first commit adds the version 3 part, which passes against the current code unchanged.
  • Version 4.
    • ChildCommitment carries the child's MetadataChecksum and FormatVersion; at version 4 the child's metadataChecksum.hex is the eighth length-prefixed field of each child record.
    • A version-4 parent requires every direct child at version 4 or later. Parents at 0, 2 and 3 verify with the old encoding.
    • MarshalJSON keeps a declared version 2, 3 or 4; the current version is 4.
  • Writers.
    • The pipeline already finalizes nodes bottom-up.
    • The publication transaction fixes every node's children checksum before any node is written. It now predicts and seals each published node's descriptor up front, so it knows each child's metadataChecksum, and writes those same descriptors.
    • A completed node below version 4 is republished (the payload is not downloaded again), so that its version-4 parent can commit to it.

Why do we need it, and what problem does it solve?

Up to version 3 a parent's childrenChecksum committed to a child's identity, content checksum and children checksum, but not to the rest of its descriptor: the volume entry (StorageClass, size, volume mode, target and artifact), sourceObjectRef, sourceName and formatVersion. An edit of a child's volume entry with the child's own metadataChecksum resealed still verified from the root. In version 4 the root's metadataChecksum authenticates every descriptor of the tree.

The same change goes into the state-snapshotter package (fox state-snapshotter MR !162), which the backup module uses. The shared vectors keep both implementations producing identical digests.

What is the expected result?

  • d8 snapshot download writes version 4 archives.
  • Archives at versions 0, 2 and 3 are still read and verified.
  • A tampered child descriptor in a version-4 archive fails verification.

Compatibility:

  • A d8 built before this change refuses version-4 archives: d8 snapshot upload and d8 snapshot local describe/get fail with "unsupported snapshot.yaml format version: 4". An older d8 resuming a download over a finished version-4 archive also fails.
  • Rerunning or resuming a download over an archive written by an older d8 rewrites its snapshot.yaml files at version 4; payloads are untouched.
  • An interrupted publication left by an older d8 (.d8-snapshot-publication-v1.json) is refused with "publication transaction checksum mismatch", and has to be removed or rerun into a fresh directory. The transaction version is not bumped, because completed receipts share it and a bump would fail every rerun over an archive finished by an older d8.

Verified locally:

  • go test ./internal/snapshot/... passes.
  • The full go test with the dev build tags passes in 113 of 114 packages. The one failure is internal/strongholddiscovery TestTokenConfigured/none, and it comes from a ~/.vault-token file on the machine, not from this change.
  • golangci-lint (v2.13.2) reports 0 issues on internal/snapshot/....
  • Mutation checks: dropping or moving the new field, removing the child version rule, or restamping in MarshalJSON makes the tests fail.

Draft until the state-snapshotter change is merged: the vectors are copied from its branch (commit 9866ab7b), and the README should name the merged commit.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

…vectors

The snapshot archive format now has its reference implementation in
state-snapshotter (api/snapshotarchive). d8 cannot import it, since a
public build cannot fetch that module, so it keeps its own
implementation and is checked against the reference vectors instead.

testdata/snapshotarchive holds those vectors, copied byte for byte from
state-snapshotter commit 9866ab7b: a five-node archive at format
versions 3 and 4, tampered version 4 descriptors, and two stand-alone
descriptors with the exact canonical JSON their metadata checksum is
taken over. The README records the source and the rule that the files
are only ever refreshed by copying them again.

vectors_test.go runs the version 3 tree through the local and the
upload verification paths, compares every node's checksum,
childrenChecksum and metadataChecksum with expected.json, and requires
rewriting each descriptor to reproduce it byte for byte. The version 4
vectors are not exercised yet: d8 does not read that version.

Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
… version 4

A parent's childrenChecksum committed to a child's identity, checksum
and childrenChecksum only. The child's formatVersion, sourceName,
sourceObjectRef and volumes entry were covered by the child's own
metadataChecksum alone, so whoever could rewrite a child could change
its volume entry, reseal it, and the archive still verified from the
root. This ports format version 4 of the state-snapshotter reference
(api/snapshotarchive, commit 9866ab7b) with the same encoding:

- ComputeChildrenChecksumAt hashes the child's metadataChecksum hex as
  the eighth length-prefixed field of every child record at version 4;
  the domain string is unchanged. A version 4 parent requires every
  direct child at version 4 or later with a well-formed
  metadataChecksum. The parent's own version selects the encoding, so
  archives at versions 2 and 3 verify exactly as before.
- ChildCommitment carries the child's MetadataChecksum and
  FormatVersion; ChildCommitmentOf fills them from a snapshot.yaml.
- MarshalJSON keeps a declared version 2, 3 or 4 and stamps the current
  version only on a snapshot.yaml without one, so a descriptor read
  from an archive encodes back unchanged. SealSnapshotYAML returns what
  a writer records.
- The current version becomes 4, so every snapshot.yaml d8 writes is
  at version 4.

The download pipeline already finalizes children before their parent.
What changes is the publication transaction, which fixes every
published node's digests before any node is finalized: it now also
predicts each published node's sealed snapshot.yaml
(volume.DescribeNodeRootedContext) and records its metadataChecksum,
which the parent commits to. The run that built the transaction writes exactly
those descriptors instead of measuring every payload twice; a run that
resumes a transaction from disk finalizes as before and checks the
result against the recorded metadataChecksum. A transaction file
written by an older d8 lacks that field and is refused on load.

A completed node at an older version is republished at version 4 on a
rerun, since a version 4 parent cannot commit to it. Resuming or
rerunning a download over an archive written by an older d8 therefore
rewrites its snapshot.yaml files bottom up; the payloads are not
downloaded again.

The reference vector tests now cover the version 4 tree, the tampered
descriptors, which must fail at their parent when the tree is verified
from the root, and mixed versions: a version 4 root over version 3
children is refused, a version 3 root over version 4 children
verifies, and a version 3 commitment relabelled as version 4 fails.

Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
deckhouse-BOaTswain pushed a commit that referenced this pull request Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant