Repository navigation
Conversation
…vectors The snapshot archive format now has its reference implementation in state-snapshotter (api/snapshotarchive). d8 cannot import it, since a public build cannot fetch that module, so it keeps its own implementation and is checked against the reference vectors instead. testdata/snapshotarchive holds those vectors, copied byte for byte from state-snapshotter commit 9866ab7b: a five-node archive at format versions 3 and 4, tampered version 4 descriptors, and two stand-alone descriptors with the exact canonical JSON their metadata checksum is taken over. The README records the source and the rule that the files are only ever refreshed by copying them again. vectors_test.go runs the version 3 tree through the local and the upload verification paths, compares every node's checksum, childrenChecksum and metadataChecksum with expected.json, and requires rewriting each descriptor to reproduce it byte for byte. The version 4 vectors are not exercised yet: d8 does not read that version. Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
… version 4 A parent's childrenChecksum committed to a child's identity, checksum and childrenChecksum only. The child's formatVersion, sourceName, sourceObjectRef and volumes entry were covered by the child's own metadataChecksum alone, so whoever could rewrite a child could change its volume entry, reseal it, and the archive still verified from the root. This ports format version 4 of the state-snapshotter reference (api/snapshotarchive, commit 9866ab7b) with the same encoding: - ComputeChildrenChecksumAt hashes the child's metadataChecksum hex as the eighth length-prefixed field of every child record at version 4; the domain string is unchanged. A version 4 parent requires every direct child at version 4 or later with a well-formed metadataChecksum. The parent's own version selects the encoding, so archives at versions 2 and 3 verify exactly as before. - ChildCommitment carries the child's MetadataChecksum and FormatVersion; ChildCommitmentOf fills them from a snapshot.yaml. - MarshalJSON keeps a declared version 2, 3 or 4 and stamps the current version only on a snapshot.yaml without one, so a descriptor read from an archive encodes back unchanged. SealSnapshotYAML returns what a writer records. - The current version becomes 4, so every snapshot.yaml d8 writes is at version 4. The download pipeline already finalizes children before their parent. What changes is the publication transaction, which fixes every published node's digests before any node is finalized: it now also predicts each published node's sealed snapshot.yaml (volume.DescribeNodeRootedContext) and records its metadataChecksum, which the parent commits to. The run that built the transaction writes exactly those descriptors instead of measuring every payload twice; a run that resumes a transaction from disk finalizes as before and checks the result against the recorded metadataChecksum. A transaction file written by an older d8 lacks that field and is refused on load. A completed node at an older version is republished at version 4 on a rerun, since a version 4 parent cannot commit to it. Resuming or rerunning a download over an archive written by an older d8 therefore rewrites its snapshot.yaml files bottom up; the payloads are not downloaded again. The reference vector tests now cover the version 4 tree, the tampered descriptors, which must fail at their parent when the tree is verified from the root, and mixed versions: a version 4 root over version 3 children is refused, a version 3 root over version 4 children verifies, and a version 3 commitment relabelled as version 4 fails. Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
deckhouse-BOaTswain
pushed a commit
that referenced
this pull request
Nov 12, 2026
Signed-off-by: suselz <suselz@mail.ru>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds snapshot archive format version 4 to d8, and pins d8's implementation of the format to the reference vectors of state-snapshotter.
internal/snapshot/archive/testdata/snapshotarchive/is a byte-for-byte copy of thesnapshotarchivetestvectors of the state-snapshotterapi/snapshotarchivepackage. The package is the shared definition of the format, and d8 cannot import it because the module is not publicly fetchable.vectors_test.goruns the version 3 and version 4 trees through the local and the upload verification paths. Every node's checksum, childrenChecksum and metadataChecksum must equal the reference, rewriting a descriptor must reproduce its bytes, and every tampered version-4 descriptor must fail from the root.ChildCommitmentcarries the child'sMetadataChecksumandFormatVersion; at version 4 the child'smetadataChecksum.hexis the eighth length-prefixed field of each child record.MarshalJSONkeeps a declared version 2, 3 or 4; the current version is 4.Why do we need it, and what problem does it solve?
Up to version 3 a parent's
childrenChecksumcommitted to a child's identity, content checksum and children checksum, but not to the rest of its descriptor: the volume entry (StorageClass, size, volume mode, target and artifact),sourceObjectRef,sourceNameandformatVersion. An edit of a child's volume entry with the child's ownmetadataChecksumresealed still verified from the root. In version 4 the root'smetadataChecksumauthenticates every descriptor of the tree.The same change goes into the state-snapshotter package (fox state-snapshotter MR !162), which the backup module uses. The shared vectors keep both implementations producing identical digests.
What is the expected result?
d8 snapshot downloadwrites version 4 archives.Compatibility:
d8 snapshot uploadandd8 snapshot local describe/getfail with "unsupported snapshot.yaml format version: 4". An older d8 resuming a download over a finished version-4 archive also fails.snapshot.yamlfiles at version 4; payloads are untouched..d8-snapshot-publication-v1.json) is refused with "publication transaction checksum mismatch", and has to be removed or rerun into a fresh directory. The transaction version is not bumped, because completed receipts share it and a bump would fail every rerun over an archive finished by an older d8.Verified locally:
go test ./internal/snapshot/...passes.go testwith the dev build tags passes in 113 of 114 packages. The one failure isinternal/strongholddiscoveryTestTokenConfigured/none, and it comes from a~/.vault-tokenfile on the machine, not from this change.golangci-lint(v2.13.2) reports 0 issues oninternal/snapshot/....MarshalJSONmakes the tests fail.Draft until the state-snapshotter change is merged: the vectors are copied from its branch (commit 9866ab7b), and the README should name the merged commit.
Checklist