Repository navigation
Conversation
The memory columns (three limbs and the final read counts) were committed over the whole 2^log_mem cube, so a run using 2^22.4 cells committed 2^23 of each. The prover now announces N_mem, 2^16 <= N_mem <= 2^32 with at most three set bits, and commits each memory column on its first N_mem cells only, a piece per set bit, largest first. The cells from N_mem to 2^kappa_mem, kappa_mem = log2_ceil(N_mem), are public: word 0, final count g^0.
c(r) = sum_i e_i(r) * P_i(r_{<k_i}) + pad * pad_N(r), e_i(r) = eq(r_{>=k_i}, bits(o_i >> k_i))
- Protocol: announced word 0 is N_mem (was log_mem); every verifier checks its range and popcount (AnnouncedTableSizes). A memory claim is one jagged stack claim; the final counts' claim moves pad_N(zeta) into its value. The bus is unchanged: a tail cell's seed and finalization carry the same tuple and cancel, and a read there is excluded (new lem:tail; cor:addrrange holds with i < N_mem). The identity is exact and the claim count unchanged, so no new soundness term. TAG and the Fiat-Shamir labels stay v1.
- verifier and verifier.py: MAX_MEMORY_PIECES = 3, `Summary::memory_cells`.
- prover: N_mem is the least cover of the cells used with pieces of at least 2^10 cells (three, or fewer through the hidden test knob `Program::with_memory_pieces`), or, below `min_log_committed`, the least such cover reaching it, found by bisection. The memory columns' sources give their pieces and pad, so the layout and the leaves follow from them.
- guest: `memory_pieces` pins the hinted piece logs and presence bits to the announced word (distinct logs below 33 summing to it bit by bit, k_0 >= 16, log_mem <= 32); the placement certifies the pieces (an absent one takes no room and gets no selector), each entry's piece read off a baked COL_MEM_PIECE; the terminal weighs each piece by its coefficient e_i(zeta), computed in circuit; the final counts' target takes the pad share. The guest and its hints are written for three pieces, which a Rust const assert and a compile-time guest assert hold, and a memory claim at any point but the two it has fails to compile.
- doc: N_mem announced, kappa_mem = ceil(log2 N_mem), columns in pieces (eq:jagged), the public tail (lem:tail), the PI claim, the range check, Annex B.
Tests: build_layout's bounds and popcount (both bounds, 3 pieces, 0, 2^16 - 1, 2^32 + 1, 4 pieces); a jagged column evaluates and builds leaves as its padded dense column; the cover policy; one program proved with 1, 2 and 3 memory pieces through both verifiers; a word-0 tamper to four pieces rejected by both as AnnouncedTableSizes; the guest's pinning and scalars against native for 1 to 3 pieces and both bounds, with eight forged hints each rejected only by the check it targets (every check shown to bite by removing it); a node rejecting forged piece hints.
Measured on Ryzen 9 9900X (shared, load 15 to 29), interleaved A/B against origin/main (df7d122), 6 pairs of --repeat 3:
cargo leanvm aggregate --xmss 900 --log-inv-rate 1
committed 2^26.050 -> 2^25.814 (69.5M -> 59.0M words, -15%), M 27 -> 26
proof size 321.7 -> 349.0 KiB (+8.5%): queries unchanged, a level-0 row carries the lanes holding data, 34 -> 57
proving time 1.182 -> 1.054 s median (-10.8%; per-pair median -12.3%, spread -18.5% to -9.5%)
peak memory 3.93 -> 3.99 GiB (+1.4%; per-pair +1.3% to +3.0%)
verify time 8.38 -> 8.51 ms (noise)
cargo leanvm recursion --n 2 --xmss-per-leaf 900 --log-inv-rate 2 (children at M 26 instead of 27)
node cycles 438,675 -> 437,680 (-995)
guest (DBG_PROF, both children): verify_sub 2,668 -> 2,690, check_opening_terminal 3,570 -> 3,890, certify_placement 962 -> 1,526, new memory_pieces 142 + memory_piece_coefficients 128 + one_plus_chain 78, the final counts' pad 408; the cheaper opening pays for it
committed 2^23.842 -> 2^23.738
proof size 232.4 -> 229.7 KiB (-1.2%)
peak memory 1.45 -> 1.36 GiB (-6.0%)
proving time 0.388 -> 0.406 s median, per-pair -5.8%, spread -14% to +44%: noise
verify time 10.8 -> 10.8 ms (noise)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
@TomWambsgans I know we discussed this couple of times, not sure if this is worth merging, I just wanted to open this to explore and see if we want this. |
Collaborator
|
I think we should keep this as an option in case, in the future, it turns out on the typical load (like 1400 XMSS and 16->1 recursion) we are just above a power of 2. The gain of this kind of opti is very dependent if it's 900, or 1400 XMSS etc. So my opinion: ideally we will not need it. But it's good to keep it in mind, as a lever to go bellow power of 2 if needed in the future |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #717 (stacking in pieces, no protocol change). Merging this PR brings it along.
Summary
Protocol change, landed in the prover,
crates/verifier,python-verifier/verifier.py, the recursive guest anddoc/leanvmtogether. The memory size is now announced as a number of cellsN_meminstead of a log: 2^16 ≤ N_mem ≤ 2^32, at most 3 set bits. The four memory columns are committed only on cells[0, N_mem), as up to three aligned power-of-two pieces. The cells[N_mem, 2^kmem)are public (value 0, final count g^0 = 1), so nobody commits the tail of zeros that a power-of-two memory used to pad to.aggregate --xmss 900 --log-inv-rate 1, 6 interleaved pairs ×--repeat 3recursion --n 2 --xmss-per-leaf 900 --log-inv-rate 2The new in-circuit work is about 0.75k cycles over the two children: the piece pinning, the piece coefficients and the final-count pad share. Each child's cheaper opening pays for it.
Why the proof grows. The query counts are the same at M = 26 and M = 27, but each level-0 opening row carries every lane that holds data:
ceil(placed / 2^(M−6))= 57 lanes at M = 26 against 34 at M = 27. No prover-side placement choice avoids this, because every verifier derives the placement from the announced sizes. The prover's one free size isN_mem: announcing about 2.0M more cells (throughmin_log_committed) lands back at M = 27 with main's proof size and main's commit cost. That is a proof-size versus proving-time policy, with no further protocol change, left for a follow-up.The construction (doc: body/04
eq:jagged, body/06lem:tail)For a virtual column with committed extent
R = sum_i 2^(k_i)(k_0 > k_1 > k_2), piece offsetso_i, and pad valuepadbeyond R:so a claim
c^~(r) = vis one stack claim with that weight and valuev − pad · padw_R(r)(lem:suffix).cor:addrrangetherefore holds with i < N_mem.Soundness
No new error terms.
What is new is deterministic, and every piece of it has a negative test:
N_mem.Checks
announced_memory_is_bounded_and_at_most_three_pieces;memory_covers_the_cells_used_in_at_most_the_pieces_asked;memory_in_one_two_or_three_pieces_verifies_in_both(Rust and Python);AnnouncedTableSizesin both verifiers);guest_memory_pieces_match_native(8 forged hints; removing each of the 7 pinning checks one at a time turns it red);forged_memory_piece_hints_are_rejected(node level).aggregate_two_to_oneandaggregate_three_levelspass. The latter verifies a two-piece child in-circuit.replay_uncheckedon all 4,777 acceptedN_memvalues with forged streams, with no panic. A probe found proofs byte-identical before and after the review rewrite, at 1, 2 and 3 pieces.cargo test --release -p verifier -p leanvm_core -p pcs -p rec_aggregation, workspace clippy-D warnings, ruff: pass.cd doc/leanvm && latexmk -pdf main.tex.🤖 Generated with Claude Code