Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Durable commits currently send and receive a channel message for every written page. Replace these with incremental write groups: requests still submit immediately, but callers receive one completion per group after all producers close and every request finishes.
Use groups for leaf and branch workers, overflow pages, freelists, and hash-table writeout. Linux workers submit replacement requests before releasing completed write buffers; the synchronous fallback processes bounded batches before cleanup. Reads retain individual completions.
Groups retain the first error and drain outstanding requests before reporting it. Completion guards also drain on drop, with callers ordered to retain raw buffer owners until completion. Existing node ownership stays unchanged; the low-level API introduces no B-tree types or buffer traits. The accounting counter uses relaxed atomics; errors use a mutex and the final result uses a channel. Hash-table write errors now propagate before fsync.
Measurements
Seeded random writes against a 2^25-item DB; 25k updates/batch, 4 workload threads, 8 commit workers, 3 I/O workers. Fixed 350 measured batches after 1M warmup updates per run, ABBA order, separate identical starting copies, affinity CPUs 0–15. Ryzen 9 7950X / Corsair MP600 PRO LPX. Runs lasted 116–146 seconds.
Aggregate elapsed time: 284s → 260s (~9% lower); total CPU ~10% lower. Individual mean batch times in ABBA order: 416, 331, 411, 397ms. SSD latency remained unstable after three 180-second conditioning passes, so the CPU reduction is stronger evidence than the precise wall-time improvement. Two repeats per variant do not establish a tight confidence interval.
Every run wrote 19,054,225 pages; all final roots matched and survived reopening. An isolated no-disk pipeline using the production completion APIs reduced total CPU by 21% with burst completions and 46% with paced completions; that test excludes actual I/O and buffer deallocation.
Both measured variants included the length/decoder changes from #985, with identical benchmark instrumentation. This PR is independently based on master; standalone performance was not remeasured. Temporary instrumentation, harnesses, and research outputs are excluded.
Validation
cargo fmt --all -- --checkcargo test -p nomt