Skip to content

Batch write completions with incremental write groups - #986

Open
rphmeier wants to merge 1 commit into
masterfrom
batch-write-completions
Open

rphmeier wants to merge 1 commit into
masterfrom
batch-write-completions

Conversation

@rphmeier

Copy link
Copy Markdown
Contributor

Durable commits currently send and receive a channel message for every written page. Replace these with incremental write groups: requests still submit immediately, but callers receive one completion per group after all producers close and every request finishes.

Use groups for leaf and branch workers, overflow pages, freelists, and hash-table writeout. Linux workers submit replacement requests before releasing completed write buffers; the synchronous fallback processes bounded batches before cleanup. Reads retain individual completions.

Groups retain the first error and drain outstanding requests before reporting it. Completion guards also drain on drop, with callers ordered to retain raw buffer owners until completion. Existing node ownership stays unchanged; the low-level API introduces no B-tree types or buffer traits. The accounting counter uses relaxed atomics; errors use a mutex and the final result uses a channel. Hash-table write errors now propagate before fsync.

Measurements

Seeded random writes against a 2^25-item DB; 25k updates/batch, 4 workload threads, 8 commit workers, 3 I/O workers. Fixed 350 measured batches after 1M warmup updates per run, ABBA order, separate identical starting copies, affinity CPUs 0–15. Ryzen 9 7950X / Corsair MP600 PRO LPX. Runs lasted 116–146 seconds.

Per batch, mean across two runs Existing Groups
Total process CPU 1,176ms 1,057ms
Main-thread CPU 66ms 13ms
Tree-sync CPU 183ms 136ms
I/O-worker CPU 513ms 499ms
Wall time 406ms 371ms

Aggregate elapsed time: 284s → 260s (~9% lower); total CPU ~10% lower. Individual mean batch times in ABBA order: 416, 331, 411, 397ms. SSD latency remained unstable after three 180-second conditioning passes, so the CPU reduction is stronger evidence than the precise wall-time improvement. Two repeats per variant do not establish a tight confidence interval.

Every run wrote 19,054,225 pages; all final roots matched and survived reopening. An isolated no-disk pipeline using the production completion APIs reduced total CPU by 21% with burst completions and 46% with paced completions; that test excludes actual I/O and buffer deallocation.

Both measured variants included the length/decoder changes from #985, with identical benchmark instrumentation. This PR is independently based on master; standalone performance was not remeasured. Temporary instrumentation, harnesses, and research outputs are excluded.

Validation

  • cargo fmt --all -- --check
  • cargo test -p nomt
  • Group tests cover incremental/concurrent producers, first-error draining, abandoned/rejected requests, external buffer lifetime, retry accounting, io_uring idle-tail/full-ring writes, and the synchronous fallback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant