Skip to content

perf: zero only what a read could see, not every new block - #160

Merged
mattmattox merged 3 commits into
mainfrom
perf/narrow-zeroing-2
Nov 16, 2026
Merged

mattmattox merged 3 commits into
mainfrom
perf/narrow-zeroing-2

Conversation

@mattmattox

Copy link
Copy Markdown
Contributor

Follow-up to #145, as the PM asked. #145 zeroed every new block before a partial write. FUSE hands writes over in 128 KiB pieces, so that doubled the bytes written for all new data.

The rule now

  • Bytes past EOF can't be read, so they may stay as the disk held them.
  • A first write into a block zeroes only what it doesn't cover below EOF: the head before it, and the tail when it fills a hole.
  • A write, truncate up or fallocate that grows the file from S zeroes [S, min(start, end of S's block)) first. It goes in as a zero segment through the normal write loop, so a block shared with a snapshot or clone is copied, with the usual lock order.
  • Truncate no longer zeroes on shrink; zeroPastEnd is gone.

Locking, so growth never wipes another node's unsized data

  • A write that grows the file takes the inode lock exclusively before its first data write and holds it through the size update.
  • If it drops the lock mid-write, to copy a block or allocate tree blocks, it first records a size covering what it wrote in the old last block.
  • A write that starts at a block boundary still appears all at once or not at all after a crash. The reserve two-block power-cut test checks that.

Cost

Tests

  • fix: never let a read show bytes nobody wrote to the file #145's stale-bytes and power-cut tests all pass.
  • New cases: append to a reused block then fallocate; snapshot a reused block, then grow the file and a clone of the snapshot.
  • TestGrowingWriteNeverZeroesAnotherNodesAppend: one node appends while another writes past EOF.

Results on .66

  • Full -race over ./internal/... and ./tests/soak: all packages pass except one load-timing test in internal/lock, a package this PR doesn't touch, which passed 3/3 when rerun alone.
  • VM, 2 nodes: Power Failure, Concurrent Writes Power Failure, fsck Repair After Power Failure, CoW / Snapshot / TRIM Power Failure, Truncate File, Concurrent Writes, Concurrent Write Stress and Data Persistence all pass, 10/10.

🤖 Generated with Claude Code

mattmattox and others added 3 commits November 16, 2026 11:20
#145 zeroed every new block before a partial write. FUSE hands writes over
in 128 KiB pieces, so that doubled the bytes written for all new data.

Bytes past the end of a file cannot be read, so they may stay as the disk
held them. A first write into a block now zeroes only the parts below the
end of the file that it does not cover. A write, truncate or fallocate
that grows the file zeroes the rest of its old last block first, through
the write path, so a block a snapshot or clone shares is copied. Truncate
no longer zeroes on shrink.

A growing write holds the inode lock exclusively from its first data write
through its size update, and if it drops the lock mid-write it first
covers what it wrote in the old last block, so another node's growth never
zeroes data it has not yet sized.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ned the gap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit daa33a6 into main Nov 16, 2026
17 checks passed
mattmattox added a commit that referenced this pull request Nov 16, 2026
The 48h soak on daa33a6 found another file's block with its first
176128 bytes zeroed on disk after a write into a hole on a different
node. That range matches the head zero of the hole write, which points
at #160's narrower zeroing. #145's full-block zeroing ran clean for
26,792 soak ops. Restore it until #160's cause is found.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox added a commit that referenced this pull request Nov 17, 2026
…182)

* test(goclusterfs): pace the uncommitted-blocks writer on the reader

TestReaderNeverSeesUncommittedBlocks required the reader to see at least 3
sizes while the writer filled 16 blocks, which depends on how the two are
scheduled: with #160's narrower zeroing the writes were fast enough that the
reader sometimes saw only 2, about 1 run in 20 on .68. After each block the
writer now waits for a read that started after it, so the reader sees every
block's size, while it keeps reading during the next block's writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(goclusterfs): pace at quarters instead of every block

Waiting after every block made the test take 30 s under -race. Three waits,
after each quarter of the file, are enough to show the reader three sizes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant