Repository navigation
feat(extent): store extents beyond the inline two in an overflow tree - #8
Merged
Merged
Conversation
mattmattox
force-pushed
the
claude/feat-extent-trees-ffec13
branch
from
October 24, 2026 15:03
8cfc636 to
be8f066
Compare
Files were capped at two extents: the third allocation returned ErrNoSpaceForExtent, which the write path retried as a concurrency conflict and then failed with EAGAIN. Adjacent extents never merged either, so every file over 2MB failed. Extents that do not fit inline move into a tree of 1MB nodes (depth 1: root leaf; depth 2: root index over leaves). Each node block holds two 512KB halves selected by generation parity, so a commit never overwrites the committed copy. Changes stay on the in-memory inode until inode.Table writes it; the new TreeFlusher hook writes the new node halves first and the inode sector write commits them. The inode records the root generation in a new ExtentTreeGen field (previously reserved). Also: merge adjacent extents, split shared extents on CoW instead of failing, allocate fallocate ranges across fragmented free space, return EFBIG when the tree is full, reserve tree blocks before taking a directory's inode lock, make snapshots and clones walk the whole map, and validate trees in fsck (whose phase 4 inode reads had never succeeded under O_DIRECT). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suites Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table.Alloc is deprecated and fails the staticcheck gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tree edit freed dropped tree blocks and unmapped data blocks right away, and CoW dropped the snapshot refcount before the inode write. When that write then lost a tree conflict, the winning inode still mapped blocks the bitmap called free, and the retried write saw refcount 1 and overwrote the snapshot's block. Released blocks now wait in the pending tree and are freed after the inode write, outside the inode lock. Tree blocks allocated for a change are freed when it is discarded. Unlink uses the new Delete, which frees immediately because no inode write follows. CoW refcount decrements run after the commit, and a conflict frees the copies. Also: sync node writes before the inode write commits them, keep the on-disk extent tree when a stale inode copy without extent edits is written, and retry directory reads that hit a rewritten tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The extent cache never invalidates, and the VM runner deletes all files before each test, so a node that read an earlier file with the same inode number served stale block mappings. Restarting the verifying node first keeps these tests independent of that cache bug, and graceful restarts keep them clear of the shared journal region. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers what snapshots guarantee today for tree files: the clone gets its own extent tree, blocks shared with it survive truncating and deleting the source, and deleting the snapshot frees nothing in use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sync after the inode write that commits a tree, before freeing released blocks. Otherwise a volatile write cache can land the next commit's node write, or another file's write into a freed block, before the inode that stops referencing them, leaving the old inode pointing at overwritten data after power loss. Also: a stale copy without extent edits keeps the on-disk tree when either side has one (checked from the sector the inode write already reads); a spill that started from inline extents conflicts if another node changed them; releases survive folding a never-committed tree back inline; frees that would break the lock order wait for a later free; journal replay discards pending tree changes when its inode write fails; and write retries back off with jitter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Writing a copy whose extent tree differs from the stored one, with no extent edits of its own, now fails with ErrTreeConflict instead of taking the stored map. Keeping the stored map but the stale Size could hide a directory's newest block from every reader. Callers that can hold a stale copy re-read and retry with jittered backoff: directory and link count updates (updateInode), Setattr, Allocate, snapshot and clone setup, SFB writes, and journal replay. Allocate frees the blocks of an attempt that did not commit, writeInternal discards uncommitted tree blocks on every error return, and fsck reports data extents that point at another inode's tree node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unmapping an inline inode freed its blocks at once. If the inode write then lost a tree conflict, the winner still mapped those blocks, and the retry freed them a second time after another file could have taken them. Every unmap now queues its frees on the pending change, so a conflicted attempt frees nothing. Journal ExtentRemove replay goes through the same path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lict Allocate, SFB migration, and directory growth freed their new blocks on any inode write error. A failed sync after the inode sector lands leaves those blocks mapped on disk. Only a tree conflict proves they are unused, so other errors now leak them instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox
force-pushed
the
claude/feat-extent-trees-ffec13
branch
from
October 24, 2026 15:38
be8f066 to
a73d0f3
Compare
This was referenced Oct 24, 2026
mattmattox
added a commit
that referenced
this pull request
Oct 24, 2026
On main, a directory growing between the read and the write is already caught: every inode write compares the on-disk extents with the ones it read and retries on a change (staleExtents, from #8). The growth tests passed without the fix and failed only on slot collisions and lock timeouts. What the lock still protects is fields outside the extents, such as a parent's link count, so test that directly with two nodes interleaved inside the update. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox
added a commit
that referenced
this pull request
Oct 24, 2026
…ates (#41) * test: create files on two nodes while their directory grows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(fs): hold the inode lock across directory time and link-count updates Create, mkdir, unlink, rmdir and their Dir variants re-read the parent inode without its lock, then wrote it back to set Mtime or LinkCount. If another node's dir.Add grew the directory to a new block in between, the write-back reverted the directory's Size and extents: the new entries vanished and their block leaked. Unlink's child LinkCount update could likewise revert another node's write to the file. updateInodeLocked holds the local and cluster inode locks from the read to the write. It is only for updates that touch no extents, since freeing blocks under the inode lock would break the lock order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: grow many small directories while two nodes create in them Each directory block holds only 15 entries with 255-byte names, and growing a directory past two non-contiguous blocks fails on its own (extent tree block allocation under the inode lock), so the test grows 50 directories from one block to two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: give the directory growth test room for 100 directory blocks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: replace the directory growth tests with a two-node unit test On main, a directory growing between the read and the write is already caught: every inode write compares the on-disk extents with the ones it read and retries on a change (staleExtents, from #8). The growth tests passed without the fix and failed only on slot collisions and lock timeouts. What the lock still protects is fields outside the extents, such as a parent's link count, so test that directly with two nodes interleaved inside the update. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(fs): log skipped link count updates at WARN with the inode number A skipped link count change leaves an inconsistency that fsck reports, so operators need to find it in the log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Files failed past 2 extents: the third allocation returned
EAGAIN. Adjacent extents never merged, so every file over 2MB failed.On-disk layout. Past 2 extents, all extents move to a tree of 1MB bitmap blocks: depth 1 is a root leaf, depth 2 is a root index over leaves. Each block has two 512KB halves, and generation
glives in halfg & 1. Leaves split at 2048 extents, and the index holds 16382 leaves. New inode field:ExtentTreeGen, bytes 188–195, previously reserved and zero, so no migration.Crash safety and locking. Edits stay on the in-memory inode.
inode.Table.Writeworks under the inode lock:Every unmap, inline or tree, frees its blocks only in step 3, so a write that loses a conflict frees nothing.
The write fails with
ErrTreeConflictin two cases:Every caller that can hold a stale copy re-reads and retries with jittered backoff: write, Setattr, fallocate, directory and link-count updates, snapshot and clone setup, SFB writes, and journal replay. All of this matters because
FS.lockMgris never set, soFilewrites run unlocked.Upgrade: every node at once. Older binaries zero bytes 188–195 whenever they write an inode. Once one touches a tree inode, the file can't be read. There's no feature flag;
CHANGELOG.mdalready carries the upgrade note.Extent cache. Splits and merges never move a mapped block. The extent cache never invalidates (follow-up), so the Docker and VM tests verify only on freshly restarted nodes.
Also fixed. fsck phase 4 never read an inode: unaligned O_DIRECT buffers. CoW now splits multi-block extents. Replayed
ExtentAddsetsFlagAllocated. Truncating an SFB file no longer passes packed slot values tobitmap.Free.Snapshots. These work the same for tree files as for inline ones. They mark extents shared and bump refcounts, and a clone gets its own tree. Nothing reads the descriptor's
ExtentTreeRoot. Onmaintoday, for all files, full-block copy-on-write never triggers (refcounts reach only 1), and in-place writes to a clone change the source.TestExtentTreeSnapshotcovers what does hold: the clone's own tree, shared blocks surviving truncate and delete, and snapshot deletion freeing nothing.Follow-ups.
FlushTreeholds the table-wideinode.Table.muthrough node writes and both syncs, so each tree commit stalls other inode I/O on that node. Truncating an inline file now also costs one sync.maintoo:Deletefrees a file's blocks beforeTable.Freeand without a sync.maintoo:deleteSnapshotdecrements the live file's current blocks, so blocks added after the snapshot underflow and abort the whole batch.Tested on 172.25.1.68 (a73d0f3 unless noted).
-race; golangci-lint with the repo config reports 0 issues.-short: 70 passed, 0 failed, with a freshly builtgoclusterfs-mount.mainrun back to back (221s vs 199s on shared tests).make test-paritypasses.🤖 Generated with Claude Code