Skip to content

feat(extent): store extents beyond the inline two in an overflow tree - #8

Merged
mattmattox merged 14 commits into
mainfrom
claude/feat-extent-trees-ffec13
Oct 24, 2026
Merged

mattmattox merged 14 commits into
mainfrom
claude/feat-extent-trees-ffec13

Conversation

@mattmattox

@mattmattox mattmattox commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Files failed past 2 extents: the third allocation returned EAGAIN. Adjacent extents never merged, so every file over 2MB failed.

On-disk layout. Past 2 extents, all extents move to a tree of 1MB bitmap blocks: depth 1 is a root leaf, depth 2 is a root index over leaves. Each block has two 512KB halves, and generation g lives in half g & 1. Leaves split at 2048 extents, and the index holds 16382 leaves. New inode field: ExtentTreeGen, bytes 188–195, previously reserved and zero, so no migration.

Crash safety and locking. Edits stay on the in-memory inode. inode.Table.Write works under the inode lock:

  1. Write the new nodes into the unused halves, then sync.
  2. Write the inode sector. This is the commit.
  3. Sync again, then free the released blocks.

Every unmap, inline or tree, frees its blocks only in step 3, so a write that loses a conflict frees nothing.

The write fails with ErrTreeConflict in two cases:

  • The copy's tree differs from the one on disk, even when the copy has no extent edits.
  • The inline extents changed under an uncommitted spill.

Every caller that can hold a stale copy re-reads and retries with jittered backoff: write, Setattr, fallocate, directory and link-count updates, snapshot and clone setup, SFB writes, and journal replay. All of this matters because FS.lockMgr is never set, so File writes run unlocked.

Upgrade: every node at once. Older binaries zero bytes 188–195 whenever they write an inode. Once one touches a tree inode, the file can't be read. There's no feature flag; CHANGELOG.md already carries the upgrade note.

Extent cache. Splits and merges never move a mapped block. The extent cache never invalidates (follow-up), so the Docker and VM tests verify only on freshly restarted nodes.

Also fixed. fsck phase 4 never read an inode: unaligned O_DIRECT buffers. CoW now splits multi-block extents. Replayed ExtentAdd sets FlagAllocated. Truncating an SFB file no longer passes packed slot values to bitmap.Free.

Snapshots. These work the same for tree files as for inline ones. They mark extents shared and bump refcounts, and a clone gets its own tree. Nothing reads the descriptor's ExtentTreeRoot. On main today, for all files, full-block copy-on-write never triggers (refcounts reach only 1), and in-place writes to a clone change the source. TestExtentTreeSnapshot covers what does hold: the clone's own tree, shared blocks surviving truncate and delete, and snapshot deletion freeing nothing.

Follow-ups.

  • FlushTree holds the table-wide inode.Table.mu through node writes and both syncs, so each tree commit stalls other inode I/O on that node. Truncating an inline file now also costs one sync.
  • Any two extent changes to one tree file conflict, even on disjoint ranges.
  • On main too: Delete frees a file's blocks before Table.Free and without a sync.
  • On main too: deleteSnapshot decrements the live file's current blocks, so blocks added after the snapshot underflow and abort the whole batch.

Tested on 172.25.1.68 (a73d0f3 unless noted).

🤖 Generated with Claude Code

mattmattox and others added 14 commits October 24, 2026 10:38
Files were capped at two extents: the third allocation returned
ErrNoSpaceForExtent, which the write path retried as a concurrency
conflict and then failed with EAGAIN. Adjacent extents never merged
either, so every file over 2MB failed.

Extents that do not fit inline move into a tree of 1MB nodes (depth 1:
root leaf; depth 2: root index over leaves). Each node block holds two
512KB halves selected by generation parity, so a commit never overwrites
the committed copy. Changes stay on the in-memory inode until
inode.Table writes it; the new TreeFlusher hook writes the new node
halves first and the inode sector write commits them. The inode records
the root generation in a new ExtentTreeGen field (previously reserved).

Also: merge adjacent extents, split shared extents on CoW instead of
failing, allocate fallocate ranges across fragmented free space, return
EFBIG when the tree is full, reserve tree blocks before taking a
directory's inode lock, make snapshots and clones walk the whole map,
and validate trees in fsck (whose phase 4 inode reads had never
succeeded under O_DIRECT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suites

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table.Alloc is deprecated and fails the staticcheck gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tree edit freed dropped tree blocks and unmapped data blocks right
away, and CoW dropped the snapshot refcount before the inode write.
When that write then lost a tree conflict, the winning inode still
mapped blocks the bitmap called free, and the retried write saw
refcount 1 and overwrote the snapshot's block.

Released blocks now wait in the pending tree and are freed after the
inode write, outside the inode lock. Tree blocks allocated for a change
are freed when it is discarded. Unlink uses the new Delete, which frees
immediately because no inode write follows. CoW refcount decrements run
after the commit, and a conflict frees the copies.

Also: sync node writes before the inode write commits them, keep the
on-disk extent tree when a stale inode copy without extent edits is
written, and retry directory reads that hit a rewritten tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The extent cache never invalidates, and the VM runner deletes all files
before each test, so a node that read an earlier file with the same
inode number served stale block mappings. Restarting the verifying node
first keeps these tests independent of that cache bug, and graceful
restarts keep them clear of the shared journal region.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers what snapshots guarantee today for tree files: the clone gets its
own extent tree, blocks shared with it survive truncating and deleting
the source, and deleting the snapshot frees nothing in use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sync after the inode write that commits a tree, before freeing released
blocks. Otherwise a volatile write cache can land the next commit's node
write, or another file's write into a freed block, before the inode
that stops referencing them, leaving the old inode pointing at
overwritten data after power loss.

Also: a stale copy without extent edits keeps the on-disk tree when
either side has one (checked from the sector the inode write already
reads); a spill that started from inline extents conflicts if another
node changed them; releases survive folding a never-committed tree back
inline; frees that would break the lock order wait for a later free;
journal replay discards pending tree changes when its inode write fails;
and write retries back off with jitter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Writing a copy whose extent tree differs from the stored one, with no
extent edits of its own, now fails with ErrTreeConflict instead of
taking the stored map. Keeping the stored map but the stale Size could
hide a directory's newest block from every reader. Callers that can hold
a stale copy re-read and retry with jittered backoff: directory and link
count updates (updateInode), Setattr, Allocate, snapshot and clone
setup, SFB writes, and journal replay. Allocate frees the blocks of an
attempt that did not commit, writeInternal discards uncommitted tree
blocks on every error return, and fsck reports data extents that point
at another inode's tree node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unmapping an inline inode freed its blocks at once. If the inode write
then lost a tree conflict, the winner still mapped those blocks, and the
retry freed them a second time after another file could have taken them.
Every unmap now queues its frees on the pending change, so a conflicted
attempt frees nothing. Journal ExtentRemove replay goes through the same
path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lict

Allocate, SFB migration, and directory growth freed their new blocks on
any inode write error. A failed sync after the inode sector lands leaves
those blocks mapped on disk. Only a tree conflict proves they are unused,
so other errors now leak them instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox force-pushed the claude/feat-extent-trees-ffec13 branch from be8f066 to a73d0f3 Compare October 24, 2026 15:38
@mattmattox
mattmattox merged commit cd52194 into main Oct 24, 2026
9 of 10 checks passed
mattmattox added a commit that referenced this pull request Oct 24, 2026
On main, a directory growing between the read and the write is already
caught: every inode write compares the on-disk extents with the ones it
read and retries on a change (staleExtents, from #8). The growth tests
passed without the fix and failed only on slot collisions and lock
timeouts. What the lock still protects is fields outside the extents,
such as a parent's link count, so test that directly with two nodes
interleaved inside the update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox added a commit that referenced this pull request Oct 24, 2026
…ates (#41)

* test: create files on two nodes while their directory grows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(fs): hold the inode lock across directory time and link-count updates

Create, mkdir, unlink, rmdir and their Dir variants re-read the parent
inode without its lock, then wrote it back to set Mtime or LinkCount.
If another node's dir.Add grew the directory to a new block in between,
the write-back reverted the directory's Size and extents: the new
entries vanished and their block leaked. Unlink's child LinkCount
update could likewise revert another node's write to the file.

updateInodeLocked holds the local and cluster inode locks from the
read to the write. It is only for updates that touch no extents, since
freeing blocks under the inode lock would break the lock order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: grow many small directories while two nodes create in them

Each directory block holds only 15 entries with 255-byte names, and
growing a directory past two non-contiguous blocks fails on its own
(extent tree block allocation under the inode lock), so the test grows
50 directories from one block to two.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: give the directory growth test room for 100 directory blocks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: replace the directory growth tests with a two-node unit test

On main, a directory growing between the read and the write is already
caught: every inode write compares the on-disk extents with the ones it
read and retries on a change (staleExtents, from #8). The growth tests
passed without the fix and failed only on slot collisions and lock
timeouts. What the lock still protects is fields outside the extents,
such as a parent's link count, so test that directly with two nodes
interleaved inside the update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(fs): log skipped link count updates at WARN with the inode number

A skipped link count change leaves an inconsistency that fsck reports,
so operators need to find it in the log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant