Skip to content

fix(journal): free truncated and unlinked blocks on replay only while the inode still maps them - #78

Merged
mattmattox merged 10 commits into
mainfrom
claude/journal-exact-frees
Nov 12, 2026
Merged

mattmattox merged 10 commits into
mainfrom
claude/journal-exact-frees

Conversation

@mattmattox

@mattmattox mattmattox commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #65. Truncate and subdirectory unlink now log an ExtentFree record with three things: the physical blocks they drop, the new size, and the inode's MetaGeneration. Replay cuts the size and frees a block only if two checks pass: the inode still has that generation, and it still maps that disk block at that file offset. It frees through refcounts, so blocks a snapshot or clone shares stay allocated. Old BlockFree records (logical blocks) still free nothing. Mount now opens the refcount region before its own replay.

Each operation now changes the inode before it frees blocks, so replay's checks fail once any block is free:

  • Truncate writes the inode, which bumps the generation, before it frees blocks.
  • Unlink frees the inode before its blocks. A crash between the two leaks the blocks, and fsck reclaims them.
  • A failed write (including CoW copies) closes its intent before it frees the blocks it allocated. It checks which blocks the inode maps on disk while it still holds the inode lock.

Failed mkdirs free their directory block. Their intent maps no blocks, so replay cannot map it again.

Lock/replay ordering is not guaranteed. A peer takes a lock as stale 15 s after its AcquireTime (lock.isLockStale) and never checks that the owner's journal was replayed. Replay waits for failure detection: a 10 s heartbeat timeout plus a 5 s margin, checked every 5 s. So a peer can change the inode first. The generation check covers this: any later inode write makes replay skip the record and leave the blocks to fsck -repair.

Not covered, follow-ups:

  • InodeDelete replay is unguarded. If another file takes the unlinked inode's slot before the intent closes, replay deletes the new file. A test on this branch reproduced it; main has the same bug.
  • If unlink cannot free the inode (lock timeout), it keeps the blocks, and fsck finds an orphan.
  • Root-directory unlink journals only DirRemoveEntry, so its blocks are left to fsck.
  • One record holds at most 256 extents. Replay leaves any further blocks to fsck.

Tests (.66, 1f44318): every unit package passes with -race, and integration -short is 130/0. The crash-window, failure and replay tests, plus #75's interrupted truncate/unlink tests, pass 10x with -race. golangci-lint v2 reports 0 issues. On main, the truncate replay test leaves the dropped blocks allocated, and the failed-write test leaks a block. The regression tests come from the review's probes. Each fix was undone in turn, and its test fails without it: unlink order, intent before free, size replay.

🤖 Generated with Claude Code

mattmattox and others added 10 commits November 12, 2026 06:47
…still mapped

BlockFree records hold logical file blocks, so #65 made their replay
free nothing, and an interrupted truncate or unlink leaks its blocks.
ExtentFree records one freed extent: its file blocks and the disk blocks
they map. Replay unmaps and frees a block only while the inode still
maps that disk block at that file block. A block the inode no longer
maps there was freed already, or holds a later write (another node may
have taken the inode's lock after a crash), so replay leaves it alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ailed writes and mkdirs

Truncate and unlink in a subdirectory now journal an ExtentFree record
for each extent they free, instead of a BlockFree with logical blocks,
so replaying an interrupted truncate or unlink frees those blocks. At
most 256 extents are journaled per operation; replay leaves any others
allocated for fsck -repair.

A write whose inode write fails now frees the blocks it allocated and
its copy-on-write copies, and a mkdir whose inode write or directory
setup fails frees the new directory's block. Both free only blocks the
inode on disk does not map, since a failed inode write may have landed.
The fsck leak tests from #72 now leak a block directly, as a failed
write no longer does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e, through refcounts

A peer can take a dead node's inode lock as stale after 15 s, before that
node's journal is replayed, and change the file. With only the mapping
check, a block the peer freed and the allocator later gave back at the
same offset would be freed again by replay. ExtentFree now records the
inode's MetaGeneration, which every inode write bumps, and replay frees
nothing unless it still matches. One record per inode holds all its
freed extents, so replay checks and writes the inode once.

Replay frees through the extent manager's refcount-aware path, so a block
shared with a snapshot or clone loses one reference instead of its bitmap
bit. Mount now creates the refcount region before it replays its own
journal so that path has refcounts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…write's intent before freeing

Replay of an unlink's ExtentFree passed both checks while the inode still
existed after its blocks were freed, and a failed write freed blocks its
open intent could map again. ExtentFree now also carries the size, so a
replayed truncate cuts the file instead of leaving zeros past the cut.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant