Skip to content

fix(fs): keep a file unlinked while open until every node closes it - #161

Merged
mattmattox merged 9 commits into
mainfrom
fix/orphan-holders
Nov 16, 2026
Merged

mattmattox merged 9 commits into
mainfrom
fix/orphan-holders

Conversation

@mattmattox

Copy link
Copy Markdown
Contributor

A descriptor opened before an unlink kept writing into whatever file took the inode next, on any node. The history checker (#156) found it; TestFileUnlinkedWhileOpenKeepsItsData proves it.

Trade-offs:

  • A node can't see a peer's opens, so with a peer mounted every last unlink waits for peers to let go. A removed file's space and inode come back milliseconds later, or up to one 5 s scan without broadcast. Removing 1000 empty files on a peer takes 1.6 s, against 1.0 s on main.
  • A full orphan list (487) waits for peers, then gives EBUSY after 10 s with no progress.
  • Only the node that holds the file can reopen it, through /proc/self/fd.
  • The new fields use inode bytes 204–211 and registry bytes 188+, which were zero, so there's no format bump.

Verified on .68 before rebasing onto #155/#160: lint, unit, -race, -short integration (the two timing tests it broke are fixed in the last commit), parity, and the unlink and recovery tests.

🤖 Generated with Claude Code

mattmattox and others added 8 commits November 16, 2026 12:34
Unlinking a file that a descriptor still had open freed its inode at
once, so the descriptor read and wrote whatever file took the inode
next, on any node.

A file that loses its last link now stays as an orphan: LinkCount 0 and
one Holders bit per registry slot that may still have it open. Each node
clears its bit at its last close, and whoever clears the last bit frees
the file. The unlinking node lists the orphan in its registry slot so
peers find it, and an Orphans broadcast tells them at once. A full list
waits while peers free entries, then fails the unlink with EBUSY.

The journal logs LinkDrop instead of an nlink update, so replay redoes a
lost drop. A peer clears a dead node's bits after replaying its journal,
and a clean unmount clears the node's own. File.Open and Create return a
file handle, without which go-fuse never calls Release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node that still has an unlinked file open may open it again through
/proc/self/fd, as local filesystems allow. Its holder bit keeps the
inode from being freed or reused meanwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a peer mounted, a removed file is freed once the peer lets go of it,
on the Orphans broadcast or at its next failure scan, so inode reuse and
freed blocks show up a moment after the remove.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node alone that unlinked a file it had open did not list the orphan,
so after a crash nothing found it and it leaked until fsck -repair. Such
orphans are now listed, and the next mount of the slot frees them.

A full orphan list now keeps waiting while a peer has stopped
heartbeating or awaits replay, as lock waits do, and gives EBUSY only
when live peers stop freeing for 20 s.

Replay now frees the SFB slot of an orphan it frees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The unlinking node lists a file before its last link goes. A peer that
looked in that window remembered the entry as done, and the node's own
prune dropped it as stale, so after the drop nobody freed the file until
its holder unmounted. Both now leave a still-linked entry alone; only
the node's own mount and unmount drop such entries, which a crash left.

The many-removes test now checks that peers free every file while
still mounted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit 0d2ec71 into main Nov 16, 2026
17 checks passed
mattmattox added a commit that referenced this pull request Nov 16, 2026
Every inode rewrite wrote reserved bytes 212-251 as zero. A future field there would be wiped whenever this build rewrote the inode, which is how older builds lost the birth generation at 196-203. The inode now keeps those bytes as read and writes them back, as #158 did for snapshot descriptors. Freeing or claiming a slot still starts from a zeroed inode.

main is merged in, including #161. Holders keeps bytes 204-211, so Reserved is now 40 bytes (212-251). The merge commit holds that change plus the new test; the first commit's message still says 204-251.

Tests:
- FuzzDeserializeInode now checks the re-encoded bytes. Its seed sets both Holders and the reserved bytes.
- New TestRewriteChangesHoldersAndKeepsReservedBytes: through the Table, a Holders change reaches disk and the reserved bytes survive. It fails if a rewrite puts back the Holders read from disk.
- Fuzzed 3 min on .66 with no failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant