Repository navigation
perf(dir): answer lookups from an in-memory directory index - #127
Merged
Merged
Conversation
Each lookup took the directory lock, read the whole 1 MB block and decoded every entry, and ls -l repeats lookups once it outlasts the kernel's 1s cache, so it went quadratic: 65s for 4,000 files. Each node now keeps an index per directory (name to inode, and free room per chunk). A lookup reads the directory inode without a lock and uses the index only while the generation on disk is the one the index was built at; a torn read fails the inode checksum and takes the locked path. Every entry change writes the directory inode before the chunk, inside one lock hold, so no node can read new entries under an old generation, and RemoveIf now does so before the child is freed. Add and RemoveIf re-check the index against the disk under the directory lock and rebuild it on mismatch. Writes that change no entries, such as mtime and link count updates, carry the index to the new generation. -dir-index-entries caps the names held, least recently used directory first, and /metrics exports hits, misses and rebuilds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…egration test within the image's blocks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck requires Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cluster lock admits every goroutine on a node at once: a second exclusive or shared acquire of a held key joins the hold. Replay and API clones bypass the kernel's per-directory lock, so two goroutines could edit the same chunk and lose an entry, and the index then named the lost entry at the current generation. dir.Manager now takes a striped per-directory lock before the cluster lock: exclusive in Add, RemoveIf and InitDirectory, shared when building an index. Directory inode writes in goclusterfs (mtime, link count, setattr) take it too, through updateDirLocked, so KeepIndex runs only while no entry change is in flight. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # tests/test-manifest.yaml
This was referenced Nov 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Option A from the PM's plan. Each node keeps an index per directory. A lookup reads the directory inode without a lock and trusts the index only while the inode's generation matches. A torn read fails the inode CRC and takes the locked path.
PM conditions:
RemoveIffinishes before the child is freed. Create, mkdir, unlink, rmdir, clone and replay all go throughAdd/RemoveIf; there is no rename.Add/RemoveIfre-check the index under the directory lock and rebuild it on a mismatch.-dir-index-entries(default 1,048,576 names, about 100 MB) with an LRU cap;goclusterfs_dir_index_{hits,misses,rebuilds}_total.On .68, with two nodes mounted outside
$HOME(load 53–66), main vs this PR:ls -lstatls -lwhile the other node creates 200At 10k entries,
ls -lstill takes 14 s. Once a pass outlasts the 1 s kernel cache it does 4 lookups per file, and each lookup reads two inodes (about 0.3 ms).On .68 at 2a950c6:
make lint, all unit tests and-raceon dir, goclusterfs, stats and api pass.TestMultiNodeDirIndexWithoutBroadcast,TestMultiNodeCreatesInOneDirectoryandTestMultiNodeAttrTimeoutZeropass 3/3. In the-shortintegration run onlyTestFsckRepairsOlderSharingfailed, and it fails the same way on main (its remount after the offline image edit times out). The new Docker and VM shell tests weren't run locally.🤖 Generated with Claude Code