Skip to content

perf(inode): start each node's free-inode search in its own part of the table - #35

Merged
mattmattox merged 3 commits into
mainfrom
claude/per-node-inode-ranges
Oct 24, 2026
Merged

mattmattox merged 3 commits into
mainfrom
claude/per-node-inode-ranges

Conversation

@mattmattox

Copy link
Copy Markdown
Contributor

FindFreeSlot always returned the lowest free inode. Nodes that create files at once picked the same inode and lost ClaimSlot races, so creates retried with backoff and could fail with EAGAIN. With #29's claim confirmation, each lost race also costs a full re-claim.

Now each node starts its search in its own part of the inode table and wraps around. The starting point is a fraction of the table chosen by the node's registry slot, bit-reversed so that slots 0, 1, 2, 3 start at 0, ½, ¼ and ¾. With plain slot × 4096/64 spacing, each node gets about 64 inodes before it runs into the next node's range. Bit reversal keeps the first few nodes at least a quarter of the table apart.

There is no on-disk change. A single node in slot 0 scans as before. cmd/mount passes the slot index (one line), and internal/lock is untouched.

Two nodes creating 150 files each in one directory at once (TestMultiNodeConcurrentCreatesDoNotCollide, .66, ext4 images):

Build Inode claim collisions per run Time for 300 creates
main (51f86a0) 10, 10, 5 51–61 s
this PR 0, 0, 0 55 s
#29 (ff0c011) 58, 60, with EAGAIN failures in both runs 3m24s–3m48s
#29 + this PR 0, 0 2m41s–2m54s

On #29, this PR removes the collisions and the EAGAIN failures and saves about 45 s. The rest of #29's cost over main (about 2m45s against 55s) is its per-claim confirmation, not collisions.

Unit tests cover the spread of the first four slots, wrap-around, and a full table that still finds every freed inode. Unit tests (-race) pass. Integration passes in both modes (-short 84/84, full 98/98) and lint is clean.

The new test does not read files back from the other node. On main, a file one node created was missing on the other node in 1 of 3 runs: concurrent directory updates lose an entry when two nodes both hold the directory lock. #29 fixes that; its runs had no missing files.

🤖 Generated with Claude Code

mattmattox and others added 3 commits October 24, 2026 17:16
…he table

FindFreeSlot always returned the lowest free inode, so nodes creating
files at once picked the same inode and lost ClaimSlot races: creates
retried and could fail with EAGAIN, and with lock claim confirmation
every collision costs a full re-claim. Each node now starts its search at
a fraction of the table chosen by its registry slot and wraps around.
Slots map to bit-reversed fractions (0, 1/2, 1/4, 3/4, ...), so the
first few nodes start a quarter of the table or more apart. No on-disk
change; a single node in slot 0 scans as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… log

The root directory's create logs "inode slot collision, retry", not the
ClaimSlot rollback message. Drop the cross-node read-back: concurrent
directory updates can still lose an entry until lock claims are confirmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… on failure

CI run 40138038165 failed with EIO on node 1's first create. Its lock
wait ended with "TIMEOUT after 3 retries": go-fuse cancels the request
context when the kernel sends FUSE_INTERRUPT, which happens when the
calling process gets a signal. That is not an inode collision, so the
test retries such creates and counts them. On failure it logs each
node's warning, error and timeout lines.

Also fix the inode reuse comment in the Docker and VM scripts: new files
take the first free inode from the node's start, not the lowest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit 170c94b into main Oct 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant