Skip to content

feat(fsck): list node registry slots and release a dead node's slot - #38

Merged
mattmattox merged 3 commits into
mainfrom
feat-fsck-release-slot
Oct 24, 2026
Merged

mattmattox merged 3 commits into
mainfrom
feat-fsck-release-slot

Conversation

@mattmattox

@mattmattox mattmattox commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #34. A dead node's slot whose journal header is damaged was never freed. Peers failed to replay it every minute, and only the same host could take it back, and only before a reboot.

  • goclusterfs-fsck -list-slots lists each slot that is in use or whose journal needs attention, with its owner, lease and journal state.
  • -release-slot N prints what releasing would discard and writes nothing. It refuses when:
    • mount would reject the superblock;
    • any node is mounted (fix(fsck): match what the runtime writes and make -repair safe #32's registry watch);
    • the slot is unreadable;
    • less than a minute has passed since the slot's lease ended;
    • without -force, a mount might still replay the journal: fsck reads it in full with transactions pending, or cannot read it to the end.
  • -discard-journal takes the slot's recovery lease the way a replaying peer does, waits the claim settle, zeroes the journal header, and then empties the slot. Register and failure detection skip a leased slot. If fsck dies midway, or the reset takes longer than 30 s, the lease lapses after 60 s and a peer finishes the job.
  • Exit codes: 0 for a listing or a dry run, 1 when a slot is released, 8 when refused, 16 for an unknown slot.

cmd/mount is untouched. internal/node gains ParseSlot, Slot.LeaseExpiry, Registry.ReleaseSlot and NodeState.String.

Tests on .66, after the rebase onto main: unit -race and the fsck integration tests pass. The new unit tests, including the lease races and fsck crashing before or after the reset, pass 3× under -race. golangci-lint v2 passes.

🤖 Generated with Claude Code

mattmattox and others added 3 commits October 24, 2026 18:44
A dead node's slot whose journal header is damaged was never freed:
peers fail to replay it every minute, and only the same host before a
reboot could take it back. goclusterfs-fsck -list-slots shows each
slot's owner, lease and journal state. -release-slot N checks that no
node runs and that the slot's lease ended a minute ago, and prints what
releasing discards; with -discard-journal it resets the slot's journal
and frees the slot, so the next mount can use it. cmd/mount is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes for slot release:
- A node mounting between the registry check and the writes could get
  its new journal header wiped. Release now takes the slot's recovery
  lease first, as a peer replaying the slot would, so Register and
  failure detection skip the slot while fsck resets its journal.
- Release requires a superblock that mount accepts, since it writes at
  offsets the superblock gives.
- A journal a mount could replay in full is refused without -force.
- The discard count includes transactions read before a bad record.
- A slot number the registry does not have exits 16; a release exits 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is unreadable

Review follow-ups for slot release:
- ReleaseSlot leaves the slot dead under its lease instead of freeing
  it when the journal reset took more than half the recovery lease.
- A dry run that cannot read the whole journal now needs -force, since
  a mount may still replay it.
- Tests for fsck crashing while it holds the lease, before and after
  the journal reset, and for a slow reset.
- The manifest names the slot release tests that exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox force-pushed the feat-fsck-release-slot branch from d6b5324 to 5e6459d Compare October 24, 2026 23:44
@mattmattox
mattmattox merged commit f474ff8 into main Oct 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant