Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
- Older binaries read a newer node's waiting mark in a lock record as a held lock.
- Older binaries free a file unlinked while open at once, even when a newer node still has it open.
- Older binaries cannot replay a journal this version wrote, because they do not know its link-drop records. After a crash, let this version replay the journal before you downgrade.
- Older binaries skip rename records when they replay a journal this version wrote, but still apply the replaced file's link drop, which can free a file whose name remains. Upgrade every node before anyone renames.
- Run `goclusterfs-fsck -repair` before the first mount after upgrading an image that has snapshots or clones. Unmount the device on every node first.
- Older versions marked a file's blocks shared, on snapshot or clone, without counting the owners. Until `goclusterfs-fsck -repair` counts them, this version never frees such blocks, so deleting those files leaks their space, and it refuses to snapshot or clone them (`ENOTSUP`). In-place writes to an old clone still change its source.
- Snapshots now record the birth generation of the file they were taken of, so a later file that reuses the inode number cannot list, clone or delete them. Snapshots older versions made do not record it, and some captured no data, so after the upgrade they are orphans: `getfattr -n user.snapshots` no longer lists them, `GET /api/v1/snapshots` lists them with `orphan: true`, and `DELETE /api/v1/snapshots/<name>` without `path` deletes them. Older binaries keep the birth generation in inode bytes 196–203 only while they do not rewrite the inode, so upgrade all nodes together.
Expand Down Expand Up @@ -59,6 +60,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

- `-cache-inodes`, `-cache-extents`, `-cache-negative` and `-broadcast-max-peers` set the cache and broadcast table sizes, and `/metrics` reports each limit next to its size.
- A node that hears no broadcasts from a node the registry shows mounted, for 3 `-broadcast-heartbeat` intervals (90 seconds by default), logs a warning naming it, reports `broadcast-peers` as degraded on `/health`, and counts it in `goclusterfs_broadcast_silent_peers`. A network that silently drops multicast was invisible before.
- Rename of files and directories, in one directory or between two, with `RENAME_NOREPLACE`. A rename over a file or an empty directory replaces it; a file another node has open stays readable there until its last close. `RENAME_EXCHANGE` and `RENAME_WHITEOUT` fail with `EINVAL`.

### Changed

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Clone paths are absolute from the filesystem root, and clones can only go in the
- `goclusterfs-fsck -repair` fixes only block refcounts, sharing that older versions left uncounted, leaked blocks, inodes no directory names, snapshot registry sectors that fail their checksum, and the superblock checksum. It reports every other problem without fixing it.
- The inode count is set at format time and cannot change. Every `statfs` (`df`) reads the whole inode table to count free inodes: 50 MB on a 200 GB device at the default ratio.
- SFB is disabled: writes never allocate SFB slots, so every file uses full 1 MB blocks. `internal/sfb` container I/O is broken (unaligned header buffers, no data offset) and stays until it is fixed or removed.
- No rename, symlinks, or hard links.
- No symlinks or hard links. Rename supports `RENAME_NOREPLACE`; `RENAME_EXCHANGE` and `RENAME_WHITEOUT` fail with `EINVAL`.

## Testing

Expand Down
24 changes: 17 additions & 7 deletions internal/dir/dir.go
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,20 @@ func removeFromChunk(chunk []byte, name string) bool {
// as it needs to allocate blocks (bitmap lock) before acquiring the inode lock.
// Returns errRetryAdd if a concurrent modification is detected and the caller should retry.
func (m *Manager) addToNewBlock(ctx context.Context, dirIno *inode.Inode, entry *DirEntry) error {
return m.appendBlock(ctx, dirIno,
func(next uint64) (*inode.Inode, bool, error) {
return m.addToExistingChunk(dirIno.InodeNum, entry, next)
},
func(data []byte) {
// The entry takes the first chunk (DirChunkSize) to keep RecLen in uint16 range
entry.RecLen = uint16(DirChunkSize)
serializeEntry(entry, data)
})
}

// appendBlock maps a new block, filled by fill, as directory dirIno's next one. Under the directory lock, settled
// may report that another goroutine or node made the block unneeded.
func (m *Manager) appendBlock(ctx context.Context, dirIno *inode.Inode, settled func(nextBlock uint64) (*inode.Inode, bool, error), fill func(data []byte)) error {
// Calculate next block number based on current (possibly stale) size
nextBlock := extent.BlocksForSize(dirIno.Size)

Expand Down Expand Up @@ -622,11 +636,11 @@ func (m *Manager) addToNewBlock(ctx context.Context, dirIno *inode.Inode, entry
}

// Another node may have added this name, or blocks with room for it, while the block was allocated.
freshIno, added, err := m.addToExistingChunk(dirIno.InodeNum, entry, nextBlock)
freshIno, done, err := settled(nextBlock)
if err != nil {
return err
}
if added {
if done {
dirIno.Size = freshIno.Size
return nil
}
Expand All @@ -647,12 +661,8 @@ func (m *Manager) addToNewBlock(ctx context.Context, dirIno *inode.Inode, entry
return fmt.Errorf("failed to check extent: %w", extErr)
}

// The entry takes the first chunk (DirChunkSize) to keep RecLen in uint16 range
entry.RecLen = uint16(DirChunkSize)

// Write the entry data to the allocated physical block
data := make([]byte, BlockSize)
serializeEntry(entry, data)
fill(data)

if err := m.writeBlock(physBlock, data); err != nil {
return err
Expand Down
11 changes: 10 additions & 1 deletion internal/dir/index.go
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,15 @@ func (m *Manager) physicalBlock(dirIno *inode.Inode, block uint64) (uint64, erro

// editChunk writes the directory inode before the chunk, so no node can read the change under the old generation.
func (m *Manager) editChunk(dirIno *inode.Inode, x *index, at uint32, edit func(chunk []byte) bool, update func(names map[string]slot)) (changed bool, err error) {
return m.changeChunk(dirIno, x, at, edit, update, func() error { return m.inodeTbl.WriteWithLockHeld(dirIno) })
}

// rewriteChunk edits a chunk of a directory whose inode the caller already wrote under the same locks.
func (m *Manager) rewriteChunk(dirIno *inode.Inode, x *index, at uint32, edit func(chunk []byte) bool, update func(names map[string]slot)) (changed bool, err error) {
return m.changeChunk(dirIno, x, at, edit, update, func() error { return nil })
}

func (m *Manager) changeChunk(dirIno *inode.Inode, x *index, at uint32, edit func(chunk []byte) bool, update func(names map[string]slot), writeInode func() error) (changed bool, err error) {
physBlock, err := m.physicalBlock(dirIno, uint64(at/chunksPerBlock))
if err != nil {
return false, err
Expand All @@ -277,7 +286,7 @@ func (m *Manager) editChunk(dirIno *inode.Inode, x *index, at uint32, edit func(
m.indexes.drop(x.num)
return false, nil
}
if err := m.inodeTbl.WriteWithLockHeld(dirIno); err != nil {
if err := writeInode(); err != nil {
m.indexes.drop(x.num)
return false, fmt.Errorf("failed to write directory inode: %w", err)
}
Expand Down
Loading
Loading