Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
- Per-node block reserves (`-block-reserve`, default 64): each node takes a batch of free blocks under one hold of the bitmap lock and hands them to its writes without it, gives unused ones back when idle, at unmount and when space runs low, and waits up to 12 seconds for other nodes' reserves before reporting `ENOSPC`. Journal replay and fsck return the unused reserve of a crashed node. See OPERATIONS.md.
- `/metrics` and `/api/v1/stats/lock` report, per lock type, the acquires served, the acquires that found the lock taken, and how long those waited (`goclusterfs_lock_acquires_total`, `goclusterfs_lock_blocked_total`, `goclusterfs_lock_wait_seconds_total`).
- Go, Docker, and QEMU VM test suites, with `tests/test-manifest.yaml` and `make test-parity` to track coverage across them.
- Fuzz tests for the on-disk decoders (superblock, inode, snapshot descriptor, journal superblock and records) and the journal recovery scan.
- Fuzz tests for the on-disk decoders (superblock, inode, snapshot descriptor, journal superblock and records, lock records, node registry slots, refcount sectors, directory blocks, extent tree nodes, SFB containers), the journal recovery scan, journal replay checked by fsck, and the HTTP API write endpoints and file paths, with seed corpora from two-hour runs.
- CI gates: unit tests with the race detector and coverage, golangci-lint (`make lint`), govulncheck (`make vuln`), and a `go mod tidy` check.
- Release workflow: pushing a `v*` tag publishes a GitHub Release with `linux/amd64` and `linux/arm64` archives, checksums, and the matching `CHANGELOG.md` section. `scripts/dist.sh` builds the same archives locally.
- `goclusterfs-fsck -version` prints the release version, commit, and commit date.
Expand Down Expand Up @@ -78,6 +78,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),

### Fixed

- A damaged block reserve record in a journal could make journal replay and `goclusterfs-fsck` allocate up to 32 GB. A reserve record now holds at most 1,048,576 blocks, and a larger `-block-reserve` acts as 1,048,576.
- A damaged journal record that claimed a length within 4 KB of 4 GB crashed journal recovery with a panic. Recovery now treats it as corrupt.
- A node registry slot with a zero checksum was accepted whatever it held, so a slot torn or zeroed at its end read as valid. Now only an all-zero slot, which no node has written, skips the checksum.
- A damaged directory entry whose name ran past the end of its record was read with bytes of the next entry as part of its name. It is now rejected as invalid.
- Deleting or rewriting a file that an older build stored in an SFB slot crashed the mount when the container header was damaged so the slot lay past its bitmap. The slot is now left allocated.
- `goclusterfs-fsck` ran for hours on a superblock with a damaged node count, reading one journal per node it claimed. It now reads at most the 64 that the node registry has.
- One failed write while a node released a lock left its claim on the device, and the node forgot it: other nodes timed out with `EIO` on every resource in that lock record until the node unmounted. The node now clears such a claim again every second until the write succeeds.
- One failed read of the snapshot registry during a snapshot create could delete other snapshots: the create counted the unreadable sector's slots as free, and a write to a sector it could not read replaced the sector with one that held only the new snapshot, losing up to 30 others. The create still reported success. Both now fail with `EIO` and leave the sector alone. A sector that stays unreadable now blocks every snapshot create; `goclusterfs-fsck` reports it but cannot repair it yet.
- A node in a higher registry slot could wait 30 seconds and fail with `EIO` for a lock that a node in a lower slot kept releasing and claiming again every few milliseconds. A node that loses a claim to a lower slot now leaves a waiting mark in its own slot, and the holder gives the lock up 250 ms after it first sees a live node's mark.
Expand Down
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/00218fddfc75a418
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("A0AAAA:")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/00a3efd66d13d083
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"00000000000000000000\xdc0\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xdc\xf1\x9700\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/011bdb79e3780716
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("[\r\r\r\r")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/01b2eea097520ae8
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x00')
string("api/v1/admin/")
bool(true)
string("0")
string("0")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/02ba9964d4360b0e
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//00000000@")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/02d3de61ffa9a1d0
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//@@@@@@@@@@@@@@@@@")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0315fe35de9ed600
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('F')
string(" 000000000//")
bool(true)
string("0")
string("")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/032b33ce145187de
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("")
string("AA0")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/03fa7580d09af98f
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("10000000000000000")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/041cef5b2ddb7755
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x02')
string("api/v1/snapshots/0")
bool(true)
string("0")
string("//000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/04e4c0bbf611a1e2
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("")
string("A.............. ")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/04f8ea13a832169b
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(false)
byte('\x01')
string("api/0")
bool(false)
string("0")
string("0")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/059a07846e8c8f3c
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"\xf3\x99\x99\xd6\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/05dbfc61b1f60db5
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"\\t\"0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/05e553fb0ef22549
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x02')
string("api/v1/snapshots/\xe1ᨨ\xa8\xa8\xa8\xa8\xe1\xe1\xe1")
bool(true)
string("")
string("")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/063f22ced942cfca
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('F')
string("&00000//")
bool(true)
string("0")
string("")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/066851239013ac48
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("")
string("")
[]byte("{\"\":\"\",\t0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/078c92070a933dd4
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\u009b')
string("api/v100000000 00")
bool(true)
string("")
string("")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/07d95161c0f657a6
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:00000000000000000000000A000000000000000000000000000000000000000000000")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0852a60ec4956e6a
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/snapshots")
bool(true)
string("")
string("")
[]byte(" ")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/08dd42847343422c
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:0000000000000000000000000000")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/093136e5b919b8a4
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/snapshots")
bool(true)
string("0")
string("//AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA@")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0950cfe68c8900e2
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("0E00000000000000000000000000000000000000000000000000000000000000000")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/09b9ead5a083174d
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(false)
byte('\x00')
string("0000000000 0/..")
bool(true)
string("0")
string("0")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/09eaf1b11fb552cd
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:00000000000000000000000000000000000000000000000000000000000000000000")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0a38867d73d6b5a4
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('V')
string("api/v1/snapshots/0/0")
bool(true)
string("0")
string("")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0a401ef1a09a6e73
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('$')
string("api/v1/snapshots")
bool(true)
string("0")
string("")
[]byte("[[[]]]")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0aa117889f00235f
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('c')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("0E00000000000000000")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0afc617fd912db67
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:000000000000000000000000000A0000")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0b495b644f6dfa39
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0bd65f4ed7fbd762
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\u0094')
string("api/v1/snapshots")
bool(true)
string("")
string("")
[]byte("0.\n")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0c0451781d419818
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\b')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0c2ffb305421ee46
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"\\f\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0c53b0029430293a
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"000\xc5\xce\xe50\x84\xd90\x870\xa50\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0c95dd94828ab2fd
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:֧")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0cdea7e8d392a935
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x0f')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("\"ŶťŶť\"")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0ce53b2b7da20de1
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/snapshots")
bool(true)
string("0")
string("")
[]byte("0e+")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0d022e6932852101
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/snapshots")
bool(true)
string("")
string("")
[]byte(" ")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0e1adf5329b5642a
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x02')
string("api/v1/snapshots/0# ")
bool(true)
string("0")
string("# ")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0e5eb5243423a0e0
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("")
string("//00000000000000000000000000000000000000000000000000")
[]byte("")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0f32cfa0135a71bb
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/admin/flush-journal")
bool(true)
string("0")
string("//:襥")
[]byte("0")
8 changes: 8 additions & 0 deletions internal/api/testdata/fuzz/FuzzWriteRequests/0fa4fb1bddca2dde
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
go test fuzz v1
bool(true)
byte('\x01')
string("api/v1/clones")
bool(true)
string("0")
string("")
[]byte("0.000000000")
Loading
Loading