From 87634393f8d841f6b1bae8f1f02e6ba49241bf26 Mon Sep 17 00:00:00 2001 From: Matthew Mattox Date: Sat, 24 Oct 2026 07:57:45 -0500 Subject: [PATCH 1/4] docs: add security policy, contributing guide, code of conduct, changelog, and templates Adds the community files GitHub expects for a public release: SECURITY.md (GitHub private vulnerability reporting, main only), CONTRIBUTING.md, CODE_OF_CONDUCT.md (Contributor Covenant 2.1), CHANGELOG.md with an Unreleased section, issue templates, and a pull request template. Co-Authored-By: Claude Opus 5.5 --- .github/ISSUE_TEMPLATE/bug_report.md | 45 ++++++++++++ .github/ISSUE_TEMPLATE/feature_request.md | 23 +++++++ .github/pull_request_template.md | 22 ++++++ CHANGELOG.md | 38 +++++++++++ CODE_OF_CONDUCT.md | 83 +++++++++++++++++++++++ CONTRIBUTING.md | 61 +++++++++++++++++ SECURITY.md | 26 +++++++ 7 files changed, 298 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md create mode 100644 .github/pull_request_template.md create mode 100644 CHANGELOG.md create mode 100644 CODE_OF_CONDUCT.md create mode 100644 CONTRIBUTING.md create mode 100644 SECURITY.md diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000..7b05057a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,45 @@ +--- +name: Bug report +about: Report a crash, data problem, or wrong behavior +title: "" +labels: bug +assignees: "" +--- + + + +## What happened + + + +## Steps to reproduce + +1. +2. +3. + +## Environment + +- GoClusterFS commit (`git rev-parse HEAD`) or release tag: +- Kernel (`uname -r`): +- Distribution: +- FUSE version (`fusermount3 --version` or `fusermount --version`): +- Number of nodes: +- Shared storage type (for example iSCSI, NVMe-oF, SAN, file image on NFS, local image file): +- Filesystem under the image, if you use an image file: + +## Mount command + + + +``` +``` + +## Logs + + + +``` +``` diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000..29ebe499 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,23 @@ +--- +name: Feature request +about: Suggest a new feature or a change in behavior +title: "" +labels: enhancement +assignees: "" +--- + +## Problem + + + +## Proposed change + + + +## Alternatives + + + +## Cluster impact + + diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..cca5f670 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,22 @@ +## What + + + +## Why + + + +## Tests + + + +- [ ] `make test-unit` +- [ ] `make test-integration` +- [ ] `make test-multinode-docker` +- [ ] `make test-multinode-vm` +- [ ] `make test-parity` + +## Checklist + +- [ ] New test scenarios are in `tests/test-manifest.yaml`, or this pull request adds no tests +- [ ] The title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..53cc3fee --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,38 @@ +# Changelog + +All notable changes to this project are documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added + +- `goclusterfs-mount`: a FUSE filesystem that several Linux nodes can mount from one shared block device. +- `O_DIRECT` device I/O, split into shards so requests can run in parallel. +- Distributed locks stored on the shared device, using compare-and-swap. +- Write-ahead journal with batch commits and a recovery check on mount. +- Intent logging for multi-step operations. +- Inode cache with generation-based coherency checks, and an extent cache. +- UDP multicast cache invalidation between nodes, with optional HMAC authentication (`-broadcast-secret`). +- Batched TRIM of freed blocks (`-trim`). +- SFB containers that pack files under 64 KB (`-sfb`). +- Copy-on-write file snapshots and writable clones, managed through `user.snapshot*` and `user.clone` extended attributes. +- `fsck` consistency checker with a repair mode (`cmd/fsck`). +- Benchmark suite (`tests/benchmark`) and an `internal/metrics` package. +- Go, Docker, and QEMU VM test suites, with `tests/test-manifest.yaml` and `make test-parity` to track coverage across them. +- Design, deployment, operations, and troubleshooting docs in `docs/`. +- Apache License 2.0. + +### Changed + +- Moved the FUSE library from `bazil.org/fuse` to `hanwen/go-fuse` v2. +- `go.mod` now requires Go 1.26. +- Reads use the inode cache, lock acquisition reuses cached lock sequence numbers, and the inode and extent caches are split into 16 shards. + +### Fixed + +- Lost updates and lock-order violations when several nodes create files and directories, or write to the same file, at the same time. +- Inode allocation races between nodes; allocation now retries with jittered backoff. +- Directory operations now update caches and send broadcast invalidations. +- Docker and VM test suites now run on fresh hosts. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..52aa23f5 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,83 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement through GitHub: use [private reporting](https://github.com/SupportTools/GoClusterFS/security/advisories/new) on this repository (**Security** tab, **Report a vulnerability**), or contact a maintainer through GitHub. All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of actions. + +**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC]. + +For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations]. + +[homepage]: https://www.contributor-covenant.org +[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html +[Mozilla CoC]: https://github.com/mozilla/diversity +[FAQ]: https://www.contributor-covenant.org/faq +[translations]: https://www.contributor-covenant.org/translations diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..1f4df149 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,61 @@ +# Contributing to GoClusterFS + +Thanks for helping. This guide covers how to build, test, and send changes. + +Report security problems as described in [SECURITY.md](SECURITY.md), not in issues. + +## What you need + +GoClusterFS builds and runs on Linux only. The storage layer opens devices with `O_DIRECT`, and the filesystem runs through FUSE. It does not build on macOS or Windows. + +- The Go version in `go.mod` (1.26 or later) +- FUSE and access to `/dev/fuse`, for integration tests +- The rest of the [Requirements](README.md#requirements) in the README + +## Build + +```bash +make build +``` + +This builds `goclusterfs-mount` in the repository root. + +## Test + +| Command | What it runs | Host needs | +|---|---|---| +| `make test-unit` | Unit tests in `internal/` | Go | +| `make test-integration` | Single-host FUSE tests in `tests/integration/` | FUSE | +| `make test-multinode-docker` | Multi-node tests in Docker containers | Docker and `docker-compose`; the containers run privileged with `/dev/fuse` | +| `make test-multinode-vm` | Multi-node tests in QEMU VMs, including power failure | x86_64 host with KVM and QEMU, see [Testing](README.md#testing) | +| `make test-parity` | Checks that each test scenario exists in every environment | None | + +Run `make test-unit` and `make test-integration` before you open a pull request. Also run the Docker or VM suite when you change multi-node behavior, such as locking, the journal, caches, broadcast, or recovery. + +CI runs `go vet`, a `gofmt` check, the unit and integration tests, and the Docker suite on every pull request. Run `make fmt` and `make vet` before you push. + +## Adding tests + +[`tests/test-manifest.yaml`](tests/test-manifest.yaml) lists every test scenario. It is the single source of truth for test coverage across the Go, Docker, and VM environments. + +When you add a test: + +1. Add the scenario to `tests/test-manifest.yaml` first. +2. Write the Go integration test. +3. Write the Docker test, or set the `docker` entry to `TODO` and say why in the pull request. +4. Write the VM test, or set the `vm` entry to `TODO` and say why in the pull request. +5. Run `make test-parity`. + +The comments at the top of the manifest explain the priority levels and the entry format. + +For tests where several nodes write to one file at the same time, create the file at its final size before the writers start, and give each node its own offset. This avoids races in the distributed locking. + +## Commits and pull requests + +- Use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) for commit messages and pull request titles, for example `fix(journal): replay batches in order`. Common types here are `feat`, `fix`, `perf`, `test`, `docs`, `ci`, and `chore`. +- Keep each pull request to one concern. Send a refactor, a fix, and a feature as separate pull requests. +- Fill in the pull request template: what changed, why, and which test suites you ran. + +## License + +By sending a contribution, you agree that it is licensed under the [Apache License 2.0](LICENSE). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..e72c43dd --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,26 @@ +# Security Policy + +## Supported versions + +GoClusterFS has no release yet. Only the `main` branch gets security fixes. This section will list supported releases after the first one ships. + +## Report a vulnerability + +Do not report security problems in public issues, pull requests, or discussions. + +Report them privately through GitHub: + +1. Open the repository's **Security** tab. +2. Click **Report a vulnerability**. +3. Fill in the form. + +Only you and the maintainers can see the report. We reply and work on the fix with you there. + +If the **Report a vulnerability** button is missing, open an issue that asks a maintainer to contact you. Leave out all details of the problem. + +## What to include + +- The commit you tested (`git rev-parse HEAD`) +- What an attacker can do, and what access they need (for example, a local user on a node, or a host on the broadcast network) +- Steps to reproduce, or a proof of concept +- Kernel version, shared storage type, and mount flags, if they matter From f227ba64e2c0c499ef7f6cab2379a8b148962859 Mon Sep 17 00:00:00 2001 From: Matthew Mattox Date: Sat, 24 Oct 2026 08:12:42 -0500 Subject: [PATCH 2/4] docs: correct build, CI, and changelog claims in community files Write CONTRIBUTING against main after the CI gates PR: make build also builds goclusterfs-fsck, and CI adds golangci-lint, govulncheck, a go mod tidy check, and race/coverage unit tests. In the changelog, mark SFB, snapshots/clones, and fsck as experimental with their known issues, say file writes skip the distributed locks, and narrow the concurrency fix to create/mkdir lock ordering. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 10 +++++----- CONTRIBUTING.md | 12 ++++++++++-- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 53cc3fee..58156383 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,15 +10,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - `goclusterfs-mount`: a FUSE filesystem that several Linux nodes can mount from one shared block device. - `O_DIRECT` device I/O, split into shards so requests can run in parallel. -- Distributed locks stored on the shared device, using compare-and-swap. +- Distributed compare-and-swap locks on the shared device for block allocation, inode updates, and directory changes. File writes do not take these locks yet, so two nodes extending the same file can lose each other's data. - Write-ahead journal with batch commits and a recovery check on mount. - Intent logging for multi-step operations. - Inode cache with generation-based coherency checks, and an extent cache. - UDP multicast cache invalidation between nodes, with optional HMAC authentication (`-broadcast-secret`). - Batched TRIM of freed blocks (`-trim`). -- SFB containers that pack files under 64 KB (`-sfb`). -- Copy-on-write file snapshots and writable clones, managed through `user.snapshot*` and `user.clone` extended attributes. -- `fsck` consistency checker with a repair mode (`cmd/fsck`). +- Experimental: SFB containers for files under 64 KB (`-sfb`). Container I/O uses unaligned buffers that `O_DIRECT` rejects, so files fall back to normal extents. +- Experimental: file snapshots and writable clones through `user.snapshot*` and `user.clone` extended attributes. Copy-on-write does not trigger yet, so a snapshot does not preserve data, and an in-place write to a clone changes the source file. +- Experimental: `goclusterfs-fsck` consistency checker with a repair mode. It fails on newly formatted volumes because `-format` does not write a superblock. - Benchmark suite (`tests/benchmark`) and an `internal/metrics` package. - Go, Docker, and QEMU VM test suites, with `tests/test-manifest.yaml` and `make test-parity` to track coverage across them. - Design, deployment, operations, and troubleshooting docs in `docs/`. @@ -32,7 +32,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Fixed -- Lost updates and lock-order violations when several nodes create files and directories, or write to the same file, at the same time. +- Lock-order violations when several nodes create files and directories at the same time. - Inode allocation races between nodes; allocation now retries with jittered backoff. - Directory operations now update caches and send broadcast invalidations. - Docker and VM test suites now run on fresh hosts. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1f4df149..51474303 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -18,7 +18,7 @@ GoClusterFS builds and runs on Linux only. The storage layer opens devices with make build ``` -This builds `goclusterfs-mount` in the repository root. +This builds `goclusterfs-mount` and `goclusterfs-fsck` in the repository root. ## Test @@ -32,7 +32,15 @@ This builds `goclusterfs-mount` in the repository root. Run `make test-unit` and `make test-integration` before you open a pull request. Also run the Docker or VM suite when you change multi-node behavior, such as locking, the journal, caches, broadcast, or recovery. -CI runs `go vet`, a `gofmt` check, the unit and integration tests, and the Docker suite on every pull request. Run `make fmt` and `make vet` before you push. +On every pull request, CI runs: + +- `go vet` and a `gofmt` check +- golangci-lint (`make lint`) +- govulncheck (`make vuln`) +- a check that `go mod tidy` changes nothing +- unit tests with `-race` and coverage, integration tests, and the Docker suite + +Run `make fmt`, `make vet`, and `make lint` before you push. ## Adding tests From 879e0f6720648c2064c6ee8142f6c6c8bbeeda7d Mon Sep 17 00:00:00 2001 From: Matthew Mattox Date: Sat, 24 Oct 2026 09:34:44 -0500 Subject: [PATCH 3/4] docs(changelog): add upgrade notes for mixed-version clusters Older binaries zero an inode's generations on free and its extent-tree generation on rewrite, so all nodes must be upgraded together. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 58156383..0d001f1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ## [Unreleased] +### Upgrade notes + +- Upgrade all nodes together: unmount on every node, upgrade, then mount again. Don't mount one shared device with a mix of old and new binaries. + - Older binaries reset an inode's generation counters to 0 when they free it, so other nodes can serve stale data after the inode number is reused. + - Older binaries zero the extent-tree generation (inode bytes 188–195) when they rewrite an inode. Newer binaries then cannot read that file if it has more than two extents, because such files use an extent tree. + ### Added - `goclusterfs-mount`: a FUSE filesystem that several Linux nodes can mount from one shared block device. From d1a8200a1db18d9812b673a69714192d20a38fdc Mon Sep 17 00:00:00 2001 From: Matthew Mattox Date: Sat, 24 Oct 2026 10:03:08 -0500 Subject: [PATCH 4/4] docs(changelog): add CI gates, prototype removal, stale-inode and journal fixes Cover PRs #5, #6, #7 and #10, which merged to main. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d001f1a..2d43a4ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - Experimental: `goclusterfs-fsck` consistency checker with a repair mode. It fails on newly formatted volumes because `-format` does not write a superblock. - Benchmark suite (`tests/benchmark`) and an `internal/metrics` package. - Go, Docker, and QEMU VM test suites, with `tests/test-manifest.yaml` and `make test-parity` to track coverage across them. +- Fuzz tests for the on-disk decoders (superblock, inode, snapshot descriptor, journal superblock and records) and the journal recovery scan. +- CI gates: unit tests with the race detector and coverage, golangci-lint (`make lint`), govulncheck (`make vuln`), and a `go mod tidy` check. - Design, deployment, operations, and troubleshooting docs in `docs/`. - Apache License 2.0. @@ -34,11 +36,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - Moved the FUSE library from `bazil.org/fuse` to `hanwen/go-fuse` v2. - `go.mod` now requires Go 1.26. +- `make build` builds `goclusterfs-fsck` as well as `goclusterfs-mount`. - Reads use the inode cache, lock acquisition reuses cached lock sequence numbers, and the inode and extent caches are split into 16 shards. +### Removed + +- The legacy single-block prototype in the repository root (`main.go`). + ### Fixed +- A node could serve a stale cached inode after another node freed and reused its inode number. Inode generations now survive free and reuse, and opening a file reloads its inode from disk. +- Journal recovery no longer panics, loops forever, or allocates gigabytes on a corrupt journal. It now rejects short entries and bad ring geometry as corrupt. - Lock-order violations when several nodes create files and directories at the same time. - Inode allocation races between nodes; allocation now retries with jittered backoff. - Directory operations now update caches and send broadcast invalidations. - Docker and VM test suites now run on fresh hosts. +- The README quick start now uses the real binary names and flags.