Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
name: Bug report
about: Report a crash, data problem, or wrong behavior
title: ""
labels: bug
assignees: ""
---

<!-- Security problem? Do not file it here. See SECURITY.md. -->

## What happened

<!-- What you saw, and what you expected instead. -->

## Steps to reproduce

1.
2.
3.

## Environment

- GoClusterFS commit (`git rev-parse HEAD`) or release tag:
- Kernel (`uname -r`):
- Distribution:
- FUSE version (`fusermount3 --version` or `fusermount --version`):
- Number of nodes:
- Shared storage type (for example iSCSI, NVMe-oF, SAN, file image on NFS, local image file):
- Filesystem under the image, if you use an image file:

## Mount command

<!-- The full goclusterfs-mount command line on each node, for example:
goclusterfs-mount -device /dev/sdb -mount /mnt/gcfs -journal -broadcast -->

```
```

## Logs

<!-- goclusterfs-mount output from each node, and fsck output if you ran it.
Mark which node each log comes from. Remove secrets such as -broadcast-secret. -->

```
```
23 changes: 23 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
name: Feature request
about: Suggest a new feature or a change in behavior
title: ""
labels: enhancement
assignees: ""
---

## Problem

<!-- What are you trying to do, and what stops you today? -->

## Proposed change

<!-- What should GoClusterFS do? -->

## Alternatives

<!-- Other ways to solve the problem, including workarounds you use now. -->

## Cluster impact

<!-- Does this change the on-disk format, the locking, the journal, or how nodes talk to each other? Say "unsure" if you don't know. -->
22 changes: 22 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
## What

<!-- What this pull request changes. Keep it to one concern. -->

## Why

<!-- The problem it solves. Link the issue if there is one. -->

## Tests

<!-- Check what you ran. Say why if you skipped a suite that covers your change. -->

- [ ] `make test-unit`
- [ ] `make test-integration`
- [ ] `make test-multinode-docker`
- [ ] `make test-multinode-vm`
- [ ] `make test-parity`

## Checklist

- [ ] New test scenarios are in `tests/test-manifest.yaml`, or this pull request adds no tests
- [ ] The title follows [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/)
54 changes: 54 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Upgrade notes

- Upgrade all nodes together: unmount on every node, upgrade, then mount again. Don't mount one shared device with a mix of old and new binaries.
- Older binaries reset an inode's generation counters to 0 when they free it, so other nodes can serve stale data after the inode number is reused.
- Older binaries zero the extent-tree generation (inode bytes 188–195) when they rewrite an inode. Newer binaries then cannot read that file if it has more than two extents, because such files use an extent tree.

### Added

- `goclusterfs-mount`: a FUSE filesystem that several Linux nodes can mount from one shared block device.
- `O_DIRECT` device I/O, split into shards so requests can run in parallel.
- Distributed compare-and-swap locks on the shared device for block allocation, inode updates, and directory changes. File writes do not take these locks yet, so two nodes extending the same file can lose each other's data.
- Write-ahead journal with batch commits and a recovery check on mount.
- Intent logging for multi-step operations.
- Inode cache with generation-based coherency checks, and an extent cache.
- UDP multicast cache invalidation between nodes, with optional HMAC authentication (`-broadcast-secret`).
- Batched TRIM of freed blocks (`-trim`).
- Experimental: SFB containers for files under 64 KB (`-sfb`). Container I/O uses unaligned buffers that `O_DIRECT` rejects, so files fall back to normal extents.
- Experimental: file snapshots and writable clones through `user.snapshot*` and `user.clone` extended attributes. Copy-on-write does not trigger yet, so a snapshot does not preserve data, and an in-place write to a clone changes the source file.
- Experimental: `goclusterfs-fsck` consistency checker with a repair mode. It fails on newly formatted volumes because `-format` does not write a superblock.
- Benchmark suite (`tests/benchmark`) and an `internal/metrics` package.
- Go, Docker, and QEMU VM test suites, with `tests/test-manifest.yaml` and `make test-parity` to track coverage across them.
- Fuzz tests for the on-disk decoders (superblock, inode, snapshot descriptor, journal superblock and records) and the journal recovery scan.
- CI gates: unit tests with the race detector and coverage, golangci-lint (`make lint`), govulncheck (`make vuln`), and a `go mod tidy` check.
- Design, deployment, operations, and troubleshooting docs in `docs/`.
- Apache License 2.0.

### Changed

- Moved the FUSE library from `bazil.org/fuse` to `hanwen/go-fuse` v2.
- `go.mod` now requires Go 1.26.
- `make build` builds `goclusterfs-fsck` as well as `goclusterfs-mount`.
- Reads use the inode cache, lock acquisition reuses cached lock sequence numbers, and the inode and extent caches are split into 16 shards.

### Removed

- The legacy single-block prototype in the repository root (`main.go`).

### Fixed

- A node could serve a stale cached inode after another node freed and reused its inode number. Inode generations now survive free and reuse, and opening a file reloads its inode from disk.
- Journal recovery no longer panics, loops forever, or allocates gigabytes on a corrupt journal. It now rejects short entries and bad ring geometry as corrupt.
- Lock-order violations when several nodes create files and directories at the same time.
- Inode allocation races between nodes; allocation now retries with jittered backoff.
- Directory operations now update caches and send broadcast invalidations.
- Docker and VM test suites now run on fresh hosts.
- The README quick start now uses the real binary names and flags.
83 changes: 83 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

## Our Standards

Examples of behavior that contributes to a positive environment for our community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.

Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement through GitHub: use [private reporting](https://github.com/SupportTools/GoClusterFS/security/advisories/new) on this repository (**Security** tab, **Report a vulnerability**), or contact a maintainer through GitHub. All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series of actions.

**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].

Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].

For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].

[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
69 changes: 69 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Contributing to GoClusterFS

Thanks for helping. This guide covers how to build, test, and send changes.

Report security problems as described in [SECURITY.md](SECURITY.md), not in issues.

## What you need

GoClusterFS builds and runs on Linux only. The storage layer opens devices with `O_DIRECT`, and the filesystem runs through FUSE. It does not build on macOS or Windows.

- The Go version in `go.mod` (1.26 or later)
- FUSE and access to `/dev/fuse`, for integration tests
- The rest of the [Requirements](README.md#requirements) in the README

## Build

```bash
make build
```

This builds `goclusterfs-mount` and `goclusterfs-fsck` in the repository root.

## Test

| Command | What it runs | Host needs |
|---|---|---|
| `make test-unit` | Unit tests in `internal/` | Go |
| `make test-integration` | Single-host FUSE tests in `tests/integration/` | FUSE |
| `make test-multinode-docker` | Multi-node tests in Docker containers | Docker and `docker-compose`; the containers run privileged with `/dev/fuse` |
| `make test-multinode-vm` | Multi-node tests in QEMU VMs, including power failure | x86_64 host with KVM and QEMU, see [Testing](README.md#testing) |
| `make test-parity` | Checks that each test scenario exists in every environment | None |

Run `make test-unit` and `make test-integration` before you open a pull request. Also run the Docker or VM suite when you change multi-node behavior, such as locking, the journal, caches, broadcast, or recovery.

On every pull request, CI runs:

- `go vet` and a `gofmt` check
- golangci-lint (`make lint`)
- govulncheck (`make vuln`)
- a check that `go mod tidy` changes nothing
- unit tests with `-race` and coverage, integration tests, and the Docker suite

Run `make fmt`, `make vet`, and `make lint` before you push.

## Adding tests

[`tests/test-manifest.yaml`](tests/test-manifest.yaml) lists every test scenario. It is the single source of truth for test coverage across the Go, Docker, and VM environments.

When you add a test:

1. Add the scenario to `tests/test-manifest.yaml` first.
2. Write the Go integration test.
3. Write the Docker test, or set the `docker` entry to `TODO` and say why in the pull request.
4. Write the VM test, or set the `vm` entry to `TODO` and say why in the pull request.
5. Run `make test-parity`.

The comments at the top of the manifest explain the priority levels and the entry format.

For tests where several nodes write to one file at the same time, create the file at its final size before the writers start, and give each node its own offset. This avoids races in the distributed locking.

## Commits and pull requests

- Use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) for commit messages and pull request titles, for example `fix(journal): replay batches in order`. Common types here are `feat`, `fix`, `perf`, `test`, `docs`, `ci`, and `chore`.
- Keep each pull request to one concern. Send a refactor, a fix, and a feature as separate pull requests.
- Fill in the pull request template: what changed, why, and which test suites you ran.

## License

By sending a contribution, you agree that it is licensed under the [Apache License 2.0](LICENSE).
26 changes: 26 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Security Policy

## Supported versions

GoClusterFS has no release yet. Only the `main` branch gets security fixes. This section will list supported releases after the first one ships.

## Report a vulnerability

Do not report security problems in public issues, pull requests, or discussions.

Report them privately through GitHub:

1. Open the repository's **Security** tab.
2. Click **Report a vulnerability**.
3. Fill in the form.

Only you and the maintainers can see the report. We reply and work on the fix with you there.

If the **Report a vulnerability** button is missing, open an issue that asks a maintainer to contact you. Leave out all details of the problem.

## What to include

- The commit you tested (`git rev-parse HEAD`)
- What an attacker can do, and what access they need (for example, a local user on a node, or a host on the broadcast network)
- Steps to reproduce, or a proof of concept
- Kernel version, shared storage type, and mount flags, if they matter
Loading