Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
6d28e06
feat(api): create internal/api package structure
mattmattox Dec 31, 2025
7daedbe
feat(api): implement route registration with groups
mattmattox Dec 31, 2025
8d7ecae
feat(api): implement middleware (auth, CORS, logging)
mattmattox Dec 31, 2025
3b24cce
feat(api): define comprehensive JSON request/response types
mattmattox Dec 31, 2025
215587b
feat(api): implement /health endpoint with component health checks
mattmattox Dec 31, 2025
098f467
feat(api): implement /api/v1/status endpoint with version and capacit…
mattmattox Dec 31, 2025
a247e98
feat(api): integrate API server into mount command
mattmattox Dec 31, 2025
dd68b18
feat(api): add CLI flags for API server configuration
mattmattox Dec 31, 2025
68450f4
feat(fs): add SnapshotRegistry() accessor for API endpoints
mattmattox Dec 31, 2025
760ffe5
feat(stats): create internal/stats package structure
mattmattox Dec 31, 2025
3e619d7
feat(stats): add TrimStats, SFBStats, and ClusterInfo types
mattmattox Dec 31, 2025
707e96f
feat(api): implement stats endpoints for cache, journal, trim, lock, …
mattmattox Dec 31, 2025
f5beb8c
style(api): gofmt the API and stats code
mattmattox Nov 10, 2026
a8146bf
fix(api): bind to loopback by default and refuse remote access withou…
mattmattox Nov 10, 2026
33b0bdc
fix(mount): warn at startup when broadcast runs without a secret
mattmattox Nov 10, 2026
4c94506
fix(api): read the build version from internal/version
mattmattox Nov 10, 2026
f1bbcf1
docs: document the HTTP API flags, loopback default and new warnings
mattmattox Nov 10, 2026
79c2ee5
test(stats): cover the filesystem provider
mattmattox Nov 10, 2026
78c425c
test(api): create the image file before sizing it
mattmattox Nov 10, 2026
76d66a8
fix(mount): skip API shutdown when the API never started
mattmattox Nov 10, 2026
cbbcc4f
Merge remote-tracking branch 'origin/main' into claude/land-api-work-…
mattmattox Nov 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
- CI gates: unit tests with the race detector and coverage, golangci-lint (`make lint`), govulncheck (`make vuln`), and a `go mod tidy` check.
- Release workflow: pushing a `v*` tag publishes a GitHub Release with `linux/amd64` and `linux/arm64` archives, checksums, and the matching `CHANGELOG.md` section. `scripts/dist.sh` builds the same archives locally.
- `goclusterfs-fsck -version` prints the release version, commit, and commit date.
- HTTP API in `goclusterfs-mount` (`-api`, on by default): `/health`, `/metrics`, `/api/v1/status` and `/api/v1/stats/...`. It listens on `127.0.0.1:8080` (`-api-addr`). Mount refuses a non-loopback `-api-addr` without `-api-token`.
- `goclusterfs-mount -version`, which prints the same version string as `goclusterfs-fsck -version`. `/api/v1/status` reports it too.
- Mount logs a `WARNING` line at startup when `-broadcast` is on without `-broadcast-secret`.
- Design, deployment, operations, and troubleshooting docs in `docs/`.
- Apache License 2.0.

Expand Down
3 changes: 3 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ fusermount3 -u mnt # or Ctrl+C the mo
| `internal/trim` | Batched `BLKDISCARD` for freed blocks |
| `internal/fsck` | The fsck phases |
| `internal/metrics` | Atomic counters, not used by `cmd/mount` yet |
| `internal/api` | HTTP API served by `cmd/mount`: `/health`, `/metrics`, `/api/v1/status`, `/api/v1/stats/...`; loopback only unless `-api-token` is set |
| `internal/stats` | Stats types and collector behind `/api/v1/stats` |
| `internal/version` | Build version, set with `-ldflags -X` by `scripts/dist.sh` |

## On-Disk Layout

Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ These are the `goclusterfs-mount` flags. Run `./goclusterfs-mount -h` for the fu
| `-format` | `false` | Format the device, then mount it (destroys data) |
| `-bytes-per-inode` | `1048576` | With `-format`: one inode per this many bytes of device (4096 to 16,777,216 inodes) |
| `-debug` | `false` | Log every FUSE request |
| `-version` | `false` | Print the version and exit |

### Cache Options

Expand All @@ -113,6 +114,8 @@ These are the `goclusterfs-mount` flags. Run `./goclusterfs-mount -h` for the fu
| `-broadcast-secret` | (empty) | HMAC secret for message authentication |
| `-broadcast-iface` | (auto) | Network interface for multicast |

With `-broadcast` on and no `-broadcast-secret`, mount logs a `WARNING` line at startup: any host that can send to the port can make nodes drop their caches. Set the same secret on every node.

### TRIM Options

| Flag | Default | Description |
Expand All @@ -128,6 +131,17 @@ These are the `goclusterfs-mount` flags. Run `./goclusterfs-mount -h` for the fu
| `-sfb` | `true` | Ignored. SFB containers are disabled and every file uses full blocks |
| `-snapshots` | `true` | Enable O(1) file snapshots |

### HTTP API Options

| Flag | Default | Description |
|------|---------|-------------|
| `-api` | `true` | Serve the HTTP API (`/health`, `/metrics`, `/api/v1/...`) |
| `-api-addr` | `127.0.0.1:8080` | API listen address |
| `-api-token` | (empty) | Bearer token required on `/api/v1/...` |
| `-api-cors` | (empty) | Comma-separated CORS allowed origins |

The API only listens on this host by default. Mount refuses to start when `-api-addr` can be reached from other hosts and `-api-token` is empty. See [HTTP API Flags](docs/API-CONFIG.md#http-api-flags).

## Multi-Node Setup

For clustered deployments:
Expand Down
124 changes: 123 additions & 1 deletion cmd/mount/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ import (
"log"
"os"
"os/signal"
"strings"
"syscall"
"time"

"github.com/hanwen/go-fuse/v2/fs"
"github.com/hanwen/go-fuse/v2/fuse"

"github.com/supporttools/GoClusterFS/internal/api"
"github.com/supporttools/GoClusterFS/internal/bitmap"
"github.com/supporttools/GoClusterFS/internal/cache"
"github.com/supporttools/GoClusterFS/internal/cache/broadcast"
Expand All @@ -29,16 +31,74 @@ import (
"github.com/supporttools/GoClusterFS/internal/storage"
"github.com/supporttools/GoClusterFS/internal/superblock"
"github.com/supporttools/GoClusterFS/internal/trim"
"github.com/supporttools/GoClusterFS/internal/version"
)

// fsStatusProvider implements api.StatusProvider by wrapping the filesystem.
type fsStatusProvider struct {
mountPath string
devicePath string
dev *storage.Device
bitmapMgr *bitmap.Region
}

func (p *fsStatusProvider) MountPath() string { return p.mountPath }
func (p *fsStatusProvider) DevicePath() string { return p.devicePath }

func (p *fsStatusProvider) MountState() api.MountState {
// If we get here, the filesystem is mounted and running
return api.MountStateMounted
}

func (p *fsStatusProvider) Capacity() *api.CapacityInfo {
totalBlocks := p.bitmapMgr.TotalBlocks()
freeBlocks := p.bitmapMgr.FreeCount()
usedBlocks := totalBlocks - freeBlocks

const blockSize uint64 = 1024 * 1024 // 1MB blocks
totalBytes := totalBlocks * blockSize
usedBytes := usedBlocks * blockSize
freeBytes := freeBlocks * blockSize

var usedPercent float64
if totalBlocks > 0 {
usedPercent = float64(usedBlocks) / float64(totalBlocks) * 100
}

return &api.CapacityInfo{
TotalBytes: totalBytes,
UsedBytes: usedBytes,
FreeBytes: freeBytes,
UsedPercent: usedPercent,
TotalBlocks: totalBlocks,
UsedBlocks: usedBlocks,
FreeBlocks: freeBlocks,
BlockSizeBytes: blockSize,
}
}

func apiConfig(addr, token, corsList string) api.Config {
var origins []string
for _, origin := range strings.Split(corsList, ",") {
if origin = strings.TrimSpace(origin); origin != "" {
origins = append(origins, origin)
}
}
return api.DefaultConfig().
WithListenAddr(addr).
WithAuthToken(token).
WithCORSOrigins(origins...)
}

func main() {
var (
devicePath = flag.String("device", "", "Path to block device or image file")
mountPoint = flag.String("mount", "", "Mount point directory")
format = flag.Bool("format", false, "Format the device (WARNING: destroys data)")
inodeRatio = flag.Uint64("bytes-per-inode", superblock.DefaultBytesPerInode,
fmt.Sprintf("With -format: one inode per this many bytes of device (%d to %d inodes)", superblock.MinInodes, superblock.MaxInodes))
debug = flag.Bool("debug", false, "Enable FUSE debug output")
debug = flag.Bool("debug", false, "Enable FUSE debug output")
showVer = flag.Bool("version", false, "Print version and exit")

// Cache configuration
cacheStrict = flag.Bool("cache-strict", false, "Enable strict cache mode (always validate)")
Expand Down Expand Up @@ -66,15 +126,36 @@ func main() {

// Snapshot configuration
snapshotsEnabled = flag.Bool("snapshots", true, "Enable O(1) file snapshots via xattr")

// API server configuration
apiEnabled = flag.Bool("api", true, "Enable HTTP API server for management")
apiAddr = flag.String("api-addr", "127.0.0.1:8080", "API server listen address; any non-loopback address needs -api-token")
apiToken = flag.String("api-token", "", "Bearer token for API authentication (empty=disabled, loopback only)")
apiCORS = flag.String("api-cors", "", "Comma-separated CORS allowed origins (empty=disabled)")
)
flag.Parse()

if *showVer {
fmt.Println("goclusterfs-mount", version.String())
os.Exit(0)
}

if *devicePath == "" || *mountPoint == "" {
fmt.Fprintf(os.Stderr, "Usage: %s -device <path> -mount <mountpoint> [-format] [-debug]\n", os.Args[0])
flag.PrintDefaults()
os.Exit(1)
}

apiCfg := apiConfig(*apiAddr, *apiToken, *apiCORS)
if *apiEnabled {
if err := apiCfg.CheckExposure(); err != nil {
log.Fatalf("Refusing to start the API: %v. Set -api-token, set -api-addr to a loopback address such as 127.0.0.1:8080, or pass -api=false", err)
}
}
if *broadcastEnabled && *broadcastSecret == "" {
log.Printf("WARNING: -broadcast is on without -broadcast-secret. Broadcast messages are not authenticated: any host that can send UDP to port %d can make this node drop its caches and add itself as a peer. Set the same -broadcast-secret on every node, or pass -broadcast=false.", *multicastPort)
}

// Check mount point exists
if _, err := os.Stat(*mountPoint); os.IsNotExist(err) {
log.Fatalf("Mount point does not exist: %s", *mountPoint)
Expand Down Expand Up @@ -351,13 +432,47 @@ func main() {
log.Fatalf("Failed to mount: %v", err)
}

// Start API server for management and monitoring
var apiServer *api.Server
if *apiEnabled {
var err error
apiServer, err = api.NewServer(apiCfg)
if err != nil {
log.Printf("Warning: Failed to create API server: %v", err)
} else {
apiServer.SetStatusProvider(&fsStatusProvider{
mountPath: *mountPoint,
devicePath: *devicePath,
dev: dev,
bitmapMgr: bm,
})
apiServer.RegisterHealthChecker(api.DefaultHealthChecker())
if err := apiServer.Start(ctx); err != nil {
log.Printf("Warning: Failed to start API server: %v", err)
apiServer = nil
} else {
log.Printf("API server listening on %s (auth=%v, cors=%v)",
apiServer.Addr(), apiCfg.AuthToken != "", len(apiCfg.CORSOrigins) > 0)
}
}
} else {
log.Println("API server: disabled")
}

// Handle signals for clean unmount
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigChan
log.Println("Received signal, unmounting...")

// Stop API server first
if apiServer != nil {
if err := apiServer.Stop(); err != nil {
log.Printf("Warning: Failed to stop API server: %v", err)
}
}

// Close filesystem (stops snapshot worker, TRIM, journal, etc.)
nodeReg.Stop()
if err := gfs.Close(); err != nil {
Expand All @@ -378,6 +493,13 @@ func main() {
// Wait for filesystem to be unmounted - this blocks until unmount
server.Wait()

// Stop API server
if apiServer != nil {
if err := apiServer.Stop(); err != nil {
log.Printf("Warning: Failed to stop API server on exit: %v", err)
}
}

// Clean shutdown - close filesystem (handles snapshot worker, TRIM, journal)
nodeReg.Stop()
if err := gfs.Close(); err != nil {
Expand Down
18 changes: 17 additions & 1 deletion docs/API-CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ These flags are passed when starting `goclusterfs`:
| `-format` | bool | `false` | Format the device (destroys all data) |
| `-bytes-per-inode` | uint | `1048576` | With `-format`: one inode per this many bytes of device, 4096 to 16,777,216 inodes |
| `-debug` | bool | `false` | Enable verbose debug logging |
| `-version` | bool | `false` | Print the version, commit and commit date, then exit |

### Cache Flags

Expand Down Expand Up @@ -81,6 +82,21 @@ The kernel caches file attributes and directory entries for `-attr-timeout`. For
| `-sfb` | bool | `true` | Ignored. SFB containers are disabled |
| `-snapshots` | bool | `true` | Enable O(1) file snapshots |

### HTTP API Flags

| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `-api` | bool | `true` | Serve the HTTP API |
| `-api-addr` | string | `127.0.0.1:8080` | Listen address |
| `-api-token` | string | (empty) | Bearer token for `/api/v1/...` requests |
| `-api-cors` | string | (empty) | Comma-separated CORS allowed origins, or `*` |

The API serves `GET /health`, `GET /metrics`, `GET /api/v1/status` and `GET /api/v1/stats/...`. Mount does not feed statistics to the API yet, so `/api/v1/stats/...` answers 503, `/health` only checks the API itself, and `/metrics` only reports `goclusterfs_up`. With `-api-token` set, `/api/v1/...` requests need `Authorization: Bearer <token>`. `/health` and `/metrics` never need the token.

By default the API only accepts connections from this host. To serve other hosts, set `-api-addr` (for example `:8080` or `10.0.0.5:8080`) and `-api-token`. Mount exits before it opens the device when the address is not loopback (`127.0.0.0/8`, `::1` or `localhost`) and the token is empty. The token is a command-line flag, so local users can read it with `ps`.

Each mount on a host needs its own `-api-addr`. When the port is taken, mount logs a warning and runs without the API.

---

## Cache Configuration
Expand Down Expand Up @@ -413,7 +429,7 @@ Interval between heartbeat messages to detect node failures.
|----------|-------|
| Default | (empty) |

HMAC secret for message authentication. All nodes must use the same secret.
HMAC secret for message authentication. All nodes must use the same secret. With broadcast on and no secret, mount logs a `WARNING` line at startup.

**Generating a secret**:
```bash
Expand Down
16 changes: 14 additions & 2 deletions docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ newgrp fuse
|-------------|---------|
| Connectivity | All nodes on same L2 network segment |
| Multicast | UDP multicast between all nodes. Unicast is only a fallback to peers learned over multicast. |
| Ports | UDP 31337 (configurable) |
| Ports | UDP 31337 (configurable). TCP 8080 on loopback for the HTTP API; open it only if you expose the API |
| Latency | < 10ms between nodes recommended |

---
Expand All @@ -87,7 +87,7 @@ sudo cp goclusterfs-mount goclusterfs-fsck /usr/local/bin/
### Verify Build

```bash
./goclusterfs-mount -h
./goclusterfs-mount -version
./goclusterfs-fsck -version
```

Expand Down Expand Up @@ -417,6 +417,18 @@ openssl rand -hex 32

See the [Security Model](OPERATIONS.md#security-model) for what this does and does not protect.

Mount logs a `WARNING` line at startup when broadcast is on without a secret.

### HTTP API Access

The API listens on `127.0.0.1:8080` by default. To scrape `/metrics` or call the API from another host, set a token and a reachable address:

```bash
./goclusterfs-mount ... -api-addr :8080 -api-token "$(openssl rand -hex 32)"
```

Mount refuses a non-loopback `-api-addr` without `-api-token`. See [HTTP API Flags](API-CONFIG.md#http-api-flags).

### Firewall Rules

```bash
Expand Down
11 changes: 9 additions & 2 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -522,13 +522,20 @@ GoClusterFS assumes every node, and everything that can reach the shared device

**Broadcast cache invalidation.** Nodes send UDP messages (multicast group `239.192.74.75`, port `31337` by default) that tell peers to drop cached inodes and extents. Heartbeat and announce messages add unicast peers, which a node uses when multicast sends fail. Messages carry inode numbers but no file data. Losing them only delays cache updates until `-cache-ttl`.

- With an empty `-broadcast-secret` (the default), messages are not authenticated. Any host that can send to the port can make nodes drop their caches, and can add any address as a unicast peer.
- With an empty `-broadcast-secret` (the default), messages are not authenticated. Any host that can send to the port can make nodes drop their caches, and can add any address as a unicast peer. Mount logs a `WARNING` line at startup in this case.
- With `-broadcast-secret`, every message carries an HMAC-SHA256 tag. Nodes drop messages with a missing or wrong tag, so all nodes need the same secret.
- The only replay protection is a sequence number check that forgets messages after 5 minutes, so a recorded message can be sent again later, even with a secret.
- Messages are never encrypted. Any host that joins the multicast group can see which inodes change.
- The secret is a command-line flag, so local users on a node can read it with `ps`.

**Network listeners.** GoClusterFS has no HTTP API. Its only listener is UDP port 31337 on all interfaces, plus the multicast group. Limit that port to the cluster nodes with a firewall.
**HTTP API.** Each mount serves an HTTP API on `127.0.0.1:8080` (`-api-addr`), so only local users can reach it. It only answers `GET` requests and changes nothing.

- To expose it, set `-api-addr` and `-api-token`. Mount refuses a non-loopback address without a token.
- `/health` and `/metrics` never need the token, so anyone who can reach the port can read them.
- The API uses plain HTTP. Put a TLS proxy in front of it if the token crosses an untrusted network.
- The token is a command-line flag, so local users can read it with `ps`.

**Network listeners.** Each node listens on UDP port 31337 on all interfaces, plus the multicast group, and on TCP `-api-addr`. Limit the UDP port, and the API port if you expose it, to the cluster nodes and your monitoring hosts with a firewall.

---

Expand Down
4 changes: 2 additions & 2 deletions docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -544,8 +544,8 @@ If this guide doesn't resolve your issue:
uname -a
cat /etc/os-release

# GoClusterFS version (goclusterfs-mount has no -version flag yet)
./goclusterfs-fsck -version
# GoClusterFS version
./goclusterfs-mount -version

# Recent mount log
tail -1000 /var/log/goclusterfs.log > mount.log
Expand Down
Loading
Loading