Skip to content

feat(api): land the HTTP API with loopback-only defaults - #48

Merged
mattmattox merged 21 commits into
mainfrom
claude/land-api-work-c8b883
Nov 10, 2026
Merged

mattmattox merged 21 commits into
mainfrom
claude/land-api-work-c8b883

Conversation

@mattmattox

@mattmattox mattmattox commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Lands the user's HTTP API work (12 commits from 2025-12-31, authorship kept) and makes it safe by default. The commits cherry-picked onto 0fca640 without conflicts, because the user had already rebased them onto Oct 24 main.

Added on top:

  • -api-addr defaults to 127.0.0.1:8080. Mount exits before opening the device when the address is reachable from other hosts and -api-token is empty.
  • A WARNING line at startup when broadcast runs without -broadcast-secret.
  • internal/api reads internal/version. New goclusterfs-mount -version.
  • gofmt for the cherry-picked code, and a test for a stats mock that lint flagged as unused.

For review:

Results (head 76d66a8):

  • CI: all 8 checks pass, including integration (3m44s) and both Docker groups.
  • .68: build, vet, gofmt, go mod tidy -diff, golangci-lint (0 issues), parity and go test -race ./internal/... ./cmd/... all pass. Integration -short: 94 pass, 17 skip, 0 fail.
  • Manual on .68: -api-addr :18080, 0.0.0.0:18080 and the host IP without a token exit 1 and leave the image zeroed. On 127.0.0.1, /health answers 200 and the host IP gets connection refused. Exposed with a token, /api/v1/status returns 401 without the token or with a wrong one, and 200 with it. The broadcast WARNING appears only without a secret. With 8080 already taken, mount warns and works.

The review follow-ups (CORS *, Host check, flag validation order) are on claude/api-hardening, a separate PR into this branch.

🤖 Generated with Claude Code

mattmattox and others added 18 commits November 10, 2026 12:35
Set up the foundation package structure for the HTTP API server that will
serve the management CLI and web dashboard.

Files created:
- doc.go: Package documentation with endpoint overview
- config.go: Config struct with DefaultConfig(), Validate(), and builder methods
- errors.go: Package-specific error definitions
- types.go: JSON request/response type definitions
- routes.go: Route registration with /health endpoint
- server.go: Server struct with Start/Stop lifecycle
- server_test.go: Comprehensive unit tests (11 tests)

The API server follows existing GoClusterFS patterns:
- Config pattern from internal/cache
- Manager lifecycle pattern from internal/journal
- Dependency injection (no global singletons)

All tests pass (11/11).

Task: #5588
- Add /api/v1/ catch-all handler for versioned endpoints
- Add /metrics endpoint for Prometheus scraping
- Add /api/v1/status endpoint returning server status
- Add /api/v1/stats/* placeholder for component stats
- Add method validation (405 Method Not Allowed)
- Add route tests for all endpoint groups

Task: #5591
- Add LoggingMiddleware that logs method, path, status, duration
- Add AuthMiddleware with Bearer token validation and skip paths
- Add CORSMiddleware with configurable origins and preflight support
- Integrate middleware chain into server (logging -> CORS -> auth)
- Add comprehensive middleware tests (11 new tests)

Task: #5592
- Add HealthStatus constants (ok, degraded, unhealthy)
- Add ComponentHealth for individual health checks
- Enhance StatusResponse with capacity, cluster, mount state
- Add CacheStatsResponse matching internal/cache.Stats
- Add JournalStatsResponse matching internal/journal.JournalStats
- Add IOStatsResponse and StatsOverviewResponse
- Add comprehensive type marshaling tests

Task: #5593
- Add HealthChecker interface for component health reporting
- Add AggregateHealth to combine multiple health check results
- Implement priority: unhealthy > degraded > ok
- Return 503 for unhealthy status, 200 for ok/degraded
- Add RegisterHealthChecker to Server for dependency injection
- Add comprehensive health endpoint tests

Task: #5594
…y info

- Add version.go with build-time ldflags support (Version, GitCommit, BuildTime)
- Add StatusProvider interface for filesystem status injection
- Enhance handleStatus with build info and optional capacity
- Add status endpoint tests with mock provider

Task: #5595
- Add fsStatusProvider to expose filesystem status via API
- Start API server on :8080 after FUSE mount succeeds
- Register health checker and status provider
- Add graceful shutdown in signal handler and exit cleanup

Task: #5596
- Add -api flag to enable/disable API server (default: enabled)
- Add -api-addr for listen address (default: :8080)
- Add -api-token for Bearer token authentication
- Add -api-cors for comma-separated CORS allowed origins
- Wire flags to API server configuration via fluent API

Task: #5597
Add thread-safe accessor method to expose snapshot registry for API
endpoints. Uses RLock since we're only reading the pointer and the
Registry itself manages its own concurrent access with internal mutex.

Task: #5598
Add unified statistics collection and aggregation package:

- stats.go: Core types (Snapshot, CacheStats, JournalStats, etc.)
  and provider interfaces for component decoupling
- collector.go: Collector with TTL-based caching to aggregate
  statistics from multiple providers
- cache.go: Cache state utilities (CacheInfo, CachedSnapshot)
- errors.go: Package-specific error definitions
- doc.go: Package documentation
- stats_test.go: Comprehensive unit tests (16 tests)

The collector uses a pull model where providers are queried on
demand, with configurable cache TTL to avoid expensive repeated
queries. Preset configs available for different use cases:
DefaultConfig (1s TTL), HighFrequencyConfig (100ms), LowOverheadConfig (10s).

Task: #5599
Add comprehensive statistics types for TRIM operations, Sparse File
Bitmap allocation, and cluster node information:

- TrimStats: TRIM/discard operation metrics
- SFBStats: block allocation and fragmentation stats
- ClusterInfo: node role, state, and peer information
- PeerInfo: individual peer node details

Also add provider interfaces and Collector support for all new types.

Task: #5600
…sfb, cluster

- Add StatsProvider interface for statistics collection
- Implement /api/v1/stats endpoints:
  - /stats (overview with cache and journal)
  - /stats/cache (inode, extent, negative cache hit rates)
  - /stats/journal (transactions, space, checkpoints)
  - /stats/trim (TRIM operations and bytes)
  - /stats/lock (lock hint cache statistics)
  - /stats/sfb (block allocation statistics)
  - /stats/cluster (node membership and peers)
  - /stats/all (combined snapshot of all stats)
- Add response types for all stats endpoints
- Add comprehensive tests for stats handlers

Task: #5602, #5603, #5604, #5605, #5606, #5607
CI fails on unformatted files. Whitespace and import order only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t a token

The API listened on :8080 on every interface with no auth, so any host
that could reach the node could read cluster state. -api-addr now
defaults to 127.0.0.1:8080. Mount exits before touching the device when
-api-addr can be reached from other hosts and -api-token is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Broadcast is on by default and its secret is empty by default, so most
clusters accept unauthenticated invalidation and peer messages without
knowing it. Mount now logs a WARNING line before it opens the device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
internal/api kept its own Version, GitCommit and BuildTime, which
scripts/dist.sh never set, so /api/v1/status always said "dev".
GetBuildInfo now reads internal/version, and goclusterfs-mount -version
prints the same string as goclusterfs-fsck -version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
golangci-lint flagged mockFilesystemProvider as unused. Use it in a test
of SetFilesystemProvider instead of deleting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox and others added 2 commits November 10, 2026 12:56
allocateImage opens an existing file, so the refusal test failed before
it ran the mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the API port is taken, mount logged a second warning at unmount
("api: server not started"). Every mount after the first on a host
hits this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit bafd01f into main Nov 10, 2026
13 of 14 checks passed
mattmattox added a commit that referenced this pull request Nov 10, 2026
main's tree equals #48's last head, already merged in the parent commit.
mattmattox added a commit that referenced this pull request Nov 10, 2026
main's tree equals #48's last head, already merged in the parent commit.
mattmattox added a commit that referenced this pull request Nov 10, 2026
Main's -short suite spends about 240s in tests against a 5m limit, so a
slow runner times out (seen on #48 at 330s with no failing test).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox added a commit that referenced this pull request Nov 10, 2026
…k API (#50)

* feat(api): create internal/api package structure

Set up the foundation package structure for the HTTP API server that will
serve the management CLI and web dashboard.

Files created:
- doc.go: Package documentation with endpoint overview
- config.go: Config struct with DefaultConfig(), Validate(), and builder methods
- errors.go: Package-specific error definitions
- types.go: JSON request/response type definitions
- routes.go: Route registration with /health endpoint
- server.go: Server struct with Start/Stop lifecycle
- server_test.go: Comprehensive unit tests (11 tests)

The API server follows existing GoClusterFS patterns:
- Config pattern from internal/cache
- Manager lifecycle pattern from internal/journal
- Dependency injection (no global singletons)

All tests pass (11/11).

Task: #5588

* feat(api): implement route registration with groups

- Add /api/v1/ catch-all handler for versioned endpoints
- Add /metrics endpoint for Prometheus scraping
- Add /api/v1/status endpoint returning server status
- Add /api/v1/stats/* placeholder for component stats
- Add method validation (405 Method Not Allowed)
- Add route tests for all endpoint groups

Task: #5591

* feat(api): implement middleware (auth, CORS, logging)

- Add LoggingMiddleware that logs method, path, status, duration
- Add AuthMiddleware with Bearer token validation and skip paths
- Add CORSMiddleware with configurable origins and preflight support
- Integrate middleware chain into server (logging -> CORS -> auth)
- Add comprehensive middleware tests (11 new tests)

Task: #5592

* feat(api): define comprehensive JSON request/response types

- Add HealthStatus constants (ok, degraded, unhealthy)
- Add ComponentHealth for individual health checks
- Enhance StatusResponse with capacity, cluster, mount state
- Add CacheStatsResponse matching internal/cache.Stats
- Add JournalStatsResponse matching internal/journal.JournalStats
- Add IOStatsResponse and StatsOverviewResponse
- Add comprehensive type marshaling tests

Task: #5593

* feat(api): implement /health endpoint with component health checks

- Add HealthChecker interface for component health reporting
- Add AggregateHealth to combine multiple health check results
- Implement priority: unhealthy > degraded > ok
- Return 503 for unhealthy status, 200 for ok/degraded
- Add RegisterHealthChecker to Server for dependency injection
- Add comprehensive health endpoint tests

Task: #5594

* feat(api): implement /api/v1/status endpoint with version and capacity info

- Add version.go with build-time ldflags support (Version, GitCommit, BuildTime)
- Add StatusProvider interface for filesystem status injection
- Enhance handleStatus with build info and optional capacity
- Add status endpoint tests with mock provider

Task: #5595

* feat(api): integrate API server into mount command

- Add fsStatusProvider to expose filesystem status via API
- Start API server on :8080 after FUSE mount succeeds
- Register health checker and status provider
- Add graceful shutdown in signal handler and exit cleanup

Task: #5596

* feat(api): add CLI flags for API server configuration

- Add -api flag to enable/disable API server (default: enabled)
- Add -api-addr for listen address (default: :8080)
- Add -api-token for Bearer token authentication
- Add -api-cors for comma-separated CORS allowed origins
- Wire flags to API server configuration via fluent API

Task: #5597

* feat(fs): add SnapshotRegistry() accessor for API endpoints

Add thread-safe accessor method to expose snapshot registry for API
endpoints. Uses RLock since we're only reading the pointer and the
Registry itself manages its own concurrent access with internal mutex.

Task: #5598

* feat(stats): create internal/stats package structure

Add unified statistics collection and aggregation package:

- stats.go: Core types (Snapshot, CacheStats, JournalStats, etc.)
  and provider interfaces for component decoupling
- collector.go: Collector with TTL-based caching to aggregate
  statistics from multiple providers
- cache.go: Cache state utilities (CacheInfo, CachedSnapshot)
- errors.go: Package-specific error definitions
- doc.go: Package documentation
- stats_test.go: Comprehensive unit tests (16 tests)

The collector uses a pull model where providers are queried on
demand, with configurable cache TTL to avoid expensive repeated
queries. Preset configs available for different use cases:
DefaultConfig (1s TTL), HighFrequencyConfig (100ms), LowOverheadConfig (10s).

Task: #5599

* feat(stats): add TrimStats, SFBStats, and ClusterInfo types

Add comprehensive statistics types for TRIM operations, Sparse File
Bitmap allocation, and cluster node information:

- TrimStats: TRIM/discard operation metrics
- SFBStats: block allocation and fragmentation stats
- ClusterInfo: node role, state, and peer information
- PeerInfo: individual peer node details

Also add provider interfaces and Collector support for all new types.

Task: #5600

* feat(api): implement stats endpoints for cache, journal, trim, lock, sfb, cluster

- Add StatsProvider interface for statistics collection
- Implement /api/v1/stats endpoints:
  - /stats (overview with cache and journal)
  - /stats/cache (inode, extent, negative cache hit rates)
  - /stats/journal (transactions, space, checkpoints)
  - /stats/trim (TRIM operations and bytes)
  - /stats/lock (lock hint cache statistics)
  - /stats/sfb (block allocation statistics)
  - /stats/cluster (node membership and peers)
  - /stats/all (combined snapshot of all stats)
- Add response types for all stats endpoints
- Add comprehensive tests for stats handlers

Task: #5602, #5603, #5604, #5605, #5606, #5607

* style(api): gofmt the API and stats code

CI fails on unformatted files. Whitespace and import order only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(api): bind to loopback by default and refuse remote access without a token

The API listened on :8080 on every interface with no auth, so any host
that could reach the node could read cluster state. -api-addr now
defaults to 127.0.0.1:8080. Mount exits before touching the device when
-api-addr can be reached from other hosts and -api-token is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mount): warn at startup when broadcast runs without a secret

Broadcast is on by default and its secret is empty by default, so most
clusters accept unauthenticated invalidation and peer messages without
knowing it. Mount now logs a WARNING line before it opens the device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(api): read the build version from internal/version

internal/api kept its own Version, GitCommit and BuildTime, which
scripts/dist.sh never set, so /api/v1/status always said "dev".
GetBuildInfo now reads internal/version, and goclusterfs-mount -version
prints the same string as goclusterfs-fsck -version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document the HTTP API flags, loopback default and new warnings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(stats): cover the filesystem provider

golangci-lint flagged mockFilesystemProvider as unused. Use it in a test
of SetFilesystemProvider instead of deleting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(api): create the image file before sizing it

allocateImage opens an existing file, so the refusal test failed before
it ran the mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mount): skip API shutdown when the API never started

When the API port is taken, mount logged a second warning at unmount
("api: server not started"). Every mount after the first on a host
hits this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(api): close CORS, DNS-rebinding and auth-path gaps

Review of #48 found three ways around the loopback default:
- CORS "*" echoed any Origin, so with no token any web page open on the
  node could read the API. CheckExposure now refuses "*" without a token,
  "*" is sent as a literal, responses carry Vary: Origin, and only GET,
  HEAD and OPTIONS are allowed.
- A token-less loopback server accepted any Host header, so a
  DNS-rebinding page could reach it. It now answers 403 unless Host is
  localhost, 127.x or [::1].
- Auth skipped /health/... and /metrics/... too. Only the exact paths
  skip it now.

doc.go no longer says /health always returns 200 or that only /health
skips auth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(mount): check the API flags before opening the device

Validate ran only after the mount, so a bad -api-cors became a warning
and -api-addr without a port got the misleading "other hosts can reach"
error. Mount now validates first and names the right fix for each
refusal. Adds tests for -version, the broadcast-secret warning and these
early errors. Also uses goclusterfs.BlockSize for status capacity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document the Host check and the CORS rule without a token

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: trigger checks after retargeting to main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(api): format and mount in one process in the loopback health test

MountedFS.Format sends fusermount -u without waiting and then kills the
mount, which can leave a dead FUSE mount behind. On a loaded .68 the
next mount then failed with "failed to access mountpoint: Permission
denied". Passing -format to Mount avoids the second process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant