Skip to content
fastabcPublic

About

FastConf: A lock-free, strongly typed Go config framework that merges multiple sources (files, env vars, CLI, remote KV) into a single struct and uses atomic. including AWS, Azure and GCP cloud configuration sources through optional modules.Pointer to safely publish new snapshots.

Topics

Resources

Stars

3 stars

Watchers

0 watching

Forks

Latest commit

 

History

67 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

FastConf

English · 中文

FastConf brings files, environment variables and cloud configuration into one typed Go configuration. Load once for a CLI or keep a service current with validated hot reloads, atomic snapshots and a lock-free, zero-allocation Get(). Failed reloads keep the application on its last valid configuration.

Go Reference CI Release

This README documents v1.6.0; see the release notes. Read the documentation at your installed tag for release-specific behavior. For v0.20.0 upgrades, see the migration guide; for v1.5.0, see the v1.6.0 upgrade notes.

Why FastConf

  • One struct, many sources. Combine YAML, JSON, TOML, INI, env, flags and remote providers; use profiles, region/host layers, patches and keyed-list merging for deployment differences.
  • Configuration that is easy to write. In v1.6.0, readable default tags preserve explicit zero values, KV strings become typed lists, and interpolation connects env values and config fields.
  • Controlled runtime changes. Preview with Plan, validate before publication, subscribe to a selected subtree, and restore a retained snapshot with history rollback.
  • Cloud refresh with less application code. In v1.6.0, Manager supervises notification consumers and periodically reconciles sources through WithNotifier and WithResync.
  • Secrets masked in diagnostics. In v1.6.0, chain Vault, age and application resolvers; resolved values are automatically redacted in Map, Dump, Diff and Explain.
  • Documentation from the config type. In v1.6.0, generate JSON Schema, annotated YAML and env examples from the same struct, defaults and field descriptions.
  • Visibility and deployment options. Trace value origins, audit changes, export metrics and traces, or expose snapshots through the CLI/sidecar. Cloud SDKs and telemetry backends are optional modules.

Installation

The root module requires Go 1.25+. Optional modules declare their own minimum Go version; install only the adapters you use.

go get github.com/fastabc/fastconf@v1.6.0

Quick start

Choose the entry point that matches the configuration lifecycle:

Need API Guide
Read exactly one required file at startup LoadFile[T](path) → *T Single file; added in v1.2.0
Load layers once, with options and diagnostics Load[T](ctx, opts...) → *State[T] Layered example
Reload configuration while running New[T](ctx, opts...) → *Manager[T] Dynamic guide

Create conf.d/base/00-app.yaml:

server:
  port: 8080

Save this complete program as main.go in your application module:

package main

import (
    "context"
    "log"

    "github.com/fastabc/fastconf"
    "github.com/fastabc/fastconf/providers/env"
)

type Config struct {
    Server struct {
        Port int `json:"port" fc:"min=1,max=65535"`
    } `json:"server"`
}

func main() {
    state, err := fastconf.Load[Config](context.Background(),
        fastconf.WithDir("conf.d"),
        fastconf.WithProvider(env.New("MYAPP_")),
        fastconf.WithUnknownFields(fastconf.UnknownError),
    )
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("port=%d", state.Value().Server.Port)
}
go run .                          # port=8080
MYAPP_SERVER_PORT=9090 go run .     # port=9090
MYAPP_SERVER_PORT=0 go run .        # validation error

Input formats use json struct tags by default. Load starts no background workers; its value and all reachable maps, slices and pointers must stay read-only. For runtime updates, use New, enable file watching with WithWatch, and close the manager on shutdown. See the complete dynamic example.

Run bundled examples from the repository root:

go run ./examples/basic
go run ./examples/reload

Exactly one required file

LoadFile was added in v1.2.0. With the Config type and YAML above:

cfg, err := fastconf.LoadFile[Config]("conf.d/base/00-app.yaml")
if err != nil {
    return err // Inside a function returning error.
}
log.Printf("port=%d", cfg.Server.Port)

Only that file is read; env and profile discovery do not participate. Unknown keys and missing files fail. The caller owns the returned value, with no manager to close. See the single-file guide.

Features

Capability What you can build Guide
Layered, typed loading Files, env/dotenv, flags, labels, HTTP, Vault, Consul and AWS/Azure/GCP sources behind one config type Sources
Deployment-specific overrides Base/overlay profiles, region/host axes, RFC 6902 patches, keyed lists and generated layers Layers, keyed lists, generators
Readable defaults and conversion (v1.6.0) default:"5s", default:"a,b", named primitives, comma-separated/JSON/indexed-map scalar lists; Azure JSON values are opt-in Field metadata, typed hooks
Env and field interpolation (v1.6.0) ${VAR:default}, required values and ${server.base_url} references with cycle detection Interpolation
Validation before publication Field constraints, Go validators and policies, with optional OPA/CUE adapters Validation, policies
Live updates and recovery File/provider watchers, subtree subscriptions, one-shot overrides, Plan previews, history and rollback Subscriptions, Plan, history
Managed cloud notifications (v1.6.0) SNS/SQS, Service Bus or Pub/Sub consumers with Manager-owned restart, shutdown and periodic reconciliation Cloud notifications
Secret resolution and masking (v1.6.0) Ordered resolvers, Vault KV v2 references, optional age decryption and automatic diagnostic redaction Secrets
Schema and examples (v1.6.0) Draft 2020-12 JSON Schema, annotated YAML and env examples from struct tags Generation guide, runnable example
Operations and deployment Explain origins, masked diffs, observers, metrics, traces, logging, CLI, Kubernetes sidecar and one Manager per tenant Observability, sidecar, tenants
Feature rules Configuration-backed targeting and deterministic percentage rollouts Feature flags

Layer precedence is files → generators → providers → one-shot overrides. Provider priorities only order providers. FastConf borrows base/overlay and patch concepts from Kustomize; it does not read kustomization.yaml or implement the full Kubernetes strategic-merge protocol.

The runtime contract defines lifecycle, precedence, decoding and ownership. The extensions guide lists optional modules, including S3, pflag, validation, logging and telemetry. NATS and Redis Streams are application-adaptable examples, not stable providers.

Benchmarks

Selected medians for FastConf v1.6.0 from the 2026-11-08 comparison: Apple M5 Pro, Darwin/arm64, Go 1.26.10, GOMAXPROCS=4, GOGC=100; six 500 ms runs per case. FastConf uses local source commit d0364e0; the report records source hashes, fixed dependency versions and scope.

Workload FastConf v1.6.0 Viper 1.21.0 Koanf 2.3.0 Cleanenv 1.5.0
4 KiB YAML startup (ms/op) 0.1186 0.1012 0.1291 0.0691
64 KiB valid reload (ms/op) 1.6452 1.5454 1.9950 —
Published struct hot read (ns/op) 0.3900 0.4181 0.4363 —

All three measured hot-read paths allocate 0 B/op and 0 allocs/op. Viper and Koanf use comparison adapters that decode, validate, deduplicate and atomically publish typed snapshots; hot reads do not perform string-key lookup. Sub-nanosecond figures measure tight-loop throughput, not request latency. Results depend on the workload and machine; cloud-service latency is outside this comparison.

See the comparison methodology and runner to reproduce the measurements on an idle machine, and the performance contract for the Get allocation guard, reload benchmarks and profiling commands.

Loading and reload pipeline

Load returns a snapshot without workers. New loads synchronously, starts the writer and configured watchers, and provides Get, Snapshot, Reload, Plan, Errors and history. The constructor context bounds initialization; close the manager to stop background work.

The following blocks are integration fragments using application types and values.

mgr, err := fastconf.New[Config](ctx,
    fastconf.WithDir("conf.d"),
    fastconf.WithUnknownFields(fastconf.UnknownError),
    fastconf.WithWatch(fastconf.Watch{}), // Enable file watching.
    fastconf.WithHistory(4),             // Retain up to four prior snapshots.
)
if err != nil {
    return err // Inside a function returning error.
}
defer mgr.Close()
cfg := mgr.Get() // Read once per business operation; keep the entire tree read-only.
_ = cfg
// Keep the manager alive until the service owner shuts down.
assemble -> merge -> transform -> secret -> defaults -> typed-hooks -> decode
         -> field-meta -> validate -> policy -> hash -> publish

A rejected candidate preserves the live snapshot and generation. Equal typed hashes skip publication unless resolved-secret paths change; those metadata commits update diagnostics/history without invoking value subscribers. Errors() is bounded and evicts the oldest pending error on overflow; consume it for ongoing failures. WithUnknownFields controls unknown struct keys; the default UnknownWarn needs WithLogger to be visible. WithStrictMerge controls raw merge type conflicts and file discovery instead.

Files, profiles and merge rules

base/* loads first, followed by matched overlays/*, then WithAxes directories such as region or host. Files and matching profile directories use lexical order; the order of names in Profile.Names does not set overlay precedence. Non-empty axis values (including hostname fallbacks) must be single directory names; . / .. and path separators are rejected with an axis warning.

fastconf.WithProfile(fastconf.Profile{Names: []string{"prod"}})
fastconf.WithAxes(fastconf.Axis{Dir: "regions", Env: "DEPLOY_REGION"})
fastconf.WithMergeKeys(map[string]string{"listeners": "name"})

Maps merge recursively; scalars and lists replace by default. mergeKeys aligns list objects by a non-empty string key. .patch.json / .patch.yaml files apply RFC 6902 operations in layer order. Root _meta.yaml uses spec for defaults and merge options; overlay _meta.yaml uses top-level match expressions. See files, profiles and patches.

Providers and generators

Provider / entry point Watch changes Notifier Features
source.NewFile Shared file watcher — Decoded document; missing file contributes an empty layer
source.NewBytes — — Copied in-memory document with an explicit codec
source.NewFS — — Required document from embedded or virtual fs.FS; added in v1.2.0
env.New — — Process environment; underscores become path separators
dotenv.New Shared file watcher — Dotenv fallback; process environment wins
cliflag.New — — Explicitly changed flags; optional pflag adapter
labels.New — — Dotted configuration labels with configurable mapping
k8s.NewDefault Shared file watcher — Raw namespaced Downward API metadata
providers/http Native polling — Remote document, ETag/content hash
providers/vault Native metadata polling — KV v2, optional authentication maintenance; field resolver in v1.6.0
providers/consul Native blocking query — KV tree or document
appconfig Via notifier + periodic reload SNS → SQS Deployed document, session tokens, service-directed poll intervals
s3 Via notifier + periodic reload SNS → SQS ETag conditional reads, optional fixed VersionID
parameterstore Via notifier + periodic reload EventBridge → SNS → SQS Recursive paginated tree, string values, optional SecureString decryption
azappconfig Via notifier + periodic reload Event Grid → Service Bus One prefix/label, colon-separated keys; WithJSONValues() decodes JSON settings in v1.6.0
azblob Via notifier + periodic reload Event Grid → Service Bus Blob document, ETag conditional reads
secretmanager Via notifier + periodic reload Pub/Sub Version/alias document, CRC32C verification
gcs Via notifier + periodic reload Pub/Sub Current object document, generation cache, generation-consistent reads

“—” means no automatic change detection. Shared file watching requires WithWatch; native provider watchers start with New independently of it. Cloud providers refresh when Reload calls Load. In v1.6.0, register a consumer with WithNotifier and schedule reconciliation with WithResync; Manager owns their workers. AppConfig reloads also respect the service's next allowed poll time.

Notifier module Acknowledgment and failure behavior
integrations/notify/sns SNS envelopes or raw messages through SQS; delete after successful refresh; failed refresh remains available for redelivery
integrations/notify/azservicebus Event Grid schema through Service Bus PeekLock; Complete after successful refresh, Abandon on failure
integrations/notify/pubsub GCS / Secret Manager events; Ack after successful refresh, Nack on failure

All three run a synchronous refresh callback with a bounded context. With v1.6.0, Manager restarts a stopped consumer with backoff and jitter, stops it on shutdown, and periodically reloads to recover missed notifications:

mgr, err := fastconf.New[Config](ctx,
    fastconf.WithProvider(remote),
    fastconf.WithNotifier(consumer),
    fastconf.WithResync(time.Minute),
)
if err != nil {
    return err
}
defer mgr.Close()

Here remote and consumer are application-created provider and consumer values; import time for the interval. Load starts no workers and ignores these options. The application owns SDK clients and provisions a separate queue/subscription for each independent replica that must receive every change. See cloud notifications for routing, retries, dead letters and periodic reconciliation. s3events is deprecated in v1.3.0; use integrations/notify/sns. Its existing Watch uses best-effort EventBridge → SQS notification and acknowledges before Manager reload completes; use the new consumer for refresh-before-ack behavior.

See embedded files and cloud sources for loading and limits. Cloud SDKs live in independent modules. Except for the S3 loader, the cloud loaders and all three notifier modules above were added in v1.2.0.

Providers implement contracts.Provider (Name, Load). Optional Describe supplies priority/file paths; optional contracts.Watcher supplies native events with Watch(ctx, from). Provider watchers start with New independently of WithWatch. Configure providers before registration; fluent methods mutate them. Within the provider class, higher priority wins and registration order breaks ties.

WithGenerator accepts contracts.Generator implementations producing encoded RawLayer values. Generators run during assembly before providers and have no built-in watch channel. Transforms registered with WithTransform then rewrite the merged tree, including schema migrations through transform.NewChain. See generators and extensions.

Types, validation and secrets

Document format is independent of struct decoding: the default JSON bridge uses json tags; WithDecodeFormat(fastconf.YAML) selects YAML tags. Default typed hooks convert duration strings and primitive string values from env/INI. In v1.6.0, KV strings also become scalar lists and named numeric/boolean types: "a, b" or '["a","b"]' can fill []string, and "3" can fill type Level int. Contiguous index maps such as {"0":"a","1":"b"} also become lists. []byte retains base64 semantics and custom unmarshalers retain their wire forms.

The v1.6.0 type fragment below imports time.

type Database struct {
    DSN     string        `json:"dsn" fc:"required,secret" desc:"Database connection"`
    Pool    int           `json:"pool" default:"10" fc:"min=1,max=100"`
    Timeout time.Duration `json:"timeout" default:"5s"`
    Hosts   []string      `json:"hosts" default:"primary,replica"`
    Enabled bool          `json:"enabled" default:"true"`
}

In v1.6.0, default:"…" fills absent keys before typed decoding: 5s becomes a time.Duration, primary,replica becomes a list, and explicit false, 0, empty strings are retained, while null counts as present. Explain records injected defaults as LayerDefault. Legacy fc:"default=…" still fills decoded Go zero values; Defaulter.Defaults provides computed defaults. Field constraints run before WithValidate and WithPolicy; policy warnings allow publication and error-severity findings reject it. Tags do not validate collection elements; use a validator for those checks. For pointers, required checks for nil; min, max and oneof check the pointed-to value and skip nil pointers.

For interpolation in v1.6.0, add this option to Load or New (import github.com/fastabc/fastconf/transform):

fastconf.WithTransform(transform.Interpolate(transform.InterpolateOptions{Refs: true}))
server:
  base_url: "https://${HOST:example.com}"
oauth:
  callback_url: "${server.base_url}/oauth/callback"
dsn: "postgres://${DB_USER:app}@${DB_HOST:localhost}/app"

${VAR:default} supplies an env fallback; ${VAR:?message} requires a non-empty value. Field references resolve recursively with cycle/depth checks. Env values are not expanded twice. Put defaults needed by references in a base layer; literal default tags are injected after interpolation and secret resolution.

In v1.6.0, WithSecretResolvers chains resolvers in registration order; the first recognizer handles a reference. PrefixSecretResolver wraps an application function for prefixes such as kms://. Use vault.NewResolver for KV v2 field references or the optional secrets/age adapter for age decryption. Resolved values are automatically masked in diagnostic views. fc:"secret" and WithSecretPaths also mask sensitive inputs that do not use a resolver. Get(), Value() and explicit Unredacted() views retain plaintext access. See field metadata, typed hooks, interpolation, policies and secrets.

Generate configuration documentation (v1.6.0)

Use schema.JSONSchema, schema.ExampleYAML and schema.ExampleEnv to generate a Draft 2020-12 schema, annotated YAML and env examples from a config struct. desc, default and fc metadata supply descriptions, defaults, required fields, constraints and secret markers. Run generation through go generate so application config types and checked-in examples share one source. The reverse direction, YAML to Go structs, remains available through fastconfgen. See the generation guide and runnable schema example.

Subscriptions, preview and recovery

cancel := fastconf.Subscribe(mgr,
    func(c *Config) *int { return &c.Server.Port },
    func(old, next *int) { log.Printf("port: %d -> %d", *old, *next) },
)
defer cancel()

if err := mgr.Reload(ctx, fastconf.WithReason("admin"),
    fastconf.WithOverride(map[string]any{"server": map[string]any{"port": 9090}}),
); err != nil {
    return err
}

Subscriptions compare a selected scalar or entire subtree after publication. For example, subscribe to Database and Log separately so a log-level change does not reconnect the database; WithEqual can narrow equality to the DSN. Callbacks run on the writer, must return quickly and must not wait on that manager's reload/Plan/rollback/shutdown. Signal a bounded worker for costly resource updates. An override applies to one reload and does not edit sources.

Plan(ctx) returns Proposed, Diff, Validators and Policies without publishing. Call result.Err() to check validator failures and policy denials even when Plan returns nil error. Reload reads inputs again. Pause suppresses watcher triggers; Resume does not replay ignored events, so reload explicitly after a paused batch. In v1.6.0, pause also skips periodic resync and returns ErrPaused to notifier refreshes; consumers must retain or negatively acknowledge those messages.

Snapshot().Map/Dump/Diff/Explain provide masked diagnostics; enable WithProvenance for origins. Unredacted().Map/Dump exposes plaintext explicitly. History().List() returns prior snapshots oldest-first; rollback republishes a retained snapshot without validation or source repair. History is nil when disabled. See subscriptions, Plan and diagnostics/history.

Observability and application features

WithObserver receives reload, stage, provider-error, drop and commit events. observe.Multi, JSONLines, Metrics and Func compose audit and telemetry. observe.Async adds bounded asynchronous delivery; its owner closes it after the manager and supplies deadlines for external I/O. Log through WithLogger and a standard slog.Handler, or an optional backend adapter.

Use one manager per tenant, selecting sources and credentials explicitly; WithTenant only labels causes and policy input. Feature rules are ordinary configuration fields evaluated with feature.Eval: targets precede deterministic percentage rollouts, then the rule default. The evaluator is not an OpenFeature SDK provider. See observability, tenants and feature flags.

CLI tools

Command Purpose
fastconfctl Offline dump, diff, validate and explain
fastconfd HTTP configuration snapshots, manual reload and SSE notifications; sidecar recipe
fastconfgen Generate Go structs from YAML; separate module

Install from source at a matching version, or use binaries from GitHub Releases:

go install github.com/fastabc/fastconf/cmd/fastconfctl@v1.6.0
go install github.com/fastabc/fastconf/cmd/fastconfd@v1.6.0
go install github.com/fastabc/fastconf/cmd/fastconfgen@v1.6.0
fastconfctl dump -dir conf.d -profile prod -format yaml
fastconfctl diff -dir conf.d -from dev -to prod -json
fastconfctl validate -dir conf.d -profile prod
fastconfctl explain -dir conf.d -profile prod server.port
fastconfgen -in conf.d/base/00-app.yaml -pkg config -type Config

fastconfctl validate loads a generic map; it does not validate your application's struct tags or business policies. Starting in v1.2.0, CLI diagnostic output masks common credential keys by default. For application structs, use fc:"secret" or WithSecretPaths for sensitive inputs; v1.6.0 also masks resolver-produced values automatically. See diagnostics.

In v1.3.0, an empty diff -from selects base only, ignoring APP_PROFILE and metadata defaults. fastconfd exits if its listen address is busy.

conf.d discovery accepts regular files (following symlinks), up to 4 MiB each, including metadata; see discovery limits (v1.3.0).

Development and documentation

make build       # root module and fastconfgen
make test        # root race tests
make example     # executable examples, including optional integrations
make check       # repository checks; see tools/README.md

License

MIT · Copyright (c) 2026 FastAbc

About

FastConf: A lock-free, strongly typed Go config framework that merges multiple sources (files, env vars, CLI, remote KV) into a single struct and uses atomic. including AWS, Azure and GCP cloud configuration sources through optional modules.Pointer to safely publish new snapshots.

Topics

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages