Skip to content

fix(remediation): apply circuit breaker at startup, fix races, fix reload strategy validation - #49

Open
mattmattox wants to merge 6 commits into
mainfrom
fix/a2-remediation-safety
Open

mattmattox wants to merge 6 commits into
mainfrom
fix/a2-remediation-safety

Conversation

@mattmattox

Copy link
Copy Markdown
Contributor

The remediator registry only received circuit breaker config on hot reload; a fresh process ran the package default (5 / 5m / 2) instead of the configured 5 / 30m / 2, and circuitBreaker.enabled was never read. Startup now goes through NewRegistryFromConfig, sharing one conversion helper with ApplyConfig. enabled is a *bool like reload.enabled: omitted means on, explicit false disables the breaker and reports Closed.

Remediate read the dry-run flag without the lock and recordRateLimitEntry read maxPerHour before taking it; both raced with the setters hot reload calls. Fixed, with -race tests.

The reload validator accepted only restart, recreate, script, webhook, none of which exist, so any config with a real per-monitor strategy failed reload validation. It now uses the strategy list exported from pkg/types; the config error message is built from the same list.

🤖 Generated with Claude Code

mattmattox and others added 6 commits October 7, 2026 09:39
Cast statfs Bsize to int64 so pkg/monitors/system builds on macOS, where
the field is uint32. Make the unit tests portable: short unix socket
paths under the 104-byte limit, squat the wildcard address in the
Prometheus bind-failure test, discover the loopback interface name, and
skip tests that need 127.0.0.0/8 aliases or inotify rename semantics
when the host lacks them.

Remove the unused command-call tracking and testKernelPanicLog from the
logpattern mocks, and revive the memory estimation assertions that were
commented out.

Closes #20

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Delete process docs copied from another project's template
(docs/development/*, docs/deployment.md, .githooks/README.md) and
replace docs/README.md with an index of the docs that exist, grouped
for operators and developers. Repoint the links that targeted the
deleted files and add scripts/check-doc-links.sh to catch broken
relative links.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ftovers

scripts/validate-pipeline-local.sh only had case branches for another
project's components, so it ran zero checks and reported success. It now
runs gofmt, GOOS=linux go vet, golangci-lint (SKIP_LINT=1 to allow it
missing), gosec when installed, go test -short -race, helm lint and
helm-verify-generated, and exits non-zero on any failure.

Makefile: validate-component passes --component=; test-e2e uses
-tags=e2e and checks for kind; COVERAGE_THRESHOLD is 70 and ci.yml
calls make coverage-threshold so there is one source; docker-push and
push-*-image require an explicit VERSION and never tag or push :latest.
Removed the template header, helm-publish stub, deploy-*, bump*,
qa-check, devils-advocate, check-kubeconfig-* and their variables.

Deleted dead template files: init-from-template.sh,
substitute-variables.sh, validate-api.sh, the GPG scripts, .githooks
pre-push and setup.sh, hooks/, .validation.json, .version-rc.

validate-repo-cleanliness.sh no longer rejects the repo's own
Dockerfile.overlay-test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The chart and deploy manifests referenced supporttools/node-doctor-controller,
but nothing built or published it, so enabling the controller always ended in
ImagePullBackOff. Add Dockerfile.controller (static Go binary on scratch,
non-root, writable /data for SQLite), wire it into the Makefile image targets,
build it in ci.yml, and publish, sign and guard it in release.yml alongside
the other two images.

The chart's controller tag now follows image.tag instead of appVersion, which
carries a v prefix that no published image tag has. deploy/controller pins the
current release instead of :latest, and the e2e harness builds and loads the
controller image kind expects.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…load strategy validation

The registry only received the circuit breaker config on hot reload, so a
fresh process ran with the package default (5 / 5m / 2) instead of the
configured values, and circuitBreaker.enabled was read nowhere outside
defaults and validation. NewRegistryFromConfig now wires the registry at
startup through the same conversion ApplyConfig uses, and a disabled
breaker bypasses trip/open checks while reporting Closed. Circuit breaker
defaults apply regardless of enabled so a disabled block no longer leaves
zeros the reload guard rejects.

Remediate read dryRun without the lock and recordRateLimitEntry read
maxPerHour before taking it; both raced with the setters reachable from
hot reload.

The reload validator accepted only restart/recreate/script/webhook, none
of which exist, so every real strategy failed reload validation. It now
uses the exported strategy list from pkg/types, and the config error
message builds from the same list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A plain bool could not tell an absent circuitBreaker block from an
explicit enabled: false, so omitting the block disabled the breaker.
CircuitBreakerConfig.Enabled is now a *bool like ReloadConfig.Enabled:
nil becomes true in ApplyDefaults and only an explicit false disables.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant