[Gerrrt]

What changed

scripts/check_mounted_config.py checks that each running container has the same single-file config the repo has. make up runs it with --fix. It could never fail. This PR makes it read what the container actually sees.

• Why it could never fail. It read the container's copy with docker cp. For a bind mount, docker cp re-resolves the mount's source path on the host, so it returns the file git just wrote. It does not return the inode the container is pinned to. Every comparison was the new file against itself.

  Measured on a scratch container after a git-style rename:

  Reader	Returned
  docker exec cat	old
  docker cp	new

• The new reader opens /proc/1/root/<path> from a throwaway helper that shares the container's PID namespace. That resolves the path in the container's own mount namespace. It covers distroless loki, where exec cat is impossible.

  • Helper capabilities: SYS_PTRACE for the cross-uid /proc check, and DAC_READ_SEARCH for 0640 keys. No network.
  • Helper image: the pinned Alloy image, via image-for.sh.

• --self-test, discovered by self-tests.sh, reproduces the rename on a non-root, capability-free container. It passes with the new reader. With the old one it fails the three cases that involve a rename and passes the other two.

What it found

• Alertmanager, on the monitoring host, had been running its config from before feat(sensitive): alerts on the phone as a title and a line (#136) #716, feat(alerting): the ntfy.sh copies rendered like the in-house ones (#136) #719 and feat(observability): a warning tier for ACME leaves, so the hours pair matches the days one (#426) #761 since 2026-09-28. The only functional loss was feat(observability): a warning tier for ACME leaves, so the hours pair matches the days one (#426) #761's inhibit of TlsAcmeRenewalLate under TlsAcmeRenewalStalled, so a stalled ACME renewal would have paged twice. It is recreated and verified with the new reader.
• trinity's sensitive stack: all 12 single-file mounts match.
• oracle's wiki has none.
• Lab and SOC hosts: not reachable from here, so unchecked.

How it first showed up

On 2026-09-30, during the #182 rollout, the check reported blackbox.yaml as matching. The exporter was actually running without the module #182 had just added.

Verification

• --self-test passes. A mutation run with the old docker cp reader fails it.
• Ran against live observability and live sensitive (read-only).
• validate.sh exit 0.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation directly addresses the faulty read path and includes focused regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Fixes mounted-config verification by reading files through the container’s mount namespace instead of docker cp.

Changes:

• Adds a capability-limited Alloy helper for accurate reads.
• Adds a Docker-based regression self-test.
• Documents the failure and operational findings.

File	Description
scripts/​check_mounted_config.py	Implements namespace-aware reads and self-testing.
docs/​changelog.md	Records the incident, fix, and deployment findings.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.