Repository navigation
Description
Follow-up to #68, which fixed the lint list and the SNMP asymmetry. The same
claim in scripts/validate.sh:3 — "Everything CI runs, runnable locally" — is
still not true, for five checks that live as inline shell inside ci.yml and
have no local equivalent at all.
Line numbers are as of #174.
CI-only, no local equivalent
ci.yml:92— Verify image versions are not duplicated outside compose.yamlci.yml:106— Verify no image uses a floating tagci.yml:120— Verify every image is pinned by digestci.yml:252— Parse every dashboard PromQL expressionci.yml:311— Assert SOPS files are actually encrypted
Confirmed absent from validate.sh by grepping for each one's distinguishing
token (not duplicated outside, floating tag, pinned by digest,
emit-promql, sops:) — all five miss.
The last one is the sharpest. A SOPS file that is not actually encrypted is a
committed plaintext secret, and the only thing that checks is a job you cannot
run before pushing.
The same check written twice, already diverging
"No decrypted artefact is tracked" exists in both places, as two independent
implementations, and they do not agree:
ci.yml:300 loops five patterns through git ls-files | grep -E "(^|/)…".
validate.sh:354 hand-writes five conditions instead:
if git ls-files --error-unmatch "${STACK}/.env" >/dev/null 2>&1 \
|| git ls-files "${STACK}/snmp-exporter/.rendered" | grep -q . \
|| git ls-files | grep -q '\.purge-secrets\.txt' \
...So they cover different things:
| tracked file | CI | make validate |
|---|---|---|
stacks/observability/.env |
caught | caught |
stacks/other-stack/.env |
caught | missed — only ${STACK}/.env |
some/other/.rendered/x |
caught | missed — only ${STACK}/snmp-exporter/.rendered |
nested/certificates/key.pem |
caught | missed — anchored ^certificates/ |
Every miss is a decrypted secret that passes locally and fails on push, which is
the better direction to fail in — but it is still two copies drifting, and the
second stack this repo grows makes the first row of misses real.
Why this was split out of #68
#68's fix was a shared tool list. These are inline CI shell, so the fix is
different in kind: extract each into a script both callers run, the way
seed-validation-env.sh, snmp-targets.sh --check and now scripts/lint.sh
already are. Doing it in #174 would have tripled that diff and mixed two
refactors.
Suggested shape
- The three image greps arguably belong inside
scripts/check_image_pins.py,
which already runs in both places and already owns this argument — its
docstring explains exactly why a grep cannot see a missing pin (make backup uses an unpinned alpine image that evades both CI pinning checks #65). Folding
them in retires three inline greps rather than relocating them. - The dashboard PromQL parse needs promtool, so it wants the same
binary-or-docker fallbackvalidate.shalready has for promtool/amtool/alloy,
and a counted skip when neither is available. - The SOPS-encrypted assert and the tracked-artefact check are pure git/grep
with no dependencies — one script each, callable from both, deleting the
second copy rather than fixing it.
Anything that can skip locally should take the --skips-file contract
scripts/lint.sh and scripts/check_loki_rules.sh now use, so SKIPPED stays
honest. #174 showed why that matters: with docker hidden, main printed five
SKIP lines and reported "4 skipped".
Activity
Merge pull request #317 from Gerrrt/gerrrt/validate-parity-175