Skip to content

feat(deploy,services): implement rbac policies, twin network policy, and loadgen (A1.7-A1.8) - #6

Merged
Harikeshav-R merged 24 commits into
feat/prod-and-system-manifestsfrom
feat/policies-and-loadgen
Sep 11, 2026
Merged

Harikeshav-R merged 24 commits into
feat/prod-and-system-manifestsfrom
feat/policies-and-loadgen

Conversation

@Harikeshav-R

Copy link
Copy Markdown
Owner

Build-Plan Step Implemented

  • Step: Phase 1A — Steps A1.7 (deploy/policies/: ServiceAccounts, Roles, twin NetworkPolicy template) and A1.8 (services/loadgen/ seeded deterministic generator), completing Phase 1A.

Checkpoint Execution & Output

Executed all commands for Checkpoint A1 on live local cluster:

  1. Pod readiness:
$ kubectl -n ust-prod get pods
NAME                            READY   STATUS    RESTARTS   AGE
auth-service-697c5969f6-mkxhg   1/1     Running   0          111m
data-service-5568c58798-t88fp   1/1     Running   0          111m
edge-gateway-76b6bc47c-pvgcq    1/1     Running   0          124m
prod-postgres-9f5469785-qvp8b   1/1     Running   0          111m
worker-5bf6db4677-pwhbb         1/1     Running   0          111m
  1. Item query and Prometheus scrape count:
$ curl -s -H "Authorization: Bearer valid-token" localhost:8080/api/items | jq .items | length
2

$ curl -s "localhost:9090/api/v1/query?query=up" | jq .data.result | length
10
  1. Load generator baseline run:
$ make loadgen RPS=20 DURATION=30
Summary: total=600, success=598, failed=2, error rate 0.00, p50=11.4ms, p90=14.3ms, p95=18.6ms, p99=52.9ms, max=204.4ms [p99 < 400ms (52.9ms)]
  1. Latency fault injection and TTL recovery:
$ curl -sX POST localhost:8081/admin/fault -H "Content-Type: application/json" -d "{\"kind\":\"latency\",\"magnitude\":450,\"ttl_seconds\":15}"
{"status":"applied","fault":{"kind":"latency","magnitude":450.0,"ttl_seconds":15.0}}

$ uv run python -m services.loadgen --rps 20 --duration 5 --seed 42 --target-url http://localhost:8080
Summary: total=100, success=100, failed=0, error rate 0.00, p50=462.3ms, p90=468.2ms, p95=473.5ms, p99=483.5ms, max=515.5ms [p99 >= 400ms (483.5ms)]

# After 15s TTL expiration:
$ uv run python -m services.loadgen --rps 20 --duration 5 --seed 42 --target-url http://localhost:8080
Summary: total=100, success=100, failed=0, error rate 0.00, p50=12.9ms, p90=25.9ms, p95=100.7ms, p99=271.7ms, max=307.6ms [p99 < 400ms (271.7ms)]

$ curl -s localhost:8081/admin/fault
{"active":null,"expires_at":null}
  1. Memory budget assertion:
$ docker stats --no-stream --format "table {{.Name}}\t{{.MemUsage}}\t{{.MemPerc}}"
NAME               MEM USAGE / LIMIT     MEM %
k3d-ust-tools      5.32MiB / 12.16GiB    0.04%
k3d-ust-serverlb   21.92MiB / 12.16GiB   0.18%
k3d-ust-server-0   1.196GiB / 12.16GiB   9.84%
ust-registry       30.75MiB / 12.16GiB   0.25%

Total container memory ~1.25 GB, well within the 3.5 GB budget.

ADRs Relied On

  • ADR-002: Production is a namespace, and the boundary is enforced via scoped RBAC (understudy-prod) and NetworkPolicy.
  • ADR-003: 18 GB RAM budget (total stack memory ~1.25 GB, well below 3.5 GB ceiling).
  • ADR-014: Twin egress containment via default-deny NetworkPolicy whitelisting only intra-namespace pods, DNS in kube-system, and backing services in ust-system.
  • ADR-015: Deterministic load generation via seeded pseudo-random engine (services/loadgen).
  • ADR-028: Actuation scoped to deployments and configmaps in ust-prod.

Mock Registry Disclosures

  • None

Documentation Updates

  • None required (manifest layouts, architecture §2.1 and §2.10, and ADR statements remain authoritative).

Definition of Done

  • Implementation complete for the named build-plan step
  • Unit tests written; 100% line and branch coverage of the added lines
  • make check green (ruff, mypy --strict, import-linter, pytest)
  • Checkpoint command run and output matches, OR marked unverified with a reason
  • Docs updated where this change made them untrue (or "none required", with why)
  • No new # MOCKED: marker without a row in docs/MOCKS.md
  • Conventional commit(s), conventional branch, PR opened, never pushed to main

Enforce the production namespace boundary and twin egress containment
per ADR-002, ADR-014, and safety invariant K6.

- deploy/policies/understudy-prod.yaml defines ServiceAccount, Role, and
  RoleBinding strictly limited to Deployments and ConfigMaps in ust-prod.
- deploy/policies/understudy-twin.yaml defines ClusterRole understudy-twin
  with full permissions to be bound within dynamic twin namespaces.
- deploy/policies/twin-network-policy.yaml and its template specify
  default-deny egress for twin pods while allowing intra-namespace,
  kube-system DNS, and ust-system backing infrastructure.
- Unit and integration tests validate manifest structure, RBAC scoping,
  and kubectl dry-run acceptance.

Implements build-plan step A1.7.
Provide reproducible evaluation traffic against the demo stack per
ADR-015 and verify all Checkpoint A1 assertions.

- services/loadgen/ implements SeededRequestGenerator using random.Random
  to output a deterministic 80/20 mix of GET/POST /api/items requests.
- Async runner paces dispatches, measures response latencies via high-
  resolution timer, computes percentiles, and outputs the summary line.
- Makefile adds loadgen target parameterized by RPS, DURATION, SEED, and
  TARGET_URL, and applies RBAC policies in deploy-prod.
- Unit tests achieve 100% line and branch coverage across services/loadgen.
- Verified Checkpoint A1 assertions on live k3d cluster and updated
  CHECKLIST.md.

Implements build-plan step A1.8.
@Harikeshav-R
Harikeshav-R added this pull request to stack #7 September 10, 2026 23:25

@rsanghvi21 rsanghvi21 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — Candidate Findings (PR #6)

Correctness

  1. Auth bypass — services/auth_service/app.py:87 accepts any Bearer valid-* / Bearer token-* token as valid, regardless of actual validity.
  2. Auth cache eviction — TOKEN_CACHE's FIFO eviction (app.py:90) can downgrade the seeded admin token's scope once evicted and re-validated through the same bypass branch.
  3. Dead RBAC grant — no deploy/prod/*.yaml Deployment sets serviceAccountName: understudy-prod, so the Role/RoleBinding in understudy-prod.yaml never actually applies to any running pod.
  4. Worker fault-injection gap — worker_loop never calls pre_request_hook, so LATENCY/ERROR_RATE/CPU_SPIN fault injection is a no-op against job processing (POOL_EXHAUSTION is a partial exception).
  5. Race condition in faults.py — apply_fault/clear_fault mutate shared state without synchronization across await points.
  6. Magnitude ambiguity at exactly 1.0 in faults.py — behavior is intentional but undocumented.
  7. Dead deploy/prod/configmap.yaml — not consumed by any Deployment.
  8. Shared Prometheus registry across services — a latent test-pollution mechanism (not yet observed failing, but real).

Quality / cleanup
9. Unjustified # type: ignore (x2) in edge_gateway/app.py; bare except Exception in loadgen/runner.py swallows real bugs into metrics.
10. Magic numbers / Settings bypass across all new services — undocumented in docs/02-architecture.md per AGENTS.md §5.10.
11. Duplication across Python services — readiness-check logic, 4-line bootstrap, and db-pool lifespan repeated in auth/data/worker app.py.
12. K8s manifest duplication — 3 near-identical postgres manifests, 4 prod service manifests; twin-network-policy.yaml has drifted from template.yaml (missing the namespace field the template has).

deploy/policies/understudy-prod.yaml grants a ServiceAccount access to
patch Deployments/ConfigMaps in ust-prod, but no Deployment in
deploy/prod/*.yaml sets serviceAccountName: understudy-prod. A
code-review pass against PR #6 flagged this as a dead grant.

It is not dead: per ADR-028 (docs/01-decisions.md), the actuator
authenticates to the Kubernetes API directly as this ServiceAccount
from the agent host process (docs/02-architecture.md §2.10), not by
being assumed as pod identity. This RBAC ships ahead of its consumer
(build-plan step A1.7 landed; actuator/apply.py is step 5.1/5.2,
still unbuilt per CHECKLIST.md). Document that explicitly so it is
not re-flagged as dead code before Phase 5 lands.

Implements build-plan step A1.7 (documentation follow-up).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
deploy/policies/twin-network-policy.yaml (the golden copy, applied
with `kubectl apply -n <namespace>`) and
twin-network-policy.template.yaml (rendered at fork time, carries
`namespace: ${TWIN_NAMESPACE}`) are maintained entirely by hand -- no
generation script exists yet (fleet manifest rendering is build-plan
step A2.2, unbuilt). The two existing unit tests validate each file's
own structure independently and never diff them against each other,
so a future edit to one that isn't mirrored in the other would pass
CI silently.

Add a test that loads both, strips the namespace field they're
expected to differ on, and asserts everything else is identical.

Also corrects docs/02-architecture.md's directory tree, which
documented a deploy/twin/ directory for "twin manifest templates"
that doesn't exist -- the templates actually live in deploy/policies/
next to their golden copies.

Implements build-plan step A1.7 (test coverage follow-up).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
edge_gateway/app.py's get_items and create_item both silenced
mypy's [no-any-return] on `return data_resp.json()` with a
type: ignore, even though both functions are declared
-> dict[str, Any]. Assign to an explicitly-typed local first instead
-- mypy treats the annotation as authoritative, so the return
type-checks with no ignore needed.

loadgen/runner.py's _execute_single_request caught bare
`except Exception` around a call that can only raise
httpx.RequestError subclasses. A bare catch there would silently
record a real bug (e.g. a TypeError from a malformed RequestSpec) as
a fake "failed request" metric instead of failing the load-gen run
loudly. Narrow to `except httpx.RequestError`.

Implements build-plan step A1.1/A1.8 (fix on already-completed
services).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
rsanghvi21 and others added 3 commits September 11, 2026 00:00
worker_loop only called fault_manager.wait_if_stalled, so LATENCY,
ERROR_RATE, and CPU_SPIN faults applied via POST /admin/fault were
no-ops against job processing -- only STALL worked. Every other
service's HTTP middleware calls pre_request_hook once per request;
the worker has no HTTP request in its polling loop, so call it
directly once per iteration instead.

Factor the loop body into _worker_iteration so it can be exercised
directly under test without driving a real polling loop, and so
pre_request_hook's ERROR_RATE branch (which raises HTTPException --
meaningful in an HTTP handler, not here) can be handled the same way
a real job-processing failure already is: logged and the job skipped
for that iteration, rather than crashing the polling task.

Implements build-plan step A1.2 (fix on an already-completed
service).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
FaultManager.apply_fault/clear_fault mutate shared state
(active_fault, expires_at, _timer_task, _leaked_buffers,
_held_connections) across multiple await points with no
synchronization. Two admin apply_fault/clear_fault calls racing each
other, or the independent _ttl_watcher background task clearing a
fault out from under a fresh apply_fault, could leave the manager in
a torn state -- e.g. a TTL expiry cancelling the wrong timer, or a
connection appended to _held_connections during clear_fault's
iteration over the same list.

Add an asyncio.Lock guarding the whole state transition. apply_fault
internally calls clear_fault's logic, so factor that logic into
_clear_fault_locked (no lock acquisition) called by both the public
clear_fault (which acquires the lock) and apply_fault (which
acquires the lock once and calls the internal version), avoiding a
self-deadlock on the non-reentrant lock. Route _ttl_watcher's clear
through the same locked path so it can't race a fresh apply_fault.

Implements build-plan step A1.3 (fix on an already-completed
service).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
…ention

FaultRequest.magnitude means something different per FaultKind, and
for ERROR_RATE specifically it silently overloads two conventions:
<= 1.0 is a fraction, > 1.0 is a percentage. A caller who means "1%"
and passes 1.0 gets 100% instead, since 1.0 lands on the fraction
side of that boundary. This is intentional, existing behavior
(test_faults.py::test_fault_error_rate already asserts 1.0 and 100.0
both mean 100%) -- not changed here, only documented: an inline
comment at the branch, and an expanded field docstring spelling out
the per-kind units and the footgun.

Implements build-plan step A1.3 (documentation follow-up on an
already-completed service).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
REQUEST_COUNT/REQUEST_DURATION/SERVICE_INFO were module-level globals
registered against the implicit default prometheus_client registry,
constructed once at import time and shared by every service's
setup_metrics call. Since all four services' app.py modules (plus
every service test file that imports one) run in the same pytest
process, this meant whichever service imported last determined what
understudy_service_info reported for the rest of the run, and
request counters/histograms accumulated across unrelated services'
tests. Currently dormant (no Duplicated timeseries crash, since the
objects were constructed once total, not once per call) but real
test pollution.

setup_metrics now builds its own CollectorRegistry and
Counter/Histogram/Info instances per call, defaulting to a fresh
registry when the caller doesn't supply one -- every service's
app.py needs no changes, since none of them pass a registry. The
/metrics route scrapes that call's own registry via closure, the
same way the middleware already closes over service_name.

Implements build-plan step A1.1 (fix on an already-completed
service).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
…tory

All four services/*/app.py files repeated an identical 4-line
bootstrap (FastAPI(...), setup_metrics, setup_fault_middleware,
setup_fault_routes), and auth_service, data_service, and worker
repeated a near-identical open-pool/assign-to-fault-manager/optional
-init/close skeleton around their connection pool lifecycle.

Add services/_common/bootstrap.py:
- create_service_app(title, fault_manager, lifespan) does the shared
  4-line bootstrap, returning the constructed app.
- db_pool_lifespan(database_url, fault_manager, on_open=None) is an
  async context manager doing the shared pool lifecycle, yielding the
  pool (or None) so each service assigns it to its own module-level
  db_pool global -- the helper has no way to reach into another
  module's globals itself. on_open lets worker/data_service pass
  their init_worker_db/init_db callbacks; a PsycopgError from it is
  logged and swallowed, matching prior behavior.

Refactor all four app.py files to use create_service_app; refactor
auth_service, data_service, and worker to use db_pool_lifespan.
edge_gateway has no database and keeps its own lifespan (it manages
an httpx.AsyncClient instead). Route registration and readiness-check
logic stay in each service's own file.

Existing tests that patched create_pool on the service module now
patch it on services._common.bootstrap, where the call actually
happens.

Implements build-plan step A1.1/A1.2/A1.3/A1.8 (refactor across
already-completed services).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
check_mocks.py only scanned src/ for # MOCKED: markers, so a marker
placed anywhere under services/ was invisible to the make check
mock-parity gate required by AGENTS.md §7.3. Add a second scan of
services/, re-keying its markers with the services/ prefix so they
match the dotted module names used in docs/MOCKS.md (e.g.
services.auth_service.app).

Also fixes a latent Windows bug uncovered while adding this: rel_path
was built with str() instead of as_posix(), so backslash-separated
paths never matched the forward-slash dotted module names produced by
_module_matches_file. This was invisible until now because
docs/MOCKS.md has been empty, but would have silently broken parity
checking for any nested src/ marker on Windows.

Discovered while investigating a code-review finding for PR #6; this
fix unblocks registering the auth_service synthetic-token mock in a
follow-up PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
services/auth_service/app.py accepted any Bearer valid-*/token-*
token as valid on the always-reachable /validate path, with no real
credential store behind it -- an unmarked, unregistered mock
(AGENTS.md section 2.3, section 7). Compounding this, TOKEN_CACHE's
FIFO eviction could evict the seeded admin token
("Bearer valid-token", scope: admin) under sustained synthetic
traffic; revalidating it afterward fell through to the
synthetic-pattern branch (it also matches "Bearer valid-"), silently
re-caching it as scope: standard with no error.

Fix both together:
- Carve the two seeded tokens into SEEDED_TOKENS, a separate,
  non-evictable dict checked before TOKEN_CACHE and before the
  synthetic-pattern branch. A seeded token can no longer be evicted
  or re-derived with a different scope.
- Gate the synthetic-pattern branch behind
  AUTH_SERVICE_ALLOW_SYNTHETIC_TOKENS, defaulting to enabled outside
  prod (mirroring how is_fault_injection_enabled gates on
  UNDERSTUDY_ROLE=prod) so it can be forced off, or explicitly
  re-enabled in prod for the eval harness.
- Mark the remaining synthetic-acceptance branch # MOCKED: and
  register it in docs/MOCKS.md (module services.auth_service.app,
  tracked in #16) -- now enforced by
  scripts/check_mocks.py per the earlier services/ scan fix.

Implements build-plan step A1.2 (fix on an already-completed
service), resolving an AGENTS.md section 8 stop-and-ask flagged
during code review.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
rsanghvi21 and others added 3 commits September 11, 2026 07:54
Every tunable across services/ was read ad hoc via os.getenv(...)
with a hardcoded string default, scattered per-module -- no
services/ equivalent of src/understudy/common/config.py's layered
pydantic Settings pattern existed, violating AGENTS.md §5.10 ("no
magic numbers in code... thresholds, weights, timeouts and ceilings
live in config/*.yaml and are named in docs/02-architecture.md").

Add services/_common/settings.py mirroring config.py's shape (layered
load, frozen pydantic BaseModel per section, cached singleton with a
reset hook for tests) as an independent module -- services/ and
src/understudy/ share no imports, and src/understudy/common must stay
psycopg/fastapi-free, so it cannot host these tunables either.

One deliberate divergence: fields keep the exact env var name they
already used before this module existed (e.g.
AUTH_TOKEN_CACHE_MAX_SIZE, FAULT_INJECTION_SEED), rather than a new
UNDERSTUDY_<SECTION>__<FIELD> scheme -- those names are already wired
into deploy/prod/*.yaml's env/configMapKeyRef entries and loadgen's
CLI flag fallbacks, and renaming them would silently break
already-deployed configuration for no benefit.

Adds config/services.yaml (defaults) and config/services.local.yaml
to .gitignore (mirroring config/local.yaml) for local overrides.
Documents every field in docs/02-architecture.md §2.5.1.

This is infrastructure only: no call site reads from Settings yet
(follow-up PR). Implements build-plan step A1.8.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
Replaces every ad hoc os.getenv(...)-with-hardcoded-default and bare
numeric literal identified across services/ with a read from
get_services_settings()/load_services_settings() (added in the prior
PR): edge_gateway's HTTP_TIMEOUT_SECONDS, auth_service's
TOKEN_CACHE_MAX_SIZE, worker's POLL_INTERVAL_SECONDS and its
previously-unparameterized `LIMIT 50` job-list query (now bound via
%s, matching the file's existing parameterized-query style), all of
loadgen's LoadgenConfig defaults and its httpx.Limits/p99 SLA
threshold, db.py's connection-pool defaults, faults.py's injection
seed default and POOL_EXHAUSTION getconn timeout, and metrics.py's
histogram buckets.

loadgen reads via the fresh (uncached) load_services_settings()
rather than the cached singleton, since both LoadgenConfig's defaults
and main.py's argparse defaults are evaluated per CLI invocation /
per-test-construction and must reflect an env var set right before
that call, not whatever was cached first in the process. Every other
service reads the cached get_services_settings() once at
module-import time, matching how these values were already evaluated
once via os.getenv(...) at import time before this change.

Implements build-plan step A1.1/A1.2/A1.3/A1.8 (migration on
already-completed services).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
deploy/system/prod-postgres.yaml, twin-postgres.yaml, and
system-postgres.yaml were three independently hand-written,
near-identical Deployment+Service pairs (identical skeleton;
differing only in namespace, image, POSTGRES_DB, resources, host
port, and system-postgres's extra init-script ConfigMap/volume) with
nothing keeping them from drifting apart.

Add deploy/generate_postgres_manifests.py: a plain Python script
(fits ADR-004 "Python everywhere" -- this repo has no
Helm/Kustomize/Jsonnet anywhere, and adopting one would need a new
ADR, out of scope here) that builds each manifest as a dict from a
shared template function plus a small per-instance parameter table,
and dumps it with PyYAML. --check renders in-memory and diffs
(structurally, not byte-for-byte, since PyYAML formatting need not
match hand-written style) against the committed files, exiting
non-zero on drift; wired into
tests/unit/deploy/test_system_manifests.py so drift fails CI.

The three committed files are now generator output (regenerated via
this script) rather than hand-maintained; the only textual diff from
the prior hand-written versions is cosmetic YAML quoting. Documents
the generator in docs/02-architecture.md §2.10.

Implements build-plan step A1.5/A1.6 (refactor on already-completed
manifests).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
rsanghvi21 and others added 9 commits September 11, 2026 08:19
…red flags

deploy/prod/configmap.yaml's app-config.FAULT_INJECTION_SEED (1337)
exactly matches services/_common/faults.py's hardcoded
DEFAULT_FAULT_INJECTION_SEED, but nothing wired the ConfigMap value in
-- every service silently fell back to the code default. A code-review
pass against PR #6 flagged this as a dead ConfigMap key.

Wire FAULT_INJECTION_SEED into all four deploy/prod/*.yaml Deployments
via configMapKeyRef so the ConfigMap is the actual source of truth in
prod, with the code constant remaining only as the non-k8s/local
fallback.

feature-flags and app-config's ENVIRONMENT/LOG_LEVEL stay unwired:
docs/02-architecture.md §2.5 requires feature flags to be read live at
request time, not from a startup env var, which needs the actuator's
not-yet-built DISABLE_FLAG/REVERT_CONFIG live-read mechanism (Phase 5).
Document that explicitly in the ConfigMap so it is not re-flagged as
dead configuration before Phase 5 lands.

Implements build-plan step A1.7/A1.8 (fix + documentation follow-up).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
deploy/prod/{auth-service,data-service,edge-gateway,worker}.yaml were
four independently hand-written, near-identical Deployment+Service
pairs (identical label scheme, Prometheus scrape annotations,
replicas, imagePullPolicy, resources, probe shape,
UNDERSTUDY_ROLE/FAULT_INJECTION_SEED env; differing only in each
service own env vars, Service type/ports, and
app.kubernetes.io/component) with nothing keeping them from drifting
apart -- the same problem generate_postgres_manifests.py (prior PR)
already solved for the postgres manifests.

Factor the shared PyYAML dumping helpers (indent-forcing Dumper,
literal-block-scalar str subclass) out of that script into
deploy/_manifest_gen_common.py so both generators use one convention
instead of duplicating it. Add deploy/generate_prod_manifests.py
following the same shape: each manifest built as a dict from one
template function plus a small per-instance parameter table, with
--check wired into tests/unit/deploy/test_prod_manifests.py so drift
fails CI.

The four committed files are now generator output; diffed against
what was already committed before touching anything, the only
difference is YAML quoting style (PyYAML's default quote-string-that-
looks-like-a-bool/number behavior), confirming the generator
faithfully reconstructs the existing, reviewed manifests including
the FAULT_INJECTION_SEED wiring from the ConfigMap PR.

Implements build-plan step A1.7/A1.8 (refactor on already-completed
manifests).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CzBeBUsZmE38K1kWeLMhNh
@Harikeshav-R
Harikeshav-R merged commit 3af2918 into main Sep 11, 2026
2 checks passed
An error occurred while trying to automatically change base from feat/prod-and-system-manifests to main September 11, 2026 19:45
@Harikeshav-R
Harikeshav-R deleted the feat/policies-and-loadgen branch September 11, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants