Skip to content

feat(planner): implement phase 3b planner, playbooks, and inverse synthesis - #28

Merged
Harikeshav-R merged 9 commits into
mainfrom
feat/phase-3b
Sep 14, 2026
Merged

Harikeshav-R merged 9 commits into
mainfrom
feat/phase-3b

Conversation

@rsanghvi21

Copy link
Copy Markdown
Collaborator

Build-Plan Step Implemented

  • Step: Phase 3B — Steps B3.1 through B3.6 and Checkpoint B3

Checkpoint Execution & Output

$ uv run ust plan --context fixtures/context_bad_deploy.json --count 3 --fake
Generated 4 candidate plan(s) for inc_bad_deploy_001:
  [0] plan_cand_0: rollback_deploy workload=data-service (inverse=rollback_deploy) - Rollback recent deployment to last known stable commit
  [1] plan_cand_1: scale_workload workload=data-service (inverse=scale_workload) - Scale up workload replicas to handle traffic load
  [2] plan_cand_2: restart_workload workload=data-service (inverse=restart_workload) - Trigger rolling restart of workload pods to resolve transient issues
  [3] plan_cand_3: no_action workload=data-service (inverse=none) - Maintain current state without automated mutation

$ uv run ust plan --context fixtures/context_bad_deploy.json --count 3 --fake --seed 1 --twice
Planner run 1 (4 plans): ['rollback_deploy', 'scale_workload', 'restart_workload', 'no_action']
Planner run 2 (4 plans): ['rollback_deploy', 'scale_workload', 'restart_workload', 'no_action']
Action-type stability: 1.00 (4/4 matching)

$ uv run ust playbook seed --from fixtures/playbooks_seed.json --fake
Seeded 3 playbook(s) from fixtures\playbooks_seed.json

$ uv run ust playbook match --context fixtures/context_bad_deploy.json --fake
Matched playbook 'pb_bad_deploy_data_service' (cosine: 1.00 > 0.8):
  Action: rollback_deploy
  Workload: data-service
  Confirmation Reason: Matches N+1 query regression on data-service; rollback to deadbeef is safe and verified.
  Plan ID: plan_pb_rollback_data_service

Live cluster workload resolution:
checkpoint: unverified (k3d cluster offline in test environment; live workload query implemented via get_cluster_workloads in graph/k8s.py with fallback to graph nodes)

ADRs Relied On

  • ADR-005 (Closed action enum: 5 actions only)
  • ADR-007 (Deterministic inverse synthesis: inverse is derived from environment/cluster state, never LLM hallucinated)
  • ADR-019 (Playbook matching and library: signature embeddings with pgvector and LLM confirmation)
  • ADR-028 (pgvector embedding dimensionality: 1024-dimension float vector embeddings)

Mock Registry Disclosures

  • None

Documentation Updates

  • Updated CHECKLIST.md ticking verified Checkpoint B3 criteria (steps B3.1–B3.6 and verified Checkpoint B3 items).

Definition of Done

  • Implementation complete for the named build-plan step
  • Unit tests written; 100% line and branch coverage of the added lines
  • make check green (ruff, mypy --strict, import-linter, pytest)
  • Checkpoint command run and output matches, OR marked unverified with a reason
  • Docs updated where this change made them untrue (or "none required", with why)
  • No new # MOCKED: marker without a row in docs/MOCKS.md
  • Conventional commit(s), conventional branch, PR opened, never pushed to main

rsanghvi21 and others added 9 commits September 13, 2026 15:30
Establish the candidate-generation prompt engine and response schemas for the Phase 3B remediation pipeline over the closed ActionType enum.

The planner needs to generate diverse, viable, and reversible remediation plans from serialized incident diagnostics while strictly adhering to safety invariants and closed action boundaries. This commit provides structured context formatters, system and user prompt builders, JSON schema extraction, response parsing with markdown fence stripping, contract conversion adapters, and the PlannerError error type.

Implements build-plan step B3.1.
…lans

Implement schema validation with retry <= 2, commit SHA and image digest resolution, cluster workload verification, and graph blast-set validation. Enforce drop, never repair for invalid plans.

Implements build-plan step B3.2.
Synthesize and append a compliant, typed NO_ACTION remediation candidate when absent from surviving candidate plans, guaranteeing that baseline non-intervention is always evaluated on tournament ballots.

Provides create_no_action_plan and ensure_no_action_candidate in the planner validation engine, integrating with generate_candidates_with_retry, LLMPlanner, and FakePlanner. Introduces the ust plan CLI entrypoint and fixtures/context_bad_deploy.json for checkpoint plan inspection.

Implements build-plan step B3.3.
Synthesize inverse remediation plans deterministically for all action types rather than relying on unverified LLM generation, ensuring strict reversibility per ADR-016 and formal invariant K9. Supports rollback roll-forward inversion, scale inverse delta, replica restoration, flag toggles, and config reversion.

Implements build-plan step B3.4.
Add deterministic playbook signature synthesis, 1024-dimension OpenRouter embedding client, pgvector cosine similarity retrieval in both Postgres and in-memory stores, and safety-constrained LLM confirmation with rejection of hallucinated IDs. Register CLI subcommands for seed, match, and list, and add seed fixture library.

Implements build-plan step B3.5.
…counters

Add playbook write-back mechanism (write_playbook and PlaybookWriter) keyed by deterministic incident signature, updating success/failure counters and atomically appending resolved run IDs to evidence_refs. Wire into record_run orchestrator node when prod_outcome is resolved or failed. Expose ust playbook write CLI subcommand and implement in-memory and PostgreSQL store persistence.

Implements build-plan step B3.6.
Address all 13 code review findings across standards and spec axes for
build-plan step B3 (Planner, Playbooks, and Inverse Synthesis).

- Graph: Expose live cluster workload discovery in k8s.py.
- Planner API & Orchestrator: Thread playbook candidate into planner
  prompt candidate generation; retrieve playbook before planning.
- Fake Planner: Emit 3 active candidates plus NO_ACTION (N+1 = 4 plans total)
  per Checkpoint B3 requirements.
- Schema Validation: Replace ActionType if-elif ladders with table dispatch;
  handle typed GraphError in blast set computation.
- Inverse Synthesis: Eliminate hardcoded configuration defaults and avoid
  trusting attached inverse plans; resolve prior state deterministically.
- Playbook Write: Prioritize exact signature lookup before playbook ID,
  and catch PlaybookEmbeddingError explicitly.
- Store & CLI: Deduplicate SQL counter increment logic and CLI planner runs;
  fall back to dependency graph nodes when cluster is unreachable.

Implements build-plan step B3.
Reflect verified Checkpoint B3 items for phase 3b. Leave live cluster
resolution box unchecked as cluster was offline during local test run.

Implements build-plan step B3.
# Conflicts:
#	CHECKLIST.md
#	src/understudy/cli.py
#	src/understudy/common/errors.py
#	tests/unit/test_cli.py
@Harikeshav-R
Harikeshav-R merged commit 8b34e30 into main Sep 14, 2026
2 checks passed
@Harikeshav-R
Harikeshav-R deleted the feat/phase-3b branch September 14, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants