Structure Phase 9 self-audit prompts and label resulting gap issues #24

Problem

Phase 9 (Self-audit) of the generated template asks the agent to "reflect on the cycle just completed" with a flat bullet list of things to consider. The prompts are open-ended, which means:

1. Self-audit findings are inconsistent across cycles (the agent's reflection covers different ground each time).
2. Filed gap issues are unlabeled and unstructured — triage requires reading each body.
3. There's no signal back to RESEARCH.md or CLAUDE.md about which kinds of gaps recur.

Motivation

Per RESEARCH.md §1, §5, structured rubrics outperform free-form prompts for LLM critique. The self-audit phase deserves the same treatment as the Phase 4 self-review.

Connected to the empirical motivation behind PR #15 (rubric-based self-review): the same logic that made self-review more reliable should make self-audit more reliable.

Confidence: medium (extrapolation from rubric findings; not directly studied for retrospective reflection).

Proposal

1. Replace free-form Phase 9 prompts with a structured checklist

Each cycle, the agent answers:

• Identity drift: did this cycle act in accordance with the stated identity? (already in template)
• Instruction clarity: did any step require interpretation or produce a wrong first attempt?
• Edge case coverage: did any failure mode arise that the Edge cases section does not cover?
• Phase friction: did any phase require significantly more or fewer steps than expected?
• Drift candidates: did any decision feel like it should be encoded in create-dev-loop.md rather than re-discovered each cycle?
• External-signal quality: did {{EXTERNAL_SIGNAL_LABEL}} (CI / manual validation) actually catch the kinds of issues it's supposed to catch? (requires Generalize Phase 4 external-signal step to support no-CI repos #21)

For each: PASS / FAIL / "no signal this cycle", with one-line justification.

2. Label gap issues by category

When filing each gap to the dev-loop repo, apply one of a fixed set of labels:

• template-rule — should be promoted into create-dev-loop.md
• repo-specific — belongs only in this skill
• edge-case — should be added to Edge cases
• research-gap — new finding worth a RESEARCH.md entry
• process — meta-issue about how the loop runs

This makes triage faster and surfaces patterns: if template-rule issues recur across multiple child-skill repos, that's a promotion signal.

Acceptance criteria

• Phase 9 has a structured checklist replacing the bullet list
• Step 4 substitution table unchanged (universal rule)
• Filed gap issues are labeled per the taxonomy above
• Labels are created during Step 6 (private repo creation) so they exist when the first cycle runs

Retrofit

Per CLAUDE.md's promotion rule, retrofit PRs needed for all four child skills.

Related

• RESEARCH.md §1, §5 (rubric > free-form, applied here to retrospective reflection)
• PR Reframe self-review as rubric-based and CI-grounded #15 (the precedent: structured rubric for self-review)
• Issue Scope create-dev-loop to also update existing dev loops #16 (update-mode would benefit from labeled gaps to know what to update)

drafted by Claude on behalf of Daniel Stephenson