Repository navigation
Structure Phase 9 self-audit prompts and label gap issues - #34
Merged
Merged
Conversation
Closes #24 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dmccoystephenson
left a comment
Member
Author
There was a problem hiding this comment.
Self-review rubric:
- Scope: PASS — 2 files, 44 lines; each change maps to one of #24's 4 AC items or the CLAUDE.md PR-27 Implementations rule.
- Tests-new: PASS — the new self-audit rubric is exercised by this very cycle (I'll run it after merge); the new label taxonomy is exercised by the Step 6 loop syntax (verified shell expansion works with
IFS=:). - Tests-fix: N/A — not a bug fix.
- Sibling structure: PASS — Phase 9 rubric block matches the Phase 4 rubric structure (bold-item-name + colon + question; PASS/FAIL/no-signal scoring); new Step 6 bash block matches the existing Step 6 bash-block style (fenced bash, no surrounding prose interruption).
- Sibling renames: PASS — the label-taxonomy names (
template-rule,repo-specific, etc.) appear verbatim in Phase 9 AND Step 6; siblings renamed/added together. - Docs: PASS — Phase 7 sources (
create-dev-loop.md,README.md) reviewed; README is high-level and doesn't enumerate Phase 9 internals (correctly out of scope). RESEARCH.md §1 and §5 get Implementations entries per PR-27 rule. - Issue resolution: PASS — all 4 AC items met: (1) structured checklist replaces bullet list; (2) substitution table unchanged; (3) gap-issue command takes
--label; (4) labels created in Step 6 before first cycle runs. - Validation: PASS — placeholder count unchanged at 22; Step 6 bash block uses standard
\``` plain fence (Step 6 is OUTSIDE the outer template, so no escape needed); Phase 9 fenced block uses leading-backslash escape (Phase 9 is INSIDE the outer template). - {{EXTERNAL_SIGNAL_LABEL}}: N/A — no CI; manual validation against diff served as anchor.
- Placeholder-consistency: N/A — no template placeholder added.
- Escaping: PASS — confirmed by reading the diff: Step 6 label-creation block uses plain triple-backticks (it's outside the embedded template), Phase 9 issue-create block uses
\``bash` with leading backslash (inside template).
Adversarial probe — 'Does the new rubric's External-signal-quality item make sense for repos where the external anchor is just "run the test command locally"?' Considered: yes, the item asks 'did it catch the kinds of issues it's supposed to catch this cycle' — for any anchor (CI, manual validation, local tests), the agent can score it. No exclusion needed.
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
Closes #24 Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three changes to the generated dev-loop template:
Phase 9 self-audit becomes a structured rubric. Replaces the free-form bullet list with a 6-item rubric (identity drift, instruction clarity, edge case coverage, phase friction, drift candidates, external-signal quality) — each scored PASS / FAIL / no-signal with a one-line justification. Mirrors the Phase 4 self-review structure shipped in PR Reframe self-review as rubric-based and CI-grounded #15. Grounded in RESEARCH.md §1 and §5 (structured rubrics outperform free-form for LLM critique).
Filed gap issues now carry a label from a 5-label taxonomy:
template-rule,repo-specific,edge-case,research-gap,process. Makes triage faster and surfaces patterns across child-skill repos.Step 6 creates the labels at repo-creation time so they exist when the first cycle's Phase 9 fires. Uses a single
forloop over a colon-delimited list ofname:color:descriptionentries.Closes #24
Test plan
--labelargument and lists the 5-label taxonomy inline{{placeholders}}introduced (universal rule — placeholder count still 22 withTEMPLATE_VERSIONandGENERATED_ATfrom PR Add template-version header to generated skills for drift detection #33)gh issue view 24and confirmed title/body matchcreate-dev-loop.md,RESEARCH.md; no matchesNote on label taxonomy duplication
The taxonomy is listed in two places: Phase 9 (where the agent picks one) and Step 6 (where the agent creates the labels). They must stay in sync. A future PR could DRY this — e.g. a single placeholder
{{GAP_LABEL_TAXONOMY}}— but that's overengineering for 5 entries. Listed in both spots verbatim; renaming-siblings-together rule applies if anyone changes the names.drafted by Claude on behalf of Daniel Stephenson