Skip to content

Add documentation-accuracy and unit-test-expansion work-mode stages to Phase 2 - #52

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/doc-and-test-work-mode-stages
Jun 8, 2026
Merged

Add documentation-accuracy and unit-test-expansion work-mode stages to Phase 2#52
dmccoystephenson merged 1 commit into
mainfrom
feature/doc-and-test-work-mode-stages

Conversation

@dmccoystephenson

@dmccoystephenson dmccoystephenson commented Jun 8, 2026

Copy link
Copy Markdown
Member

What

Adds two additional first-class cycle work-modes to Phase 2 of the generated dev-loop template, so a cycle's unit of work can be more than implementing issues or grooming a PR:

  • Stage A — Documentation accuracy review (sweep): a proactive, repo-wide check for drift between the docs and the actual source. Complements the existing PR-scoped check in Phase 7 (which now labels itself PR-scoped and points to Stage A). References the generated skill's own Phase 7 doc table, so it auto-adapts to each project's documentation sources of truth.
  • Stage B — Unit-test expansion (functionality confidence): add characterization tests to under-covered, behavior-bearing code; never change production code under a test-only cycle (file a bug issue instead). Defensively phrased so it no-ops for repos with no test suite. References the generated skill's own Phase 3 test conventions.

Supporting edits: Phase 3 cross-references Stage B; the Plan summary generalizes "Issues in scope" → "Work in scope".

Design notes

  • The new text introduces no new {{placeholders}} — it references existing phases (Phase 7 table, Phase 3 conventions, Phase 3 scope ceiling) by number, which are stable. So no Step 4 substitution-table rows are needed, and the stages adapt per project automatically.
  • Phase numbers are unchanged (edge-case instructions reference them by number).
  • No new Step added, so the README Step enumeration is unaffected.

Research grounding

  • RESEARCH.md §2 — agent-authored documentation and CI/build PRs merge at substantially higher rates than feature/bugfix PRs; elevating doc-accuracy and test work to first-class modes leans into that gradient.
  • RESEARCH.md §3 — regression/characterization tests are the signal that distinguishes a real fix (and real behavior) from a coincidental patch; Stage B hardens that signal.

Follow-up

Retrofit PRs against every existing generated skill that predates this change are being opened separately, per the "Promoting a rule into the template" convention in CLAUDE.md.

@claude
Add documentation-accuracy and unit-test-expansion work-mode stages t…
004f581 
…o Phase 2

Phase 2 now presents two additional first-class cycle work-modes beyond
implementing issues: Stage A (repo-wide documentation accuracy sweep,
complementing the PR-scoped Phase 7 check) and Stage B (unit-test
expansion via characterization tests that never change production code).
Both reference the generated skill's own Phase 7 doc table and Phase 3
test conventions, so they auto-adapt per project and introduce no new
placeholders. Phase 7 now labels itself PR-scoped and points to Stage A;
Phase 3 cross-references Stage B; the plan summary generalizes "Issues in
scope" to "Work in scope".

Grounded in RESEARCH.md §2 (documentation and test PRs merge at higher
rates) and §3 (regression/characterization tests as confidence signal).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dmccoystephenson
dmccoystephenson merged commit 4567145 into main Jun 8, 2026
@dmccoystephenson
dmccoystephenson deleted the feature/doc-and-test-work-mode-stages branch June 8, 2026 01:21
@dmccoystephenson dmccoystephenson mentioned this pull request Jul 25, 2026
Merged
6 of 7 tasks
dmccoystephenson added a commit that referenced this pull request Jul 25, 2026
@dmccoystephenson @claude
Fix documentation drift across the three sources of truth (#57)
43cdcdd 
* Fix documentation drift across the three sources of truth

Stage A documentation-accuracy sweep findings:

- create-dev-loop.md Phase 1 hardcoded "tests green" / "CI must be green
  first" where the rest of the template uses {{EXTERNAL_SIGNAL_LABEL}};
  drift introduced after PR #28 generalized the anchor
- RESEARCH.md was missing Implementations entries for PR #28 (cites §1)
  and PR #52 (cites §2 and §3), which CLAUDE.md requires
- RESEARCH.md carried literal \" heredoc-escaping artifacts in the §3 and
  §7 Implementations entries
- README Step 3 named only template-version, omitting generated-at;
  Step 6 omitted the gap-issue label seeding; the generated-skill
  structure list omitted the external signal anchor

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Record this PR's own Implementations entry under RESEARCH.md finding 1

Self-review rubric caught the omission: a PR whose purpose is to backfill
missing Implementations entries must not itself omit one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Avoid implying the external signal anchor is a CI-or-manual binary

The Step 4 EXTERNAL_SIGNAL_LABEL row allows a third case (a
project-specific phrase such as "snapshot regression suite"), which the
README bullet's two-way phrasing excluded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dmccoystephenson dmccoystephenson mentioned this pull request Aug 3, 2026
Merged
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
@dmccoystephenson
Fold in round-2 readiness nice-to-haves: symlink flag, doc-table clar…
9b6aa89 
…ity, test comment (#103)

- README.md: use ln -sf for the Installation symlink, matching Step 5's
  own registration convention (create-dev-loop.md:624).
- CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob
  deliberately excludes config.yml, which has no restatable conventions.
- tests/test_check_docs.py: note in the module docstring that any new
  test class must reset check_docs.errors in its own setUp(), so a
  future addition doesn't silently leak state between tests.

Not included, with reasons:
- RESEARCH.md's "Observed effect: pending — needs N cycles of data"
  entries (PRs #29/#30/#52/#54): this needs real usage data the
  maintainer would have to supply; nothing to mechanically edit here
  without fabricating an observed effect.
- The flagged openai.com RESEARCH.md link: re-verified via an
  independent fetch, still 403 (not 404) — consistent with bot/WAF
  blocking a non-browser client, not a confirmed dead link. Left as-is.
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
@dmccoystephenson @claude
Add documentation-accuracy and unit-test-expansion work-mode stages t…
865ef56 
…o Phase 2 (#52)

Phase 2 now presents two additional first-class cycle work-modes beyond
implementing issues: Stage A (repo-wide documentation accuracy sweep,
complementing the PR-scoped Phase 7 check) and Stage B (unit-test
expansion via characterization tests that never change production code).
Both reference the generated skill's own Phase 7 doc table and Phase 3
test conventions, so they auto-adapt per project and introduce no new
placeholders. Phase 7 now labels itself PR-scoped and points to Stage A;
Phase 3 cross-references Stage B; the plan summary generalizes "Issues in
scope" to "Work in scope".

Grounded in RESEARCH.md §2 (documentation and test PRs merge at higher
rates) and §3 (regression/characterization tests as confidence signal).

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
@dmccoystephenson @claude
Fix documentation drift across the three sources of truth (#57)
6a5a857 
* Fix documentation drift across the three sources of truth

Stage A documentation-accuracy sweep findings:

- create-dev-loop.md Phase 1 hardcoded "tests green" / "CI must be green
  first" where the rest of the template uses {{EXTERNAL_SIGNAL_LABEL}};
  drift introduced after PR #28 generalized the anchor
- RESEARCH.md was missing Implementations entries for PR #28 (cites §1)
  and PR #52 (cites §2 and §3), which CLAUDE.md requires
- RESEARCH.md carried literal \" heredoc-escaping artifacts in the §3 and
  §7 Implementations entries
- README Step 3 named only template-version, omitting generated-at;
  Step 6 omitted the gap-issue label seeding; the generated-skill
  structure list omitted the external signal anchor

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Record this PR's own Implementations entry under RESEARCH.md finding 1

Self-review rubric caught the omission: a PR whose purpose is to backfill
missing Implementations entries must not itself omit one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Avoid implying the external signal anchor is a CI-or-manual binary

The Step 4 EXTERNAL_SIGNAL_LABEL row allows a third case (a
project-specific phrase such as "snapshot regression suite"), which the
README bullet's two-way phrasing excluded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
@dmccoystephenson
Fold in round-2 readiness nice-to-haves: symlink flag, doc-table clar…
090ba3c 
…ity, test comment (#103)

- README.md: use ln -sf for the Installation symlink, matching Step 5's
  own registration convention (create-dev-loop.md:624).
- CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob
  deliberately excludes config.yml, which has no restatable conventions.
- tests/test_check_docs.py: note in the module docstring that any new
  test class must reset check_docs.errors in its own setUp(), so a
  future addition doesn't silently leak state between tests.

Not included, with reasons:
- RESEARCH.md's "Observed effect: pending — needs N cycles of data"
  entries (PRs #29/#30/#52/#54): this needs real usage data the
  maintainer would have to supply; nothing to mechanically edit here
  without fabricating an observed effect.
- The flagged openai.com RESEARCH.md link: re-verified via an
  independent fetch, still 403 (not 404) — consistent with bot/WAF
  blocking a non-browser client, not a confirmed dead link. Left as-is.
@dmccoystephenson dmccoystephenson mentioned this pull request Aug 8, 2026
Merged
6 of 7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dmccoystephenson