Skip to content

Give COMPILE_CMD/TEST_CMD explicit no-build-system guidance - #58

Merged
dmccoystephenson merged 2 commits into
mainfrom
fix/no-build-system-guidance
Jul 25, 2026
Merged

dmccoystephenson merged 2 commits into
mainfrom
fix/no-build-system-guidance

Conversation

@dmccoystephenson

@dmccoystephenson dmccoystephenson commented Jul 25, 2026 •

Copy link
Copy Markdown
Member

Summary

{{COMPILE_CMD}} and {{TEST_CMD}} assumed every target repo has a runnable build and test command, so a docs-only or config-only repo (including create-dev-loop itself) forced the generator to improvise the same fallback per generation instead of following an encoded rule. LINT_CMD ("omit section if absent") and EXTERNAL_SIGNAL_CMD (explicit manual-validation branch) already handle their absence; these two did not.

  • COMPILE_CMD row — adds no-build-system guidance: substitute the cheapest mechanical consistency check that would actually catch breakage (placeholder grep, jq -e, schema lint), or a shell comment naming the real gate so the fence stays valid bash.
  • TEST_CMD row — adds no-test-suite guidance: confirm the absence (don't assume), substitute a shell comment pointing at the project's manual validation steps, and use the same text at every {{TEST_CMD}} site (Phase 3 verify block, Phase 8 rebase fence).
  • New {{VALIDATION_NOTE}} (conditional, Phase 3) — one-line parenthetical under the verify fence stating what actually gates correctness when the fence holds comments rather than commands. Omitted entirely for repos with a real build.
  • New {{REVALIDATE_INSTRUCTION}} (Phase 6) — replaces the hardcoded "Run {{TEST_CMD}} after every fix", which rendered as nonsense ("Run # re-run the checklist") for a repo with no test command.
  • Step 2 exploration table — the "What is the test command?" row now says to record "none" explicitly rather than leaving it implicit.
  • Step 4 prose — one paragraph requiring COMPILE_CMD / TEST_CMD / REVALIDATE_INSTRUCTION / VALIDATION_NOTE / EXTERNAL_SIGNAL_CMD to name the same check, and directing the generator to reword the three test-naming Edge cases entries accordingly.
  • README — "Build & test commands" bullet now states the no-build-system fallback, matching the "External signal anchor" bullet's shape.

No Step was added, removed, or renumbered, so README's "What it does" list stays 1:1 with Steps 1–7. Phase numbers are unchanged.

Research grounding

RESEARCH.md §8 (prompt and model drift) is adjacent but not implemented here: its point is that forward-only template fixes leave existing skills silently lagging, and a rule re-derived per generation is the same silent-lag failure by another route. No finding speaks directly to how a template should model build/test commands — flagging that honestly rather than reaching for one. No Implementations entry was added under §8, because this PR does not implement any of that finding's implication bullets (version header, regression fixtures, retrofit channel — already shipped in #33/#35).

Existing generated skills predating this change are unaffected (their placeholders are already substituted); the create-dev-loop-dev-loop skill already renders the improvised form this PR encodes, so no retrofit PR is needed.

Test plan

  • Every {{TOKEN}} in the embedded template has a Step 4 substitution-table row — verified by diffing the token list against the table rows (VALIDATION_NOTE, REVALIDATE_INSTRUCTION rows added).
  • Fence escaping intact — no fenced block was added or removed inside the template; the new line sits after the existing escaped fence close.
  • Phase numbers unchanged; README "What it does" Steps still 1:1 with create-dev-loop.md.
  • UNVERIFIED — the external anchor (/create-dev-loop run end-to-end against a real repo) could not run in this headless session: it writes outside the repo (~/local-skills/, ~/.claude/commands/) and creates a GitHub repo. This PR changes the embedded template and the Step 4 table, so per the loop's own rule the anchor is required — do not merge without a human running /create-dev-loop against a repo with no test suite and confirming the Phase 3/6 sites render sensibly.

Closes #56


This PR description was drafted during a Gardener session (Stephenson-Software/gardener).

dmccoystephenson and others added 2 commits July 25, 2026 19:20
Closes #56

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dmccoystephenson

dmccoystephenson commented Jul 25, 2026 •

Copy link
Copy Markdown
Member Author

Self-review rubric (one intrinsic-critique pass; external anchor is UNVERIFIED — see below):

  • Scope: PASS — 2 files, +11/−5. Every hunk traces to COMPILE_CMD/TEST_CMD assume a runnable test suite exists #56: the two table rows it names, the three sites it names (Phase 3 fence, Phase 6 sentence, Phase 8 rebase fence via the shared TEST_CMD substitution), plus the two new placeholder rows those sites require and the README bullet that documents the behavior. No unrelated formatting or renames.
  • Tests-new: N/A — this repo has no automated test suite (CLAUDE.md, "Testing changes"); the PR adds no script or tool with testable behavior.
  • Tests-fix: PARTIAL — the drift is observable by inspection rather than by a revertible check: with the change reverted, Run \{{TEST_CMD}}` in Phase 6 renders as "Run \# re-run the Phase 3 manual validation checklist` after every fix" for a repo with no test command, and the COMPILE_CMD/TEST_CMD rows give the generator no instruction for that case. With it applied, {{REVALIDATE_INSTRUCTION}} renders "Re-run the Phase 3 manual validation checklist after every fix" and both rows carry explicit branches. The existing create-dev-loop-dev-loop skill is the empirical exhibit — it already improvises exactly this rendering, which is what COMPILE_CMD/TEST_CMD assume a runnable test suite exists #56 reports.
  • Sibling structure: PASS — the two new rows sit in the substitution table adjacent to the placeholders they qualify (VALIDATION_NOTE/REVALIDATE_INSTRUCTION after LINT_CMD, before the EXTERNAL_SIGNAL_* pair they must agree with) and follow the same "how to determine it, with an example" shape as their neighbors.
  • Sibling renames: PASS — no identifier renamed. COMPILE_CMD/TEST_CMD are a parallel pair and both were amended in the same commit rather than one being fixed alone.
  • Docs: PASS — README's "Build & test commands" bullet updated in the same PR; RESEARCH.md needed no change (see the PR body's honest "no finding applies directly"); CLAUDE.md unchanged (its rules still describe the template accurately).
  • Issue resolution: PASS — gh issue view 56 re-read before pushing; its title, its three named line references, and its "Suggested fix" (explicit no-build/no-test guidance + graceful degradation at those sites, phase numbers stable, table rows for any new placeholder) are all addressed.
  • Placeholder table completeness: PASS — token list extracted from the file and diffed against the table rows; VALIDATION_NOTE and REVALIDATE_INSTRUCTION are the only additions and both have rows.
  • Fence escaping intact: PASS — no fenced block was added or removed inside the template; the new {{#if VALIDATION_NOTE}} line sits after the existing escaped close at what is now line 265, so the escaped-fence count is unchanged.
  • README/Step 1:1: PASS — no Step added, removed, or renumbered; README's "What it does" list still enumerates Steps 1–7.
  • RESEARCH.md citation: PASS — the PR body cites §8 as adjacent, states plainly that no finding speaks directly to build-command modeling, and explains why no Implementations entry was added.
  • manual validation (external anchor): UNVERIFIED — /create-dev-loop cannot run end-to-end in this headless session (it writes to ~/local-skills/ and ~/.claude/commands/ and creates a GitHub repo). This PR changes the embedded template and the Step 4 table, so per Phase 4's rule the anchor is required: do not merge until a human runs /create-dev-loop against a repo with no test suite and confirms the Phase 3 verify block, the Phase 6 sentence, and the Phase 8 rebase fence render sensibly.

One FAIL found and fixed in e1f7de5 before this comment: the reword list omitted the "re-run tests" step in the Phase 8 "Branch is behind main" edge case — the same class of test-assuming prose the PR is fixing, and one #56 explicitly flags (create-dev-loop.md:530), so leaving it out would have half-solved the issue.

Two judgment calls left deliberately, for a human to weigh rather than silently resolve: (1) the Phase 3 lead-in still reads "Verify the build is clean:" even for a repo with no build — {{VALIDATION_NOTE}} directly beneath it carries the correction, and generalizing the sentence would churn every generated skill for a cosmetic gain; (2) the Phase 8 rebase fence keeps a bare {{TEST_CMD}} rather than gaining its own placeholder, on the theory that one substitution used consistently is easier for a generator to get right than a third near-duplicate placeholder.


drafted by Claude on behalf of Daniel Stephenson

@dmccoystephenson
dmccoystephenson merged commit 5905662 into main Jul 25, 2026
@dmccoystephenson
dmccoystephenson deleted the fix/no-build-system-guidance branch August 3, 2026 06:01
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
* Give COMPILE_CMD and TEST_CMD no-build-system guidance

Closes #56

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Include the Phase 8 rebase step in the test-wording reword list

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

COMPILE_CMD/TEST_CMD assume a runnable test suite exists

1 participant