Repository navigation
Stop prescribing commands that headless harnesses refuse - #176
Conversation
Phase 5 now reads reviews through gh pr view and treats the inline-comment gh api read as optional, since it can be refused even though it only reads. A refusal or an empty result is no longer readable as "no review comments". Phase 3's command-substitution rule now also covers shell variable loops and $-anchored sed/awk, with a single jq-filtered gh call as the replacement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Self-review rubric (head Universal items:
Repo-specific items:
Points for the maintainer's review, none of which were changed:
Summary: the change reads correctly and CI is green, but the two FAIL items hold it for a maintainer, so it is not being merged by this dispatch. This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener). drafted by Claude on behalf of Daniel Stephenson |
There was a problem hiding this comment.
🟡 Changes recommended
The review flow lacks a blocking fallback when review summaries are insufficient and inline comments are inaccessible, allowing progression toward merge without the findings Phase 6 needs.
1 open issue
What changed in this PR
This PR updates the generated dev-loop template to reduce command refusals in headless harnesses, addressing the command patterns reported in #135 and #165.
Changes:
- Expands Phase 3 guidance to avoid shell-variable loops and favor a single filtered
ghcall. - Uses
gh pr viewcommands as Phase 5 defaults, retaining the inline-comment API read as optional and documenting the defaults’ limitations.
| File | Description |
|---|---|
create-dev-loop.md |
Revises Phase 3 command guidance and Phase 5 review retrieval instructions for restricted harnesses. |
Review effort: Balanced
| gh pr view <number> --json reviews | ||
| \`\`\` | ||
|
|
||
| These show conversation comments and each review's state and summary, but not the text or `path:line` anchor of inline review comments. Where the summary names the findings, Phase 6 can work from it. Where the harness allows it, this read fills the gap: `gh api repos/{{GITHUB_OWNER}}/{{GITHUB_REPO}}/pulls/<number>/comments --jq '.[] | "\(.path):\(.line) — \(.body)"'`. A command allow-list or attribution hook can refuse it even though it only reads. **A refusal, or an empty `--comments` result, is never evidence that no review comments exist** — check `--json reviews` for a submitted review before concluding there is nothing to address. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The Copilot finding on
The self-review rubric was not re-run, as Phase 4 caps it at one pass. This comment was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener). drafted by Claude on behalf of Daniel Stephenson |

Summary
Commands that harnesses refuse were prescribed, or left unmentioned, in two parts of the generated-skill template. Both are corrected in the template body of
create-dev-loop.md.gh pr view <number> --commentsplusgh pr view <number> --json reviews. What is left out by those two commands is stated: the text andpath:lineanchor of inline review comments. Thegh api .../pulls/<number>/commentsread is kept as an optional extra, withBody:removed from its--jqtemplate as suggested in Phase 5's read-only 'gh api .../pulls/<n>/comments --jq' is refused by the attribution hook as an unsigned post #135. It is also stated that a refusal or an empty--commentsresult is never evidence that no review comments exist.$-anchoredsed/awkare added to the "Avoid command substitution" rule, and onegh ... --json <fields> -q '<jq filter>'call is prescribed in place of a per-item shell loop.Closes #135
Closes #165
Deviations from #135, based on source checks made in this dispatch:
gh pr view --comments"returns the same information" as thegh apicall. That was not confirmed. When it was run against PR Make Steps 6 and 7 safe to re-run under overwrite mode #175, conversation comments and the Copilot review summary were shown, but no inline comment text was shown.--jqtemplate without a literalBody:label is accepted. In this dispatch,gh api repos/Stephenson-Software/create-dev-loop/pulls/175/comments --jq '.[] | "\(.path):\(.line) — \(.body)"'was still refused withThis command requires approval. The call is therefore written as optional, not as the fix.gardener interface: Steps 3, 5, and 6 are untouched, so no breaking change is introduced for gardener.
Retrofit: the Phase 5 snippet and the Phase 3 rule are copied into every generated skill, so a retrofit pass is likely needed for every existing
<slug>-dev-loopskill. Per #135, Phase 5 was already fixed locally inmedieval-factions-dev-loop. The old snippet is still carried bycreate-dev-loop-dev-loop.Research grounding
No
RESEARCH.mdfinding applies. These corrections are based on refused commands that were observed, and no empirical claim is relied on, so no Implementations entry is added.Doc sync check
README.md's "What it does" Step list still matchescreate-dev-loop.md's Steps 1:1 (no Step was changed){{placeholder}}added or changed has a corresponding Step 4 substitution-table row (none were added;GITHUB_OWNERandGITHUB_REPOare unchanged)Test plan
python3 scripts/check_docs.py: "Doc consistency check passed."python3 -m unittest discover -s tests: 42 tests were run, OK.gh pr view 175 --commentswas run, and the claims made in Phase 5 were checked against its output.gh apiread was attempted in this dispatch and was refused, as warned in Phase 5. Its output format has therefore not been observed.gh pr list --state merged --json number,mergedAt -q '.[]|select(.number|IN(174,175))'./create-dev-looprun was performed, because a GitHub repo is created in Step 6 and files outside the checkout are written in Steps 3 and 5, and neither is authorized in a headless dispatch ([self-audit] The repo's own validation anchor is structurally unrunnable in a headless dispatch, so template changes can never merge autonomously #109). None of the fiveCLAUDE.mdchecklist items was exercised, so this PR is left for a maintainer to merge.The template body is not covered by anything automated, so no evidence about generated-skill behavior is given by a green
doc-consistencyjob. The loop refusal named in #165 was not re-run here. It is taken from the observations recorded in #165 and from a later dispatch's notes.Issues not picked this cycle
CLAUDE.mdhalf of Step 4 and CLAUDE.md still place the version comments at the top of the generated skill #173: explicit authorization is required forCLAUDE.mdedits.ghexample, andgrepreplaced throughout generated checklists) than the two small edits made here.reviewRequestsevidence in its comments.This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).
🤖 Generated with Claude Code
drafted by Claude on behalf of Daniel Stephenson