Skip to content

Add hard PR scope gates: LOC ceiling, do-not-auto-merge paths - #30

Merged
dmccoystephenson merged 2 commits into
mainfrom
feature/pr-scope-gates
May 25, 2026
Merged

dmccoystephenson merged 2 commits into
mainfrom
feature/pr-scope-gates

Conversation

@dmccoystephenson

@dmccoystephenson dmccoystephenson commented May 25, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds two structural gates to the generated dev-loop template, grounded in RESEARCH.md §2 (autonomous-agent PRs fail at higher rates when large or touching sensitive paths):

  1. Phase 3 scope ceiling. Before pushing, check the cumulative diff. Soft ceiling: ~400 net LOC, ~10 files. Hard stop: ~800 LOC, ~20 files. If exceeded, drop an issue or split the PR.

  2. Phase 8 do-not-auto-merge check. Before invoking gh pr merge, compare modified paths against a list. Universal entries cover .github/workflows/*, security/, and >50-line deletions. New {{DO_NOT_AUTO_MERGE}} placeholder lets each generated skill add repo-specific paths (e.g. plugin.yml, pom.xml).

The bug-fix-requires-test gate mentioned in #17's proposal section is deferred to #18 (which has its own AC for a Phase 8 regression-test gate); avoiding overlap.

Closes #17

Test plan

  • Phase 3 has scope-ceiling check with git diff --stat command and explicit "what to do if exceeded" language (drop an issue or split)
  • Phase 8 has do-not-auto-merge path check with git diff --name-only command, before gh pr merge
  • New {{DO_NOT_AUTO_MERGE}} placeholder appears in template body AND in Step 4 substitution table (verified with grep -oP '\{\{[A-Z_]+\}\}')
  • New {{#if DO_NOT_AUTO_MERGE}} conditional wraps repo-specific section so generated skills with no repo-specific entries don't render an empty block
  • This PR's own diff respects the new ceiling: 27 net LOC across 3 files (well under 400/10 soft, 800/20 hard)
  • README.md "Generated skill structure" gets a bullet for the new repo-specific encoding ({{DO_NOT_AUTO_MERGE}}), keeping it in sync with the template
  • Per the rule shipped in PR Promote 'verify each Closes #N' to template Phase 4 #26, ran gh issue view 17 and confirmed title/body match
  • Per the rule shipped in PR Track implementation outcomes back into RESEARCH.md #27, added an Implementations entry under RESEARCH.md §2

Fixture-validation note

#17's test plan asks for "a fixture issue that intentionally would produce a too-large diff" to confirm the gate stops it. Out of scope for this PR (would require a fixture repo + tooling that doesn't exist). The gate is currently authority-based, not enforced — a future improvement would be a pre-push hook that fails the push if git diff --stat exceeds the threshold. Filing as a follow-up after merge.


drafted by Claude on behalf of Daniel Stephenson

BuildTools and others added 2 commits May 25, 2026 12:46
Closes #17

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nested backticks in the format example were ambiguous; describe the
format in prose instead of trying to inline-code a code-snippet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@dmccoystephenson dmccoystephenson left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review rubric:

  • Scope: PASS — 3 files modified (create-dev-loop.md, README.md, RESEARCH.md); every change traces to #17 AC, the CLAUDE.md PR-27 rule (Implementations entry), or doc-sync (README's Generated skill structure list).
  • Tests-new: PASS — the new {{DO_NOT_AUTO_MERGE}} placeholder has a substitution-table row with format guidance; the scope ceiling is exercised by this PR's own git diff --stat origin/main (27 LOC, 3 files — under the soft ceiling).
  • Tests-fix: N/A — not a bug fix.
  • Sibling structure: PASS — new Phase 3 'Scope ceiling' paragraph matches the 'Bold label. explanation + fenced command' pattern of its siblings; new Phase 8 do-not-auto-merge block similarly uses the bold-label + fenced-command pattern.
  • Sibling renames: N/A — no identifiers renamed.
  • Docs: PASS — README updated to add 'Do-not-auto-merge paths' to the Generated-skill-structure list (it's a new repo-specific encoding); RESEARCH.md §2 gets an Implementations entry for PR #30 per the CLAUDE.md rule.
  • Issue resolution: PASS-with-deferral — #17 ACs 1-3 met (ceiling check, do-not-auto-merge check, new placeholder in table); AC 4 (fixture-issue test) explicitly deferred in PR body, since no fixture tooling exists.
  • Validation: PASS — placeholder grep confirms {{DO_NOT_AUTO_MERGE}} exists in template body, has matching {{#if DO_NOT_AUTO_MERGE}} conditional wrap, and has substitution-table row; new fenced blocks (lines 194-196, 334-336) are properly escaped with leading-backslash triple-backticks.
  • {{EXTERNAL_SIGNAL_LABEL}}: N/A — this repo has no CI; manual validation against the diff text is the anchor (which I just did).
  • Placeholder-consistency: PASS — {{DO_NOT_AUTO_MERGE}} appears in template body + conditional + substitution-table row; no orphans.
  • Escaping: PASS — both new fenced code blocks (one in Phase 3 line 194, one in Phase 8 line 334) use \``` (leading backslash on each triple-backtick), confirmed by reading the surrounding lines.

Adversarial probe — 'Is the 400 LOC ceiling arbitrary?' Considered: yes it is — RESEARCH.md §2 cites 'agent PRs that are larger are unmerged more often' but doesn't give a specific threshold. The 400/800 numbers come from §6's reference to the SmartBear/Microsoft 200-400 LOC industry sweet spot, which is medium-confidence. Documented in the PR body that this is a defensible heuristic, not a measured threshold. Followup-issue tracking the empirical calibration could be filed but the current numbers are reasonable starting points.

@dmccoystephenson
dmccoystephenson merged commit a6b749e into main May 25, 2026
@dmccoystephenson
dmccoystephenson deleted the feature/pr-scope-gates branch May 25, 2026 18:47
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
…ity, test comment (#103)

- README.md: use ln -sf for the Installation symlink, matching Step 5's
  own registration convention (create-dev-loop.md:624).
- CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob
  deliberately excludes config.yml, which has no restatable conventions.
- tests/test_check_docs.py: note in the module docstring that any new
  test class must reset check_docs.errors in its own setUp(), so a
  future addition doesn't silently leak state between tests.

Not included, with reasons:
- RESEARCH.md's "Observed effect: pending — needs N cycles of data"
  entries (PRs #29/#30/#52/#54): this needs real usage data the
  maintainer would have to supply; nothing to mechanically edit here
  without fabricating an observed effect.
- The flagged openai.com RESEARCH.md link: re-verified via an
  independent fetch, still 403 (not 404) — consistent with bot/WAF
  blocking a non-browser client, not a confirmed dead link. Left as-is.
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
* Add hard PR scope gates: LOC ceiling, do-not-auto-merge paths

Closes #17

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Clean up DO_NOT_AUTO_MERGE substitution-table description

Nested backticks in the format example were ambiguous; describe the
format in prose instead of trying to inline-code a code-snippet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
…ity, test comment (#103)

- README.md: use ln -sf for the Installation symlink, matching Step 5's
  own registration convention (create-dev-loop.md:624).
- CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob
  deliberately excludes config.yml, which has no restatable conventions.
- tests/test_check_docs.py: note in the module docstring that any new
  test class must reset check_docs.errors in its own setUp(), so a
  future addition doesn't silently leak state between tests.

Not included, with reasons:
- RESEARCH.md's "Observed effect: pending — needs N cycles of data"
  entries (PRs #29/#30/#52/#54): this needs real usage data the
  maintainer would have to supply; nothing to mechanically edit here
  without fabricating an observed effect.
- The flagged openai.com RESEARCH.md link: re-verified via an
  independent fetch, still 403 (not 404) — consistent with bot/WAF
  blocking a non-browser client, not a confirmed dead link. Left as-is.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add hard PR scope gates: LOC ceiling, files-touched ceiling, do-not-auto-merge path list

1 participant