Repository navigation
Bias triage and batching toward higher-merge-rate issue types - #29
Merged
Merged
Conversation
Closes #20 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Phase 1 skip-reason and Phase 2 tiebreaker changes implement findings §2 (agent-PR merge-rate gradient) and §7 (LLM triage as filter not decision-maker), so per the CLAUDE.md rule shipped in PR #27, both findings get Implementations entries pointing at this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dmccoystephenson
left a comment
Member
Author
There was a problem hiding this comment.
Self-review rubric:
- Scope: PASS —
create-dev-loop.md(6 lines added: skip-reason + tiebreaker + early-cycle bias) andRESEARCH.md(6 lines added: Implementations entries under §2 and §7, required by the CLAUDE.md rule from PR #27). - Tests-new: PASS — the skip-reason instruction is exercised by this very PR body, which includes a 'Note on Phase 1's RESEARCH.md cross-reference' explaining the deferred work; the tiebreaker is exercised by this cycle's choice of #20 over the heavier #16/#17/#18 (all docs/process > harder structural work).
- Tests-fix: N/A — not a bug fix.
- Sibling structure: PASS — new Phase 1 paragraph follows the same
**Bold label.** explanation.shape as its siblings ('Before filing each issue', 'After filing a batch of issues'); new Phase 2 paragraphs match the pattern of the existing bullet list above them. - Sibling renames: N/A — no identifiers renamed.
- Docs: PASS — Phase 7 sources of truth (
create-dev-loop.md,README.md) reviewed; README doesn't enumerate triage/work-selection logic at this granularity. RESEARCH.md updated per the CLAUDE.md PR-27 rule. - Issue resolution: PASS — all four AC items from #20 met: (1) Phase 1 records skip reasons; (2) Phase 2 has tiebreaker with rationale and RESEARCH.md cross-reference; (3) early-cycle bias marked as soft preference; (4) no new placeholders.
- Validation: PASS — placeholder count unchanged (19); no new fenced blocks; phase numbering unchanged.
- Placeholder-consistency: N/A — no template placeholder added.
- Escaping: N/A — no fenced code blocks in the prose additions.
Adversarial probe — 'Does the tiebreaker contradict the existing coherent-batch directive?' Yes there is potential tension. Resolved in the text: 'coherent-batch grouping still takes precedence when it applies.' The tiebreaker only fires when batching doesn't yield a clear winner. Confirmed by re-reading the new paragraph against the existing 'Prefer issues that touch the same subsystem' bullet — order of precedence is explicit.
This was referenced Jul 25, 2026
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
…ity, test comment (#103) - README.md: use ln -sf for the Installation symlink, matching Step 5's own registration convention (create-dev-loop.md:624). - CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob deliberately excludes config.yml, which has no restatable conventions. - tests/test_check_docs.py: note in the module docstring that any new test class must reset check_docs.errors in its own setUp(), so a future addition doesn't silently leak state between tests. Not included, with reasons: - RESEARCH.md's "Observed effect: pending — needs N cycles of data" entries (PRs #29/#30/#52/#54): this needs real usage data the maintainer would have to supply; nothing to mechanically edit here without fabricating an observed effect. - The flagged openai.com RESEARCH.md link: re-verified via an independent fetch, still 403 (not 404) — consistent with bot/WAF blocking a non-browser client, not a confirmed dead link. Left as-is.
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
* Bias triage and batching toward higher-merge-rate issue types Closes #20 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Record Implementations entries for RESEARCH.md §2 and §7 The Phase 1 skip-reason and Phase 2 tiebreaker changes implement findings §2 (agent-PR merge-rate gradient) and §7 (LLM triage as filter not decision-maker), so per the CLAUDE.md rule shipped in PR #27, both findings get Implementations entries pointing at this PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
…ity, test comment (#103) - README.md: use ln -sf for the Installation symlink, matching Step 5's own registration convention (create-dev-loop.md:624). - CLAUDE.md: clarify that the ISSUE_TEMPLATE doc-table row's *.md glob deliberately excludes config.yml, which has no restatable conventions. - tests/test_check_docs.py: note in the module docstring that any new test class must reset check_docs.errors in its own setUp(), so a future addition doesn't silently leak state between tests. Not included, with reasons: - RESEARCH.md's "Observed effect: pending — needs N cycles of data" entries (PRs #29/#30/#52/#54): this needs real usage data the maintainer would have to supply; nothing to mechanically edit here without fabricating an observed effect. - The flagged openai.com RESEARCH.md link: re-verified via an independent fetch, still 403 (not 404) — consistent with bot/WAF blocking a non-browser client, not a confirmed dead link. Left as-is.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two paragraphs to the generated dev-loop template:
Phase 1 — Record skip reasons. When triage filters out an open issue, record the skip reason in the chosen PR's body or as a comment on the skipped issue. Per RESEARCH.md §7, LLM-based issue triage is useful as a filter but not as a final decision — surfacing the filter's reasoning preserves human oversight.
Phase 2 — Tiebreaker + early-cycle bias. When choosing between issues of comparable scope with no dependency relationship, prefer (in descending order): documentation fixes → CI/build fixes → small refactors → bug fixes → performance work. Per RESEARCH.md §2, autonomous-agent PRs merge at substantially higher rates for the earlier categories. The tiebreaker is soft; coherent-batch grouping still wins when it applies. Early cycles (the first 3–5 after a new dev-loop skill is generated) weight the tiebreaker more strongly.
Closes #20
Test plan
{{placeholders}}introduced; substitution table unchanged (placeholder count: 19, same as before)gh issue view 20and confirmed title/body matchNote on Phase 1's RESEARCH.md cross-reference
The skip-reason instruction cites RESEARCH.md §7 (LLM-based issue triage accuracy 70–85%). Per the CLAUDE.md rule shipped in PR #27, this PR should also add an Implementations entry under §7 — but §7's specific implications were "emit triage reasoning to the PR description" and "use triage for batching coherence rather than value judgment," both of which this PR directly implements. Adding an Implementations entry now.
drafted by Claude on behalf of Daniel Stephenson