Repository navigation
Add RESEARCH.md to ground design in empirical findings - #14
Merged
Merged
Conversation
Records the published research and benchmark results that inform the current design of create-dev-loop and the dev-loop skills it generates. CLAUDE.md now requires citing RESEARCH.md (or explicitly noting the absence of evidence) when proposing template or phase changes, so design decisions are anchored in the literature rather than intuition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closed
4 tasks
dmccoystephenson
added a commit
that referenced
this pull request
Aug 3, 2026
Records the published research and benchmark results that inform the current design of create-dev-loop and the dev-loop skills it generates. CLAUDE.md now requires citing RESEARCH.md (or explicitly noting the absence of evidence) when proposing template or phase changes, so design decisions are anchored in the literature rather than intuition. Co-authored-by: BuildTools <unconfigured@null.spigotmc.org> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RESEARCH.md— a curated record of empirical findings on autonomous software development agents, with citations, confidence levels, and the specific implication of each finding forcreate-dev-loopand the dev-loop skills it generates.CLAUDE.mdwith a new "Grounding work in research" section requiring future PRs to cite findings fromRESEARCH.md(or explicitly note absence of evidence) when proposing template or phase changes. Also addsRESEARCH.mdto the documentation sources-of-truth table.README.mdwith a brief "Research basis" section pointing outside readers toRESEARCH.md.The goal is to anchor design decisions in published research and reproducible benchmarks rather than intuition or vendor marketing.
What's in RESEARCH.md
Eight findings drawn from peer-reviewed and benchmark literature (2023–2026), covering:
Each finding includes a citation, confidence level (high/medium/low), and a specific implication for one or more phases of the dev-loop. The doc also calls out which existing design choices are already aligned with the evidence (e.g., the 0–3 issue batch ceiling, the retrofit checklist) and where the empirical record is thin (autonomous-merge defect rates, optimal phase decomposition).
A "how to add a new finding" template is included so future research can be incorporated consistently.
Test plan
RESEARCH.mdresolve to valid arXiv/venue/blog URLsCLAUDE.mdsources-of-truth table renders correctly and the new section is clearly wordedREADME.md"Research basis" link toRESEARCH.mdresolves on GitHub🤖 Generated with Claude Code
drafted by Claude on behalf of Daniel Stephenson