Repository navigation
Description
Problem
The "How to use" section of RESEARCH.md requires citing findings forward when proposing changes, but it does not require recording outcomes backward when a change ships. This means:
- We can't tell from
RESEARCH.mdwhich findings have been implemented vs. which are still pending. - When a research-backed change ships, we don't capture whether it worked in practice — we have the prediction (expected impact from the literature) but no record of the observed effect.
- Future contributors can't distinguish "this finding hasn't been acted on yet" from "this finding was acted on and it didn't help / was reverted."
Motivation
The whole point of grounding work in research is to learn from outcomes. Without an outcome log, we're consuming the literature but not contributing back to our own understanding. Over a few template-promotion cycles, this becomes the difference between empirically-informed and empirically-grounded design.
Connected to RESEARCH.md §8 — drift: without outcome tracking, we can't tell when a prior finding has been superseded by what we've actually observed in our own loops.
Confidence: low (this is a process improvement, not an empirical claim — but the lack of any back-pressure mechanism is visible).
Proposal
Extend each finding section in RESEARCH.md with an optional "Implementations" subsection:
**Implementations.**
- PR #15 (rubric-based self-review): shipped 2026-05-25. Observed effect: [pending — needs N cycles of data].
- PR #X: ...When PR #15-style work ships, the same PR (or a follow-up) adds an entry under the relevant finding's "Implementations" subsection. Initially the observed-effect column reads "pending"; after a few cycles, the entry is updated based on observed behavior (rate of caught issues, false positives, etc.).
The CLAUDE.md "Grounding work in research" section gets a new bullet:
If your PR implements a finding from RESEARCH.md, also add an "Implementations" entry under that finding with the PR number and a placeholder "observed effect: pending".
Acceptance criteria
- RESEARCH.md "How to use" section adds the implementations-tracking convention
- CLAUDE.md "Grounding work in research" section references it
- Existing findings get retroactive "Implementations" entries where applicable (§1, §5 for PR Reframe self-review as rubric-based and CI-grounded #15; Scope create-dev-loop to also update existing dev loops #16 once that update-mode ships; etc.)
- At least one updated outcome entry (i.e., not just "pending") exists after the first measurable cycle
Open question
Where do "observed effects" actually get measured? Phase 9 self-audit is the natural place if #24 lands (structured self-audit), but until then this risks being aspirational.
Related
- RESEARCH.md (the document being extended)
- Issue Structure Phase 9 self-audit prompts and label resulting gap issues #24 (structured self-audit would provide the data source)
- PR Add RESEARCH.md to ground design in empirical findings #14 (which created RESEARCH.md without outcome tracking)
drafted by Claude on behalf of Daniel Stephenson
Activity
Track implementation outcomes back into RESEARCH.md (#27)
Track implementation outcomes back into RESEARCH.md (#27)