Skip to content

shepard: leave the code simpler, and think the fix through before pushing - #45

Merged
jasperboerhof merged 2 commits into
mainfrom
lift/shepard-bug-class
Sep 11, 2026
Merged

jasperboerhof merged 2 commits into
mainfrom
lift/shepard-bug-class

Conversation

@jasperboerhof

@jasperboerhof jasperboerhof commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Why

We measured the patching problem before settling on a change. Four auditors replayed 13 long review chains in kendo, emmie, lokalekeuze and crit: 144 crit rounds, and 230 findings filed in round 2 or later. They tested a set of stop rules against them: a "same bug class at a second site" stop, a blame check, a per-file round counter, and a sweep for twins.

The stop rules would have saved about 7–30 of the 144 rounds, and they fired falsely about half the time. Detection was not the gap. In every repo, the old process already stopped at about the same rounds. What decided the outcome was the fix that came next:

  • About half the later findings (111 of 230) sat inside the previous fix. That fix had added a flag, a guard, a lock or a mode, and the new state had holes of its own.
  • The chains ended when a fix deleted something: one owner for a piece of state instead of two, a field taken out of a payload, a construct replaced, or one decision recorded for a whole class.
  • Design calls that chose to add state kept the chains going.

So this PR drops the stop rules and changes only how a fix is made.

What

skills/shepard/SKILL.md, steps 4 and 5 only:

  1. "Make the minimal fix" becomes "Leave the code simpler, not the diff smaller." When a fix adds a flag, guard, lock or mode, look for the version that does not need it. Unrelated refactoring still stays out of the diff.
  2. New: "Think the fix through before you push it." Three questions about the fix itself: what new states or values it adds, what happens when the call fails, overlaps, retries or runs twice, and who else reads what it changed. Pin the failure path with a test. A rushed fix costs a whole review round.
  3. Step 5: between fix shapes, recommend the one that leaves less state.

The branch's first commit added the stop rules, a Class: commit line and a Hazards section in the template. The second commit reverts all of that. The net diff against main is only the three points above.

Not in this PR

  • PR size. In the same data it is the strongest predictor of long chains: PRs under 200 lines averaged 1.4 crit rounds, PRs over 2,500 lines averaged 6. That belongs in the build and plan steps, not in /shepard.
  • Late discovery. Many extra rounds came from the reviewer finding old code late. No author-side rule fixes that.
  • Drift. The catalog and personal copies still differ elsewhere, in both directions, mostly in the scripts. A resync is its own PR.
  • Mirror. lokalekeuze carries a copy of this skill in .claude/skills/shepard. It needs the same change after this merges.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AiJB5Aakjd3rGcm6jdNf3y

The stop rule was "three rounds on the same seam", judged by the agent
that sees every finding as new. On a corpus of 446 review findings it
never fired while PRs ran 8, 9 and 12 rounds. Over half the findings
filed in round 2 or later sat on a file that already had a finding.

Now the skill counts instead of judging:
- Step 1 lists the bug classes already fixed, from `Class:` lines in
  the branch's commits. A class at its second site is a design call.
- Step 3 names the bug class of every finding and sweeps the diff for
  twins.
- Step 4 fixes every twin in the diff in one push. A fix that adds
  state carries a transition table.
- Step 5 adds a third option: fix the construct in this PR. It must
  shrink the state space.
- Step 7 adds a `Class:` line to every review-fix commit.

The reference template gets a Hazards section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AiJB5Aakjd3rGcm6jdNf3y
@jasperboerhof jasperboerhof added the Agent Review Requested Ask an agent to review this PR label Sep 11, 2026
A replay of 13 long review chains (kendo, emmie, lokalekeuze, crit)
showed the bug-class stop rules would save about 7-30 of 144 rounds,
and fire falsely about half the time. The old process already stopped
at the same rounds. What kept the chains going was the next fix:
about half the later findings sat inside the previous fix.

This reverts the previous commit and keeps two rules in step 4:
- Leave the code simpler, not the diff smaller.
- Think the fix through before you push it.

Step 5 now recommends the fix shape that leaves less state.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AiJB5Aakjd3rGcm6jdNf3y
@jasperboerhof jasperboerhof changed the title shepard: stop at a bug class's second site, fix the construct shepard: leave the code simpler, and think the fix through before pushing Sep 11, 2026
@jasperboerhof
jasperboerhof merged commit bb275c7 into main Sep 11, 2026
bjvle added a commit that referenced this pull request Oct 2, 2026
$.state is the session's, and a /clear starts a new session, so the
marker session.end wrote was gone at the next prompt: the first live
/clear delivered nothing (finding #45). The marker now lives in $.store
under a random key each fresh process mints in session.start and keeps
in its own environment, which outlasts a /clear and a hot reload.
$.state tells a fresh process (a child claude inherits the env) from a
reload. prompt.submit now writes a debug line either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bjvle added a commit that referenced this pull request Oct 2, 2026
…arker

The handoff reached the first prompt after a live /clear: session.end
stored lastClear:<key>, prompt.submit delivered it with a GATE line, and
the handoff is now consumed. Finding #45 gets the retest, D32 and the
CHANGELOG drop it from the unmeasured list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Agent Review Requested Ask an agent to review this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant