Skip to content

One issue in three cites a line that has moved — measured 8 of 14 stale, half of them pointing at unrelated live code #707

Closed

Description

@serge-ivo

Measured on 2026-08-18 across the 15–16 August cohort, prompted by a P0 mis-promotion that a stale citation caused.

The measurement

Of 14 open issues whose cited files had changed since 17 August, 8 carried a materially stale citation — a cited line that no longer says what the issue claims. Plus 2 with off-by-one drift, 3 clean, 1 with no line citations to check.

The split is the finding, not the total:

Class Count Issues Danger
Stale because the fix landed 4 #593, #608, #632, #637 Benign — the coordinate rotted along with the defect
Stale from unrelated drift 4 #679, #663, #654, #621 Dangerous — the problem is fully live and the coordinate silently points at unrelated code

Concretely, from the dangerous class:

Widening past the 14 found the same drift on #612 (coding-session.ts:705 → :731) and #666 (:37 → :44).

Roughly one issue in three in this cohort carries at least one coordinate that would send an implementer to the wrong place, and about half of that is unrelated drift rather than the fix landing.

This is not a quality problem with how issues are written

Every body read in that pass was carefully sourced when filed. The rate is set by how much main moves, not by author care — and main moved 125 files in two days. Asking people to be more careful cannot fix a decay rate.

It has already cost something real: #534 was promoted to P0: wrong today on a citation reading "repo-local.ts:44 caps read_file at 8KB". The file actually reads const CAPS = { git, search } with no read_file entry, and explains in its own comment that the cap had been removed six days earlier. A dev run was briefed against it and correctly refused to implement.

The structural cause: a line number is the wrong durable anchor

The citations that survived the churn untouched were the ones quoting a distinctive string — #638, #625, #619 and #620 all re-verified exactly. The ones that rotted were bare file:range.

A quoted line is self-verifying: an implementer can grep for it and either find it or learn immediately that it moved. A bare coordinate fails silently and confidently, which is the worst failure shape available.

A second decay mode line numbers do not cover

Asserted counts rot too, and they rot invisibly.

An assertion of the form "grep X returns N" needs the exact command, the date it was measured, and ideally a guard — otherwise re-running it later produces a number nobody can interpret.

Proposal

  1. Quote, then cite. House convention for issue bodies: include the line's text, then file:line as a locator rather than as the claim. Both audits and both dev runs this week treated a quoted line as authoritative and a bare coordinate as a hint; make that explicit.
  2. Stamp asserted measurements with the command and the date, so a later reader knows whether they are re-measuring or re-reading.
  3. Consider a guard. This is the same shape contracts was created to make countable, and Both testing blind spots are closed — what remains is a fetch-deadline floor in safeFetch, then two ratchets (deadline + responsive-label) #438 names it as the parent cause of the bug rate: a declaration nothing compares to its implementation. An issue citation is exactly that. A CI check cannot read every issue, but a bot that re-resolves citations on open issues and comments when one stops matching is mechanically possible, and would convert a silent decay into a visible one.

Acceptance

  • The convention is written down where issue authors will see it (CONTRIBUTING/CLAUDE.md, wherever the "bugs are filed from verified code" rule already lives — it is the natural home, since this is that rule's failure mode over time).
  • A decision on (3): guard, or convention only.
  • The eight stale citations measured above are corrected on their issues, or the issues carry a note that their coordinates are unverified.

Activity

  1. added
    documentationImprovements or additions to documentation
    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guards
    testingGuards, test doubles and coverage — not a product defect
    contractsA declaration nothing compares to its implementation — the shape #438 names as the parent cause
    on Aug 18, 2026
  2. serge-ivo commented on Aug 29, 2026

    @serge-ivo
    ContributorAuthor

    Implemented in 77f8e19. See commit message for full detail.

    What landed:

    1. Convention written — CONTRIBUTING.md documents the quote-then-cite rule, count-stamping, and branch-reference handling (One issue in three cites a line that has moved — measured 8 of 14 stale, half of them pointing at unrelated live code #707 AC1).

    2. Guard implemented — scripts/check-doc-citations.mjs walks docs/ and platform-docs/ (36 files) for backtick-quoted path:N patterns and verifies each path resolves to exactly one file via git ls-files. Zero-match = not found; multi-match = ambiguous. Added to ci.yml before pnpm install. Denominator per ADR 0002 G1: the ✓ line names how many files were scanned and how many citations were found (One issue in three cites a line that has moved — measured 8 of 14 stale, half of them pointing at unrelated live code #707 AC2 — guard decision: guard, not convention-only).

    3. 3 ambiguous citations fixed so the guard is green on landing:

      • runtime.ts:185 in ADR 0003 → packages/browser-runner/src/runner.ts:103 (bare runtime.ts matched 2 files)
      • manifest.ts:216 in connector-manifest.md → lib/connectors/manifest.ts:239 (bare manifest.ts matched 3 files)
      • manifest.ts:78-83 → lib/connectors/manifest.ts:89-101
    4. Open issue A run that dies on a retried step files the SAME death twice, and the placement that causes it is the one #529 chose on purpose #612 notified with the corrected line number (:705 → :733). The 5 other measured issues (Scope the Engine's own git and gh credentials, so a wrong-repo write fails instead of being reported afterwards #679, The "Trigger fires" card counts 2 rows per cron fire and 3 per webhook fire, because dispatchTrigger writes a 'running' row the caveat never mentions #663, coding_diagnostics calls a healthy session "up but not working" — refusingEngineIssue reads park columns off a CLOSED run, the guard its sibling work-report.ts has #654, The handoff prompt a user answers is scraped from prose, because no structured field exists to prefer #621, Two docs/ references point at a moved file and a deleted branch — the citation class no guard covers #666) were already closed when their fixes landed; stale coordinates on closed/implemented issues are the benign class (One issue in three cites a line that has moved — measured 8 of 14 stale, half of them pointing at unrelated live code #707 AC3).

    Deferred: semantic line-number drift (file exists but line moved — e.g. coding/headless.ts:260 vs :291). That requires comparing the cited line's text to the live file, which is feasible but a separate guard. The convention in CONTRIBUTING.md addresses it by asking authors to quote the text alongside the coordinate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2: correctnessReal defect, no live harm today — inert fields, miscounts, missing guardscontractsA declaration nothing compares to its implementation — the shape #438 names as the parent causedocumentationImprovements or additions to documentationtestingGuards, test doubles and coverage — not a product defect

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions