Repository navigation
[SCR-181] Check an issue body against the standard - #190
Conversation
The issue standard was documentation. Nothing checked it, so an agent that skipped Job to be done got no feedback until a human read the issue, which is the attention that does not scale. Add the offline half: read a body, judge its shape, say what is wrong. No network, so a PreToolUse hook can run it before an issue exists. The API and template halves are #189. A heading whose body still holds the form placeholder counts as empty. That is the most common way an issue looks filled in and is not, and an empty heading is worse than a missing one because it looks answered. The parent-in-the-body rule is kept separate from the per-kind rules because it does not depend on a kind. An issue carrying no kind label is exactly the unlabelled backlog this standard exists to clean up, and the rule still has to fire there. Headings match loosely and fenced blocks are skipped. A human types what the form suggests, not what a regex wants, and rejecting honest work is the direction this check must not be wrong in.
|
Warning Review limit reachedNext included review available in 48 minutes. View limit detailsLimit details: You’ve used the included review currently available. Your 132 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (3)
Comment |
There was a problem hiding this comment.
Council reviewed this PR (correctness / performance / security / maintainability / scope lenses). I verified every claim against the code myself before acting on it. No fixes pushed — see the size-cap note below for why.
Confirmed, open (correctness lens)
Major — validateNativeLinks matches inside fenced code (issue-standards.mjs:94). NATIVE_LINK_RE runs against the raw body, not the fence-stripped body. An issue that shows Parent: #1119 as a documentation example inside a triple-backtick block fails the check even though it isn't a real parent reference. This contradicts the PR's own stated design principle that rejecting honest work is the direction this check must not be wrong in — it's the same class of bug the existing prose test guards against, just via a code fence instead of a sentence.
Major — mismatched fence delimiters end a block early (issue-standards.mjs:55). fenced = !fenced toggles on any triple-backtick or ~~~ line regardless of which one opened the block. A ~~~ block containing a nested triple-backtick example (common when documenting markdown itself) closes prematurely, so a ##-looking line inside the example is read as a real heading — which can flip FORBIDDEN_HEADINGS/REQUIRED_HEADINGS results on content that was never meant to be parsed.
Minor — acceptance-criteria checkbox check is under-strict (issue-standards.mjs:151). CHECKBOX_RE only requires the - [ ] marker to appear somewhere in the section, not that it has content. Two independent ways to fake compliance: a bare - [ ] marker with no text alongside unrelated real prose in the same section, and a - [ ] shown only as a fenced-code example rather than a real bullet — both currently pass.
Minor — stray positional args are silently accepted (parseArgs, issue-standards.mjs around line 229). precheck --body-file F --kind feature typo collects typo into options.positional and never checks it, so a malformed invocation exits 0/1 as if well-formed instead of failing as a configuration error.
All four are real, each with a concrete reproduction, and I drafted fixes plus regression tests for all of them: fence-delimiter tracking, a shared fence-stripping helper reused by the native-link and checkbox checks, a tightened checkbox regex, and a positional-arg guard.
Why nothing was pushed
Those fixes total +96/-7 lines (git diff --numstat, verified locally). This PR is already at 491 of the 500 counted-line cap, confirmed against the PR's own How I verified section (git diff --stat origin/main reports 491 insertions). Pushing the fix would land at roughly 594 counted lines, over the fleet's own pr-standards cap that this repo enforces on every other PR. I don't get to spend the PR's remaining budget on my own follow-on work, so I reverted the patch rather than push it.
Recommended split: a single follow-up PR against issue-standards.mjs and issue-standards.test.mjs, something like Harden issue-standards against fenced-code edge cases, carrying all four fixes above together since they share one root cause (fenced-block handling and validation-regex strictness), landing after this PR and #189. It's a self-contained, single-concern change that comfortably fits under the 500-line cap on its own (roughly 103 lines including tests).
PR-standard check
Branch, title, Closes #181, the What/Why/How I verified sections, the Assisted-by trailer, and size/file caps (491 lines, 3 files) all conform. Proof is solid: the verification section names real commands with real results (test run counts, deliberate rule-deletion checks, an exit-code matrix, and a check against the real content-rabbit#1134 body that motivated the rule) — evidence matches every code path this diff changes.
🧑⚖️ Council
| Model | Lens | Result |
|---|---|---|
| GPT-5.6 (Codex) (via OpenRouter) | correctness | confirmed (4 findings, de-duped from 5 raw claims; not fixed — would exceed the 500-line cap) |
| Gemini 3 Pro | performance | no findings (both gemini and openrouter routes failed/timed out) |
| Kimi K3 (via OpenRouter) | security | no findings |
| Grok 4.5 | maintainability | no findings (timed out) |
| GPT-5.6 (scope) | scope | no findings |
| // this standard exists to clean up. | ||
| export function validateNativeLinks(body) { | ||
| const failures = []; | ||
| const match = NATIVE_LINK_RE.exec(String(body || '')); |
There was a problem hiding this comment.
Major. NATIVE_LINK_RE.exec(String(body || '')) runs against the raw body, so a Parent: #1119 shown inside a fenced code example (documenting the convention, not using it) fails the check as a real parent reference. Same class of false positive the existing R4 prose test guards against, just via a code fence instead of a sentence. Fix: strip fenced blocks before matching, the same way heading parsing already does.
| let current = null; | ||
| let fenced = false; | ||
| for (const line of String(body || '').split('\n')) { | ||
| if (/^\s*(```|~~~)/.test(line)) fenced = !fenced; |
There was a problem hiding this comment.
Major. fenced = !fenced toggles on any triple-backtick or ~~~ line, regardless of which delimiter opened the block. A ~~~-fenced example that itself contains a nested triple-backtick line (e.g. documenting markdown syntax) closes the outer fence early, so a ##-looking line still inside the example gets parsed as a real heading. Fix: track the opening delimiter's character and length, and only close on a matching (or longer) one of the same character.
Closes #181
What
Adds
issue-standards precheck, the offline half of the checker: read an issue body, judge its shape against the standard, say what is wrong. No network, so a PreToolUse hook can run it before an issue exists. The API and template halves are #189.Why
issue-standards.mdlanded in #179 and nothing enforced it. An agent that skippedJob to be donegot no feedback until a human read the issue, which is exactly the attention that does not scale.Three decisions worth review:
A placeholder counts as empty. The forms ship
Today:/Wanted:and bare checkboxes as scaffolding. An issue submitting them unchanged is the most common way an issue looks filled in and is not, and an empty heading is worse than a missing one because it looks answered.The parent-in-the-body rule does not depend on the kind. It is kept out of the per-kind path so it still fires on an issue carrying no kind label. That is not an edge case: 85 open issues across the fleet carry no labels at all, and
content-rabbit#1134has both no labels and aParent: #1119line.Matching is deliberately loose. Headings normalise, so
Today/WantedandToday / Wantedare one heading, and##inside a fenced block is not a heading. A human types what the form suggests, not what a regex wants, and rejecting honest work is the direction this check must not be wrong in. There is a test asserting prose that merely mentions a parent is not a false positive.It checks shape and nothing else. Whether the job is real, whether the criteria cover it, and whether the size and route are honest are judgements a program cannot make.
How I verified
One test per rule, each written to fail if the rule is deleted. Verified by deleting them: removing the placeholder rule drops it to 18 pass 1 fail, removing the native-link rule to 17 pass 2 fail.
Exit codes match
pr-standards:The directory case is a real bug the tests caught.
existsSyncpasses on a directory andreadFileSyncthen throwsEISDIRdeep inside, which surfaced as an ordinary failure rather than the configuration error it is. Fixed withstatSync().isFile().Against the real body of
content-rabbit#1134, which is the issue that motivated the rule:Size, against the caps:
On the size
The first draft was 741 counted lines and failed the 500 cap. The standard says the cap forces decomposition before the work is written, and I did not decompose, so it caught me rather than the other way round. Split into this and #189 rather than asking for an exception, because there is no label that clears the cap and no one to ask for one.
Assisted-by: claude-personal-1:claude-opus-5