Skip to content

[SCR-181] Check an issue body against the standard - #190

Merged
itsryanarmani merged 1 commit into
mainfrom
scr-181-add-the-issue-standards-checker
Sep 2, 2026
Merged

itsryanarmani merged 1 commit into
mainfrom
scr-181-add-the-issue-standards-checker

Conversation

@itsryanarmani

@itsryanarmani itsryanarmani commented Sep 2, 2026 •

Copy link
Copy Markdown
Owner

Closes #181

What

Adds issue-standards precheck, the offline half of the checker: read an issue body, judge its shape against the standard, say what is wrong. No network, so a PreToolUse hook can run it before an issue exists. The API and template halves are #189.

Why

issue-standards.md landed in #179 and nothing enforced it. An agent that skipped Job to be done got no feedback until a human read the issue, which is exactly the attention that does not scale.

Three decisions worth review:

A placeholder counts as empty. The forms ship Today: / Wanted: and bare checkboxes as scaffolding. An issue submitting them unchanged is the most common way an issue looks filled in and is not, and an empty heading is worse than a missing one because it looks answered.

The parent-in-the-body rule does not depend on the kind. It is kept out of the per-kind path so it still fires on an issue carrying no kind label. That is not an edge case: 85 open issues across the fleet carry no labels at all, and content-rabbit#1134 has both no labels and a Parent: #1119 line.

Matching is deliberately loose. Headings normalise, so Today/Wanted and Today / Wanted are one heading, and ## inside a fenced block is not a heading. A human types what the form suggests, not what a regex wants, and rejecting honest work is the direction this check must not be wrong in. There is a test asserting prose that merely mentions a parent is not a false positive.

It checks shape and nothing else. Whether the job is real, whether the criteria cover it, and whether the size and route are honest are judgements a program cannot make.

How I verified

node issue-standards.test.mjs
  -> tests 19, pass 19, fail 0

One test per rule, each written to fail if the rule is deleted. Verified by deleting them: removing the placeholder rule drops it to 18 pass 1 fail, removing the native-link rule to 17 pass 2 fail.

Exit codes match pr-standards:

./issue-standards precheck --body-file complete-feature.md --kind feature   -> exit 0
./issue-standards precheck --body-file missing-heading.md  --kind feature   -> exit 1
./issue-standards precheck --body-file /tmp                --kind feature   -> exit 2
./issue-standards precheck --body-file complete.md         --kind nonsense  -> exit 2
./issue-standards nonsense                                                  -> exit 2

The directory case is a real bug the tests caught. existsSync passes on a directory and readFileSync then throws EISDIR deep inside, which surfaced as an ordinary failure rather than the configuration error it is. Fixed with statSync().isFile().

Against the real body of content-rabbit#1134, which is the issue that motivated the rule:

gh api repos/pooriaarab/content-rabbit/issues/1134 --jq .body   -> body saved
gh api repos/pooriaarab/content-rabbit/issues/1134 --jq .body > 1134.md
./issue-standards precheck --body-file 1134.md --kind chore
  -> 5 failure(s), exit 1. Among them:
     FAIL  parent or blocker in the body
           got:      Parent: #1119
           expected: a native sub-issue or issue dependency
     The other four are the spine headings it never had.

Size, against the caps:

git diff --stat origin/main
  -> 3 files changed, 491 insertions(+)

On the size

The first draft was 741 counted lines and failed the 500 cap. The standard says the cap forces decomposition before the work is written, and I did not decompose, so it caught me rather than the other way round. Split into this and #189 rather than asking for an exception, because there is no label that clears the cap and no one to ask for one.

Assisted-by: claude-personal-1:claude-opus-5

The issue standard was documentation. Nothing checked it, so an agent
that skipped Job to be done got no feedback until a human read the
issue, which is the attention that does not scale.

Add the offline half: read a body, judge its shape, say what is wrong.
No network, so a PreToolUse hook can run it before an issue exists.
The API and template halves are #189.

A heading whose body still holds the form placeholder counts as empty.
That is the most common way an issue looks filled in and is not, and
an empty heading is worse than a missing one because it looks
answered.

The parent-in-the-body rule is kept separate from the per-kind rules
because it does not depend on a kind. An issue carrying no kind label
is exactly the unlabelled backlog this standard exists to clean up,
and the rule still has to fire there.

Headings match loosely and fenced blocks are skipped. A human types
what the form suggests, not what a regex wants, and rejecting honest
work is the direction this check must not be wrong in.
@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 48 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 132 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: bef99394-b667-4590-80f8-d20b0d183ed3

📥 Commits

Reviewing files that changed from the base of the PR and between c960fa9 and d60443b.

📒 Files selected for processing (3)
  • issue-standards
  • issue-standards.mjs
  • issue-standards.test.mjs

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Council reviewed this PR (correctness / performance / security / maintainability / scope lenses). I verified every claim against the code myself before acting on it. No fixes pushed — see the size-cap note below for why.

Confirmed, open (correctness lens)

Major — validateNativeLinks matches inside fenced code (issue-standards.mjs:94). NATIVE_LINK_RE runs against the raw body, not the fence-stripped body. An issue that shows Parent: #1119 as a documentation example inside a triple-backtick block fails the check even though it isn't a real parent reference. This contradicts the PR's own stated design principle that rejecting honest work is the direction this check must not be wrong in — it's the same class of bug the existing prose test guards against, just via a code fence instead of a sentence.

Major — mismatched fence delimiters end a block early (issue-standards.mjs:55). fenced = !fenced toggles on any triple-backtick or ~~~ line regardless of which one opened the block. A ~~~ block containing a nested triple-backtick example (common when documenting markdown itself) closes prematurely, so a ##-looking line inside the example is read as a real heading — which can flip FORBIDDEN_HEADINGS/REQUIRED_HEADINGS results on content that was never meant to be parsed.

Minor — acceptance-criteria checkbox check is under-strict (issue-standards.mjs:151). CHECKBOX_RE only requires the - [ ] marker to appear somewhere in the section, not that it has content. Two independent ways to fake compliance: a bare - [ ] marker with no text alongside unrelated real prose in the same section, and a - [ ] shown only as a fenced-code example rather than a real bullet — both currently pass.

Minor — stray positional args are silently accepted (parseArgs, issue-standards.mjs around line 229). precheck --body-file F --kind feature typo collects typo into options.positional and never checks it, so a malformed invocation exits 0/1 as if well-formed instead of failing as a configuration error.

All four are real, each with a concrete reproduction, and I drafted fixes plus regression tests for all of them: fence-delimiter tracking, a shared fence-stripping helper reused by the native-link and checkbox checks, a tightened checkbox regex, and a positional-arg guard.

Why nothing was pushed

Those fixes total +96/-7 lines (git diff --numstat, verified locally). This PR is already at 491 of the 500 counted-line cap, confirmed against the PR's own How I verified section (git diff --stat origin/main reports 491 insertions). Pushing the fix would land at roughly 594 counted lines, over the fleet's own pr-standards cap that this repo enforces on every other PR. I don't get to spend the PR's remaining budget on my own follow-on work, so I reverted the patch rather than push it.

Recommended split: a single follow-up PR against issue-standards.mjs and issue-standards.test.mjs, something like Harden issue-standards against fenced-code edge cases, carrying all four fixes above together since they share one root cause (fenced-block handling and validation-regex strictness), landing after this PR and #189. It's a self-contained, single-concern change that comfortably fits under the 500-line cap on its own (roughly 103 lines including tests).

PR-standard check

Branch, title, Closes #181, the What/Why/How I verified sections, the Assisted-by trailer, and size/file caps (491 lines, 3 files) all conform. Proof is solid: the verification section names real commands with real results (test run counts, deliberate rule-deletion checks, an exit-code matrix, and a check against the real content-rabbit#1134 body that motivated the rule) — evidence matches every code path this diff changes.

🧑‍⚖️ Council
Model Lens Result
GPT-5.6 (Codex) (via OpenRouter) correctness confirmed (4 findings, de-duped from 5 raw claims; not fixed — would exceed the 500-line cap)
Gemini 3 Pro performance no findings (both gemini and openrouter routes failed/timed out)
Kimi K3 (via OpenRouter) security no findings
Grok 4.5 maintainability no findings (timed out)
GPT-5.6 (scope) scope no findings

Comment thread issue-standards.mjs
// this standard exists to clean up.
export function validateNativeLinks(body) {
const failures = [];
const match = NATIVE_LINK_RE.exec(String(body || ''));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major. NATIVE_LINK_RE.exec(String(body || '')) runs against the raw body, so a Parent: #1119 shown inside a fenced code example (documenting the convention, not using it) fails the check as a real parent reference. Same class of false positive the existing R4 prose test guards against, just via a code fence instead of a sentence. Fix: strip fenced blocks before matching, the same way heading parsing already does.

Comment thread issue-standards.mjs
let current = null;
let fenced = false;
for (const line of String(body || '').split('\n')) {
if (/^\s*(```|~~~)/.test(line)) fenced = !fenced;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major. fenced = !fenced toggles on any triple-backtick or ~~~ line, regardless of which delimiter opened the block. A ~~~-fenced example that itself contains a nested triple-backtick line (e.g. documenting markdown syntax) closes the outer fence early, so a ##-looking line still inside the example gets parsed as a real heading. Fix: track the opening delimiter's character and length, and only close on a matching (or longer) one of the same character.

@itsryanarmani
itsryanarmani merged commit f110f34 into main Sep 2, 2026
5 checks passed
@itsryanarmani
itsryanarmani deleted the scr-181-add-the-issue-standards-checker branch September 2, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check an issue body against the standard

1 participant