Skip to content

[SCR-189] Check a filed issue and lint the forms - #191

Closed
itsryanarmani wants to merge 1 commit into
scr-181-add-the-issue-standards-checkerfrom
scr-189-check-a-filed-issue
Closed

itsryanarmani wants to merge 1 commit into
scr-181-add-the-issue-standards-checkerfrom
scr-189-check-a-filed-issue

Conversation

@itsryanarmani

Copy link
Copy Markdown
Owner

Closes #189

What

Adds check, lint and --selfcheck: the half of the checker that needs the GitHub API and the template files on disk. Stacked on #190, which is the offline body check.

Why

#190 can judge a body you already have. Nothing could fetch a filed issue, validate its labels, or notice that the forms and the checker had drifted apart.

Labels carry classification because GitHub issue types are an organisation feature and this is a user account. Exactly one label from each of four groups: kind, size, route, state. Two from one group is a contradiction, not extra information.

An issue with no kind label is still checked. There is nothing to validate the headings against, so it says so rather than guessing, but the rules that do not depend on a kind still run. That is not an edge case: 85 open issues across the fleet carry no labels at all.

lint exists because the forms are what people actually fill in. If a form stops offering a heading the checker demands, an issue filed through that form fails the check the form exists to satisfy. The two must not drift.

How I verified

node issue-standards.test.mjs
  -> tests 24, pass 24, fail 0        19 before, 5 added here

./issue-standards --selfcheck
  -> exit 0, 11 rules asserted to fire

Against content-rabbit issue 1134, the unlabelled issue with a parent written in prose that motivated both rules:

./issue-standards check  (that issue)
  -> exit 1
     FAIL  kind label       got: none
     FAIL  size label       got: none
     FAIL  route label      got: none
     FAIL  state label      got: none
     FAIL  parent or blocker in the body   got: Parent line naming 1119

The parent line is the one that matters. It is found even though the issue carries no kind label, which is the case the rule was separated out for.

Against the forms merged in content-rabbit 1205:

./issue-standards lint --dir .github/ISSUE_TEMPLATE
  -> exit 0, PASS bug.yml feature.yml chore.yml epic.yml

The shipped forms and this checker agree, which is the property lint exists to hold.

Size, against the caps:

git diff --stat
  -> 2 files changed, 258 insertions(+), 8 deletions(-)

Note on how this was built

Delegated twice, failed twice, so I wrote it. Codex hit its usage limit and exited with no output and an untouched worktree. Two pi workers died at exit 144 on long generation jobs, while short scoped tasks on the same routes succeeded the same hour. Both outages are recorded in agent-routing.json via #188, and the lesson is in the routing note: judge a delegated job on its diff, never on its exit code.

Assisted-by: claude-personal-1:claude-opus-5

The offline body check landed in #181. This is the half that needs the
GitHub API and the template files on disk.

check fetches an issue, validates its labels, and validates its body
against the kind its label declares. Labels carry classification
because issue types are an organisation feature and this account is a
user, so /orgs/pooriaarab/issue-types returns 404. Exactly one label
from each of four groups: two from one group is a contradiction, not
extra information.

Without a kind label there is nothing to check the headings against.
Say so rather than guessing, but still run the rules that do not need
a kind. That is not an edge case: 85 open issues across the fleet
carry no labels at all, and content-rabbit#1134 has none plus a
Parent: line in its body.

lint asserts the forms still offer every heading the checker demands.
Without it the two drift and an issue filed through the form fails the
check the form exists to satisfy.

selfcheck asserts each rule fires on a body built to break it, so a
rule silently deleted shows up there rather than in a pull request
that should have been refused.
@coderabbitai

coderabbitai Bot commented Sep 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 272fd0ae-cdbb-4886-aed3-fa7e25eb3392

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@itsryanarmani
itsryanarmani deleted the branch scr-181-add-the-issue-standards-checker September 2, 2026 05:04

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Council-driven review + fixes

Read pr.diff and the full issue-standards.mjs/issue-standards.test.mjs against council-findings.md. All four correctness claims from GPT-5.6 (Codex) checked out against the actual code; the other three lenses returned no findings or timed out (nothing to verify there). I confirmed and fixed all four:

  1. fetchIssue accepted pull requests as issues (Major). GitHub's repos/{owner}/{repo}/issues/{number} endpoint returns PRs too (a PR is an issue with a pull_request key), so check --repo o/n --number <PR#> would silently validate a PR's labels/body as though it were a filed issue. Fixed: fetchIssue now throws ApiError when issue.pull_request is present.

  2. --selfcheck silently swallowed other arguments (Major). parseArgs matched --selfcheck anywhere in argv and returned immediately, so e.g. check --selfcheck --repo bogus ran selfcheck and silently discarded check/--repo bogus instead of erroring — exactly the "say so rather than guessing" failure mode this tool is built to avoid elsewhere. Fixed: --selfcheck now must be the only argument (besides --json), otherwise it's a ConfigurationError.

  3. lintTemplates matched heading text anywhere in the raw YAML, not just in the field's own label (Major). A required heading's words surviving elsewhere in the file (a description, a comment, a different field) after the actual field was renamed/removed would make lint report a false pass — defeating the exact drift lint exists to catch. Fixed: now extracts only label: lines and matches against those.

  4. Unexpected errors were misreported as "GitHub API" failures (Minor). main()'s catch-all classified anything that wasn't a ConfigurationError as a GitHub API error by name check, so e.g. a filesystem error inside lintTemplates/runPrecheck would print a misleading "check gh authentication" message. Fixed: classify by instanceof ApiError explicitly, with a generic "unexpected error" bucket for anything else (still exits 1, same as before).

All four fixes are pushed as 548f295, stacked cleanly on the current branch tip (the branch was force-updated/rebased upstream mid-review; I rebased my fix by cherry-picking rather than replaying, to avoid re-litigating already-merged history). Tests added for fixes 2 and 3 (fix 1 needs a live gh call / PR fixture to exercise and isn't mockable with this repo's existing test harness, so it's unit-untested like the rest of fetchIssue). I could not execute node issue-standards.test.mjs myself in this sandboxed environment (command execution was denied), so I traced the new and existing tests by hand against the updated code instead of getting a live pass/fail — flagging that as the residual gap in my own verification, not the PR's.

PR-standard check: branch/title/body match the scr-<issue>-<slug> / [SCR-189] <subject> / Closes #189 + ## What/## Why/## How I verified + Assisted-by: shape. Diff is 295 counted lines across 2 files (including my fix commit) — well under the 500-line/40-file caps.

Proof lens: ## How I verified names real commands (node issue-standards.test.mjs, ./issue-standards --selfcheck, check/lint against real content-rabbit issues/forms, git diff --stat for size) with results attached, and those results align with the diff's new surface (check, lint, --selfcheck). No gaps found there.

No Critical severity issues remain open.

🧑‍⚖️ Council
Model Lens Result
GPT-5.6 (Codex) via OpenRouter correctness confirmed (4/4, all fixed)
Gemini 3 Pro performance no findings (timed out / 403)
Kimi K3 via OpenRouter security no findings
Grok 4.5 maintainability no findings (timed out)
GPT-5.6 (scope) scope no findings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant