Skip to content

Validate Triage categories in the correction turn and give Report Triage the repository categories - #2139

Merged
JacobStephens2 merged 4 commits into
mainfrom
issue-2110
Nov 12, 2026
Merged

JacobStephens2 merged 4 commits into
mainfrom
issue-2110

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

Both Triage modes now read the repository categories before the session starts and put them in the prompt. A proposed issue with an invalid category now gets the one correction turn, instead of failing later in check_publication.

 investigate
+  categories = repository_categories(repo)          # Report and existing issue
-  prompt::triage(skill, report)                     # Report: no categories
+  prompt::triage(skill, report, categories)         # "Available repository categories (JSON): …"
   prompt::triage_issue(skill, issue, categories)    # now inherits that line from prompt::triage
-  correction::recommendation(sessions, prompt, target)
+  correction::recommendation(sessions, prompt, target, categories)
     validate
       read_recommendation
+      category error for any proposed issue (issue_category)
       resolve_recommendation + existing::check_recommendation
+      both errors → one message, so the correction turn hears about both
-  check_publication(rec, repo, github)                # fetched labels again
+  check_publication(rec, repo, categories, github)    # same issue_category, canonical spelling kept

Confirmed cause: correction::validate checked only parse_recommendation and existing::check_recommendation. The category check ran only in check_publication, after the correction loop had finished. Report Triage also never received the categories it was checked against.

Docs updated: docs/guide.md (and the regenerated manual), skills/thirdshift-triage/SKILL.md, the regenerated prompts, and the hand-written pages site/index.html, site/how-it-works/ and site/architecture/. They now say Report sessions receive the categories too, and that an invalid category gets the correction turn.

Evidence

Execution evidence

  • Before (commit e802b10 plus only the new test):
    Command: cargo test --test triage_correction a_factory_or_unknown_category_gets_one_correction_in_both_modes

    Recommendation: security-finding (ready-for-agent)
    ...
    : the Triage category must be a repository category label, not a triage state or another factory label
      left: Some(1)
     right: Some(0)
    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 11 filtered out
    

    The Report run exited 1 straight from check_publication, with no TriageCorrection turn.

  • After: cargo test --test triage_correction:

    test result: ok. 12 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out
    

    This includes:

    • a_factory_or_unknown_category_gets_one_correction_in_both_modes: Report and existing-issue modes × security-finding and an unknown label. Checks the categories prompt line, the exact correction error and the corrected category.
    • a_proposed_issue_category_override_gets_one_correction
    • a_bad_category_is_reported_alongside_another_validation_error
  • Other targeted runs:

    • cargo test --test triage_run --test triage_issue --test triage_spec_regressions: all pass.
    • cargo test --bin thirdshift triage: 26 passed.
    • cargo test prompts_page: 14 passed.
    • npm run docs:check --prefix site: passes.
  • scripts/affected-tests main selected the full suite and stopped at an intermittent cargo test --test process failure in findings:: Security finding Step tests:

    • merged_fix_is_rechecked_after_a_later_fix_fails
    • fixing_dispatches_each_reproduced_finding_and_keeps_unreproduced_findings_private

    Both also fail intermittently at this branch's base commit e802b10 with none of this change (2/6 and 1/6 runs). They don't touch Triage code.

  • A full cargo test --no-fail-fast after merging origin/main passed (exit 0).

Visual evidence

There are no visual or layout changes. The hand-written site pages change by one sentence each, so I made no screenshots.

Merge Danger

Door: two-way

Blast Radius: Triage

Report Triage now reads repository labels before its session. If listing labels fails, a Report run now stops before investigating, instead of after investigating; this matches existing-issue mode.

Unaddressed findings

Standards

  • 1. Docs promise that every recommended category is validated, including comment-only recommendations. I declined the code change and narrowed the wording instead:
    • docs/guide.md now says "A proposed issue whose category is not one of the repository categories…".
    • The src/prompt.rs doc comment now reads "Each proposed issue's category…".
    • The spec scopes the check to "when recommendation.issues is non-empty".
    • Run: I also validated the overall category for comment-only recommendations. With that, the reviewer's test a_comment_only_recommendation_with_a_factory_category_gets_one_correction passed, but 6 existing cargo test --test triage_run tests failed (for example triage_comments_on_a_covering_issue_without_filing_or_changing_labels, recommend_only_names_the_covering_issue_and_comment_without_github_writes). Those recommendations only comment on an existing issue and never use the category. I reverted it.
  • 3. Duplicated Code in investigate's two match arms (judgement). I declined. In the Issue arm, existing::check_trust and images::hand_over run before the label read. Hoisting the read would reorder the trust check. The repetition is one line per arm.
  • 4. Data Clumps: a Categories newtype (judgement). I declined. categories is a &[String] passed between three functions in one module. Finding 2's issue_category helper already gives the resolution rule a single home, so a newtype would be speculative.

Spec

None. All three findings were addressed:

  1. The category error is now reported alongside other validation errors.
  2. Per-issue override coverage was added.
  3. A "lookup" case covers a GitHub read failure after the session.

Security

None. The Security review of this change found no introduced finding: repository categories reach the prompt JSON-encoded from maintainer-owned labels with factory labels filtered, and the category allowlist check is unchanged and now also runs in the correction turn.

Security review outcome

No unaddressed introduced findings.

Review coverage

Reviewer reports cover commit 7224e0c. Every changed file was read by both axes. Generated outputs were excluded and checked by regeneration, with changed lines inspected:

  • prompts/triage*.md
  • site/prompts/index.html
  • site/docs/index.html

The follow-up commit dbaccc3 (review fixes) and the merge of origin/main came after the review. No changed file was left unread.

Closes #2110

Built with claude 2.1.314 · opus · default effort

@JacobStephens2
JacobStephens2 merged commit 943166d into main Nov 12, 2026
24 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-2110 branch November 12, 2026 18:55
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Triage category validation runs outside the correction turn, and report Triage omits repository categories

1 participant