Skip to content

Fail a Security run before any session when Issues are off - #2133

Merged
JacobStephens2 merged 4 commits into
mainfrom
issue-2096
Nov 12, 2026
Merged

JacobStephens2 merged 4 commits into
mainfrom
issue-2096

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

A Security run now reads whether the repository has Issues turned on before its first session. With Issues off, it fails at once with no model spend.

 security::run_through
   ready_issue                          # skip gate
   security_records                     # private fallback reads repos/{slug}, now keeps has_issues
   failed_fix / next_fix                # skip gate
   base_unchanged_since_security_audit  # skip gate
+  records.issues_enabled()             # reuse the fallback's has_issues
+    else outside.issues_enabled()      # GET repos/{slug} → has_issues
+  if Issues off → fail: "<owner>/<repo> has Issues turned off;
+                  turn on Issues for <owner>/<repo> or remove its Security schedule"
   processes::security::run             # Harness check, Node check, audit, ...
  • Where the check sits: after the skip gates, so a quiet skip makes no extra request, and existing unit tests already put skip gates before prerequisites. Before the Harness check and every session.
  • Repository reads: no repository read happened unconditionally at run start, so on the public-repository path this adds one repos/{slug} read. On a private repository's advisory-404 fallback, the existing read's has_issues is reused, so there is no second request.
  • Run notification: gives Result: pre-audit failed, Audit: not started and the fixed Cause: Issues turned off.
  • Docs: GLOSSARY.md Security run, docs/guide.md (manual regenerated), and the gates paragraph on site/index.html, site/how-it-works/ and site/architecture/.

Closes #2096

Evidence

Execution evidence

This host has no C compiler (which cc gcc clang finds none), and ring/aws-lc-sys need one in their build scripts. So cargo test cannot build here (error: linker 'cc' not found), and I could not record a local failing Before run of the new tests. CI on this PR (run 43268667219, head 652ab63) is the After evidence. All four rust-x86 shards passed:

Summary [ 346.382s] 1135 tests run: 1135 passed, 3520 skipped
Summary [ 434.937s] 1133 tests run: 1133 passed (1 slow), 3522 skipped
Summary [ 381.084s] 1097 tests run: 1097 passed (1 slow), 3558 skipped
Summary [ 346.968s] 1091 tests run: 1091 passed (1 slow), 3564 skipped

The first attempt of shard 2 failed on accounts::password_codes::email_code_creates_and_resets_an_account_without_revealing_the_allowlist (no code reached Resend after its 10-second wait). That Console test is untouched by this change. Its re-run passed, as did console-test and site-test.

These tests ran in that suite:

  • cargo test --test security_run a_repository_with_issues_off_fails_before_any_session_and_says_so: a fake-GitHub scenario with has_issues: false, run with fixing both off and on. Expects exit 1, the stderr reason, no Claude or Codex calls, no issue create, and Cause: Issues turned off.
  • cargo test --test security_run a_repository_with_issues_on_audits_as_before: has_issues: true audits as before.
  • cargo test --test security_run a_private_fallback_reads_the_repository_once_before_failing_on_issues_off: from review finding P1; reads repos/acme/widgets once.
  • cargo test --bin thirdshift security::tests: the gate comes after the skip gates and before the Harness check; existing call sequences gain IssuesCheck.

Run locally:

  • cargo fmt --check: clean.
  • npm run docs --prefix site then npm run docs:check --prefix site: exit 0.
  • npm test --prefix site: 7 pass and 61 fail. Every failure is Playwright's "Executable doesn't exist" (no Chromium on this host); the docs check inside it passed.

Visual evidence

Only prose changed on the site pages, and no headless browser is installed on this host (Playwright's Chromium is missing), so there are no captures.

Merge Danger

Door: two-way

Blast Radius: narrow

  • Affects only Security runs that get past their skip gates.
  • On a public repository, the run makes one more REST read (repos/{slug}).
  • If that read fails (rate limit, outage), the run fails before its audit with the usual GitHub dependency or rate-limit Cause, where before it would have started the audit.

Review

  • Standards: 7 findings, all judgement calls.
    • Fixed: S1 (the other site pages), S2 (guide sentence order), S3 (shared GitHub::repository read), S4 (the fake's flag is now positive), S6 (the unit test no longer repeats the message assertion).
  • Spec: 1 finding, P1 (a second repository read on the private fallback). Fixed, and its test is kept.

Changed files the review left unread as changes: the review ran on 77a5594. The follow-up commit 652ab63 changed src/github/advisories.rs, site/index.html and site/architecture/index.html. Reviewers read those files only in their pre-change state. site/docs/index.html is generated output: its exclusion was accepted after npm run docs:check --prefix site passed, and its two changed paragraphs were inspected.

Unaddressed findings

Standards

  • S5 (Repeated Switches / Shotgun Surgery, src/run_ending.rs:180-183): declined.
    • The finding: each fixed pre-audit Cause adds an .or_else to the chain.
    • Why declined: this follows the existing pattern beside it (crate::launch::base_branch_cause), and three causes do not warrant a registry. Building one now would be the Speculative Generality the baseline warns against.
  • S7 (low-value test, tests/security_run.rs a_repository_with_issues_on_audits_as_before): declined.
    • The finding: the fake already defaults has_issues to true, so the test adds little.
    • Why kept: the issue's acceptance criterion reads "With has_issues: true, behavior is unchanged". This test sets has_issues: true explicitly, so the fake's response field is exercised as GitHub sends it rather than through a default.

Spec

None.

Built with claude 2.1.322 · claude-opus-5-5 · high

Security review outcome

  • Security review incomplete: session failed, ended early or omitted a valid final line; see Session trace

A Security run now reads the repository's has_issues after its skip
gates and before its Harness check. With Issues off it fails at once,
naming the repository and saying to turn on Issues or remove the
Security schedule; its Run notification gives Cause: Issues turned off.

Closes #2096
Address review findings: a private repository's advisory-404 fallback
already reads the repository, so its has_issues answer is kept and the
Security run reads the repository only once. Share the repository read,
keep the InMemory fake's flag positive, and mention the check on every
site page that lists the Security run's gates.
#2094 (on main) and #2096 (this branch) both handle a repository with
Issues turned off, in incompatible ways: #2094 lets a Security run audit
and skip fixing; #2096 fails the run before any session so no audit is
paid for. This branch implements #2096, so its fail-fast gate wins:

- run_through keeps #2096's check after the skip gates and drops #2094's
  fixing_skipped path, which that check makes unreachable, along with the
  Ended::fixing_skipped field and its Run notification text.
- One repository read: Outside::has_issues and GitHub::has_issues (from
  #2094) replace this branch's issues_enabled duplicates, reading through
  the shared GitHub::repository helper; the private fallback's answer is
  still reused.
- #2094's guards in the fix-findings and security-audit Process Steps
  stay: Custom processes do not pass through the Security run's gate.
- Tests asserting a Security run audits with Issues off are removed; the
  malformed-metadata tests stay, with the new call order. The fake gh
  keeps main's has_issues passthrough.
- ADR 0015, docs/guide.md and site/index.html now say a Security run fails
  with Issues off; the manual is regenerated.
@JacobStephens2

Copy link
Copy Markdown
Owner Author

Delivery completed on agent-authored or reviewed head eb1b94e.

@JacobStephens2
JacobStephens2 merged commit acd9be1 into main Nov 12, 2026
24 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-2096 branch November 12, 2026 18:09
@JacobStephens2 JacobStephens2 mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check that Issues are enabled before a Security run starts its audit

1 participant