Skip to content

Phase 4/8: green CI is not verification when CI scope excludes the changed files #46

Closed

Description

@dmccoystephenson

Template gap

Phase 4 anchors the self-review rubric on green CI ("without it, the rubric is just opinion"). But when a PR's changed files are outside CI's scope — platform-specific scripts (.ps1, .bat, .command), installer configs, OS-gated shell paths — CI is green while none of the deliverable was tested. A real bug (a batch %errorlevel% delayed-expansion trap) shipped under green CI and was caught only by adversarial hand-review.

Rule to add to the template (Phase 4 self-review + Phase 8 merge note)

If the PR's changed files include code the CI structurally cannot execute (platform-specific scripts/tooling, OS-gated paths), green CI does not constitute verification of those files. State the no-automated-coverage gap explicitly in the PR body and CHANGELOG, lean harder on adversarial hand-review for that code, and recommend a real-platform smoke test before merge. Do not let a green run on the other language imply the script was tested.

Consolidates (close on merge)

  • example-|-dev-loop#6 — platform-specific scripts have no automated coverage; green CI is misleading

Consolidated upstream from per-loop self-audit findings. drafted by Claude on behalf of Daniel Stephenson

Activity

  1. added a commit that references this issue on Jul 18, 2026
  2. added 2 commits that reference this issue on Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    template-ruleShould be promoted into create-dev-loop.md

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions