Skip to content

Fail loudly instead of skipping when no POSIX shell resolves (#1833) - #1857

Merged
bradygaster merged 1 commit into
devfrom
squad/1833-gh-aw-quality-posix-shell
Aug 24, 2026
Merged

bradygaster merged 1 commit into
devfrom
squad/1833-gh-aw-quality-posix-shell

Conversation

@bradygaster

Copy link
Copy Markdown
Collaborator

Closes #1833.

The defect, measured

test/gh-aw-quality.test.ts gated two describe blocks on existsSync('/bin/sh'), absent on a stock Windows dev box.

Tests Skipped
Before 96 passed 13 skipped (109)
After 111 passed 0 (111)

The 13 hidden tests now execute — and pass — on Windows. jq and awk resolve through Git Bash, so no previously-hidden product defect surfaced.

The fix

The correct shape already existed in gh-aw-command-parse.test.ts (from #1832). Rather than copy it a third time — the issue's explicit warning — it moves to test/posix-shell.ts and all three gh-aw suites import it.

The audit for acceptance criterion 4 found the propagation had already happened: gh-aw-activate-roster-binding.test.ts carried a verbatim third copy plus 8 more it.skipIf(!POSIX_SHELL) cases. Those are folded in here — deduping two copies while leaving the third would have left the vector intact.

  • Every skipIf-on-shell-probe removed: 2 describes + 8 its.
  • Absence throws via requirePosixShell() and is asserted by a named guard test.

Proven capable of failing

Forcing resolvePosixShell() to return null:

AssertionError: No POSIX shell found. These cases are behavioral — skipping them
would report a green suite for assertions that never ran. Install Git for
Windows (provides bash.exe) or run on a POSIX host.

Red from both the guard assertion and every behavioral case. Per #1832's finding that a status-only check passes a generic-diagnostic mutation, the diagnostic text names the missing dependency explicitly.

Verification

  • gh-aw-quality + gh-aw-command-parse + gh-aw-activate-roster-binding: 157 passed, 0 skipped
  • npx eslint on all four files: clean
  • No skipIf on a shell probe remains in any gh-aw suite

Remaining skipIf, deliberately untouched

gh-aw-quality.test.ts:1094 — it.skipIf(!ghAwAvailable) for the gh aw compile gate. Same defect class, but it is #1834's subject and needs a CI job that installs gh aw; the finding is recorded on that issue. The other two (aspire-integration, docs-build) gate on non-gh-aw build prerequisites and carry explicit skip reasons.

test/gh-aw-quality.test.ts gated two describe blocks on
existsSync('/bin/sh'), which is absent on a stock Windows dev box. Measured
before: 96 passed | 13 skipped (109) — the suite reported green for 13 tests
that never ran.

Replaces the probe with the Git Bash resolver already proven in
gh-aw-command-parse.test.ts, extracted to test/posix-shell.ts so the gh-aw
suites share one implementation. The resolver existed in two verbatim copies
and gh-aw-activate-roster-binding.test.ts held a third; all three now import
the shared helper.

Removes every skipIf-on-shell-probe (2 describes, 8 its). Absence now throws
via requirePosixShell() and is asserted by a named guard test, so a missing
shell reports red and names the dependency.

Measured after: 111 passed | 0 skipped (111) for gh-aw-quality.test.ts — the
13 hidden tests execute and pass on Windows. All three suites: 157 passed, 0
skipped.

Mutation-verified: forcing resolvePosixShell() to return null turns the suite
red with a diagnostic naming Git for Windows, satisfying the requirement that
a status-only assertion is insufficient.

Closes #1833

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 20afe6d2-444e-414e-8a39-e67ab67ca6df
Copilot AI lite review requested due to automatic review settings August 24, 2026 19:33
@github-actions

Copy link
Copy Markdown
Contributor

🟢 Impact Analysis — PR #1857

Risk tier: 🟢 LOW

📊 Summary

Metric Count
Files changed 4
Files added 1
Files modified 3
Files deleted 0
Modules touched 1

🎯 Risk Factors

  • 4 files changed (≤5 → LOW)
  • 1 module(s) touched (≤1 → LOW)

📦 Modules Affected

tests (4 files)
  • test/gh-aw-activate-roster-binding.test.ts
  • test/gh-aw-command-parse.test.ts
  • test/gh-aw-quality.test.ts
  • test/posix-shell.ts

This report is generated automatically for every PR. See #733 for details.

@github-actions

github-actions Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 44a47e5

PR Scope: 🔧 Infrastructure

⚠️ 2 item(s) to address before review

Status Check Details
✅ Single commit 1 commit — clean history
✅ Not in draft Ready for review
✅ Branch up to date Up to date with dev
❌ Copilot review No Copilot review yet — it may still be processing
✅ Changeset present No source files changed — changeset not required
✅ Scope clean No .squad/ or docs/proposals/ files
✅ No merge conflicts No merge conflicts
❌ Copilot threads resolved 3 unresolved Copilot thread(s) — fix and resolve before merging
✅ CI passing All checks passing

Files Changed (4 files, +93 −55)

File +/−
test/gh-aw-activate-roster-binding.test.ts +13 −27
test/gh-aw-command-parse.test.ts +5 −18
test/gh-aw-quality.test.ts +25 −10
test/posix-shell.ts +50 −0

Total: +93 −55


This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical Windows execution issues and missing no-shell guard coverage remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR centralizes POSIX-shell resolution and removes silent shell-dependent test skips.

Changes:

  • Adds shared shell discovery and explicit failure handling.
  • Enables previously skipped behavioral tests.
  • Deduplicates shell-resolution logic across gh-aw suites.
File summaries
File Summary
test/posix-shell.ts Adds shared shell resolution; guard coverage needs a mocked no-shell case.
test/gh-aw-quality.test.ts Enables shell-dependent tests; Windows execution should use execFileSync with Bash’s -c arguments.
test/gh-aw-command-parse.test.ts Reuses shared shell resolution.
test/gh-aw-activate-roster-binding.test.ts Removes duplicated resolution and shell-based skips.
Review details

Suppressed comments (1)

test/gh-aw-command-parse.test.ts:21

  • This suite still keeps its own if (!POSIX_SHELL) throw new Error('no POSIX shell resolved') path in runDeclared, so it bypasses the shared fail-loud diagnostic. When resolution is unavailable, the guard reports the detailed missing-dependency message but every parser case reports the generic one, leaving this suite inconsistent with the other callers and with requirePosixShell()'s contract. Import and use requirePosixShell() in runDeclared instead.
import { POSIX_SHELL } from './posix-shell';
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

return execSync(
`git show HEAD:.squad/team.md 2>/dev/null | awk '{sub(/\\r$/,"")} /^## Members/{f=1;next} f&&/^#/{f=0} f&&/^\\|/&&!/^\\|[-: |]*\\|$/&&!/\\| *Name *\\|/' | grep -q . && echo TEAM_PRESENT || echo TEAM_ABSENT`,
{ cwd, shell: '/bin/sh', encoding: 'utf8' }
{ cwd, shell: requirePosixShell(), encoding: 'utf8' }
return execSync(
`echo '${jsonInput.replace(/'/g, "'\\''")}' | jq -r '${filter}'`,
{ shell: '/bin/sh', encoding: 'utf8' }
{ shell: requirePosixShell(), encoding: 'utf8' }
Comment thread test/posix-shell.ts

/** Resolve a POSIX shell or throw naming the missing dependency. Never skips. */
export function requirePosixShell(): string {
if (!POSIX_SHELL) throw new Error(NO_POSIX_SHELL_MESSAGE);
@bradygaster
bradygaster merged commit 0b5d83a into dev Aug 24, 2026
17 checks passed
@bradygaster
bradygaster deleted the squad/1833-gh-aw-quality-posix-shell branch September 9, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gh-aw quality suite silently skips 13 tests (28 assertions) on Windows via existsSync('/bin/sh') probe

2 participants