Skip to content

feat: Init/Upgrade Parity & SDK Compatibility Testing — PRD #817

Closed

Description

@diberry

Problem Statement

When Squad adds new scaffolding to squad init (e.g., .squad/casting/), the squad upgrade path is not updated to match. This causes squad doctor failures for users upgrading from older versions. Additionally, there is zero runtime compatibility testing against @github/copilot-sdk updates — all tests mock the SDK, so breaking changes are only discovered at user runtime.

Background

  • Casting gap: .squad/casting/ was added to init (PR Errors and warnings after squad-cli init v0.9.1 #579, Mar 23) but never added to ENSURE_DIRECTORIES in upgrade.ts
  • Other potential gaps: .squad/agents also missing from ENSURE_DIRECTORIES
  • SDK testing: copilot-sdk at ^0.1.32 (caret range) — users can get breaking minor versions without team knowing
  • ESM patches: Postinstall patches for vscode-jsonrpc and copilot-sdk are brittle — no verification they actually work

Found during v0.9.1 insiders testing by Brady.

Requirements

FR-1: Init/Upgrade Parity Invariant

Every directory and file created by squad init MUST be ensured by squad upgrade.

Acceptance criteria:

  • .squad/casting added to ENSURE_DIRECTORIES in upgrade.ts
  • Default casting files (registry.json, policy.json, history.json) scaffolded if missing
  • Audit all init scaffolds vs ENSURE_DIRECTORIES — document gaps
  • Test: init → delete casting/ → upgrade → doctor passes

FR-2: Init/Upgrade Parity Test Suite

Automated tests that prevent future init/upgrade drift.

Acceptance criteria:

  • Test: ENSURE_DIRECTORIES covers all directories init creates
  • Test: upgrade on pre-casting repo → doctor all-green
  • Test: upgrade on pre-sessions repo → doctor all-green
  • Parametric test: delete each ENSURE dir → upgrade recreates it

FR-3: CI Gate — Upgrade→Doctor Validation

CI workflow step that runs upgrade→doctor on every PR touching init or upgrade code.

Acceptance criteria:

  • New CI step in squad-ci.yml
  • Triggers on changes to: upgrade.ts, init.ts, doctor.ts, templates.ts
  • Fails CI if doctor reports any fail status after upgrade

FR-4: SDK Compatibility — Dependabot + Nightly Canary

Automated detection of @github/copilot-sdk breaking changes.

Acceptance criteria:

  • .github/dependabot.yml configured for daily npm updates
  • Nightly canary workflow tests against copilot-sdk@next
  • Alert on failure (issue comment or notification)

FR-5: SDK Compatibility — Unmocked Smoke Test

At least one test that imports @github/copilot-sdk WITHOUT mocking.

Acceptance criteria:

  • Test validates CopilotClient is importable and has expected interface
  • Test validates vscode-jsonrpc exports field exists
  • Test validates postinstall patches were applied
  • Runs as part of standard CI (not just nightly)

FR-6: Postinstall Patch Verification

Defense-in-depth for ESM patches.

Acceptance criteria:

  • verify-patches.mjs script runs after patch-esm-imports.mjs
  • Fails install loudly if patches didn't apply
  • CI validates patches on clean install

FR-7: Doctor UX for Global Installs

Doctor warnings should not alarm users on expected global install configuration.

Acceptance criteria:

  • Doctor detects global vs local install mode
  • Global install: ESM checks show "ℹ️ info" not "⚠️ warn"
  • Message explicitly says "expected for global CLI installs"

Non-Functional Requirements

NFR-1: No False Positives

Parity tests must not fail on intentionally different init vs upgrade behavior (e.g., agents/ is user data, not scaffolding).

NFR-2: Backward Compatibility

Upgrade must work on repos from v0.7.x through v0.9.x without data loss.

NFR-3: CI Performance

New CI steps must complete in <60 seconds.

Design Decisions

DD-1: ENSURE_DIRECTORIES is the Single Source of Truth

All directories that upgrade should create are listed in one array. No implicit directory creation elsewhere.

DD-2: SDK Caret Range Acceptable with Canary

Keep ^0.1.32 caret range but add nightly canary testing. This allows automatic minor updates while catching breaks early.

DD-3: Postinstall Patches Are Temporary

ESM patches should be tracked with TODOs. When copilot-sdk ships native ESM support, remove patches.

Effort Estimates

Requirement Effort Priority
FR-1: Casting fix 30 min 🔴 Critical (in progress — PR fix/squad-doctor-issues)
FR-2: Parity test suite 2-3 hrs 🔴 Critical
FR-3: CI gate 2-3 hrs 🟠 High
FR-4: Dependabot + canary 4-6 hrs 🟠 High
FR-5: Unmocked smoke test 3-4 hrs 🟡 Medium
FR-6: Patch verification 2-3 hrs 🟡 Medium
FR-7: Doctor UX 1-2 hrs 🟡 Medium
Total ~16-24 hrs

References

  • Retro analysis from v0.9.1 insiders testing (2026-04-04)
  • Related PR: fix/squad-doctor-issues (casting dir fix)
  • SDK dependency: @github/copilot-sdk@^0.1.32
  • ESM patch scripts: packages/squad-cli/scripts/patch-esm-imports.mjs

Activity

  1. diberry commented on Apr 6, 2026

    @diberry
    ContributorAuthor

    ✈️ Flight PRD Assessment

    Verdict: PRD is complete and ready for implementation.

    Comprehensive PRD covering init/upgrade parity (FR-1–FR-3), SDK compatibility (FR-4–FR-6), and doctor UX (FR-7). Design decisions and NFRs are well-defined.

    Strengths:

    • FR-2 (parametric parity test) is the highest-value gate — prevents future init/upgrade drift permanently
    • FR-3 (CI gate) closes the loop with automated enforcement
    • FR-7 (doctor UX for global installs) addresses a real user confusion point
    • DD-1 (ENSURE_DIRECTORIES as single source of truth) is the right architectural call

    Minor gaps (non-blocking):

    Recommendation: Split implementation — FR-1 (casting gap fix) ships immediately as a patch. FR-2 + FR-3 ship together as a test suite PR. FR-4–FR-6 defer to #818. FR-7 is independent.

  2. added 2 commits that reference this issue on Apr 8, 2026
    3f391eb
    b75d623
  3. diberry commented on Apr 10, 2026

    @diberry
    ContributorAuthor

    Closing — descoping high-touch feature work to focus on existing PRs and low-touch fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or improvement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions