Research: Run Squad from the Copilot Coding Agent (CCA) #25

Activity

1. bradygaster

   added

   documentation

   enhancement

   squadSquad triage inbox — Lead will assign to a memberSquad triage inbox — Lead will assign to a member

   on Feb 12, 2026
2. bradygaster commented on Feb 12, 2026

   

   bradygaster

   on Feb 12, 2026

   CollaboratorAuthor

   More actions

   @copilot — Assignment

   Task: Research whether the Copilot Coding Agent (CCA) can run Squad as its governing agent, and produce a go/no-go assessment.

   Deliverable format:

   1. Answer each of the 7 research questions in the issue body with evidence (docs, observed behavior, or test results)
   2. Post a GO or NO-GO recommendation
   3. If GO: include a sprint plan with work items and squad-time estimates
   4. If NO-GO: document exactly what's missing and what would need to change

   Key files to read for context:

   • .github/agents/squad.agent.md — the full coordinator governance (this is what CCA would need to load)
   • .ai-team/team.md — current team roster
   • docs/features/copilot-coding-agent.md — how @copilot currently integrates with Squad
   • team-docs/proposals/032a-provider-abstraction-architecture.md — provider abstraction (relevant to cross-client portability)

   Related issues: #10 (client parity), #24 (/fleet research)
3. bradygaster

   added

   go:needs-researchNeeds investigationNeeds investigation

   release:backlogNot yet targetedNot yet targeted

   type:spikeResearch/investigation — produces a plan, not codeResearch/investigation — produces a plan, not code

   release:v0.5.0Targeted for v0.5.0Targeted for v0.5.0

   squad:kujan

   and removed

   enhancement

   documentation

   release:backlogNot yet targetedNot yet targeted

   on Feb 13, 2026
4. bradygaster commented on Feb 18, 2026

   

   bradygaster

   on Feb 18, 2026

   CollaboratorAuthor

   More actions

   CCA Compatibility Research — Go/No-Go Assessment

   I've completed the research on whether Squad can run from the Copilot Coding Agent (CCA). Here's the assessment:

   ---

   Summary: ❌ NO-GO for v0.5.0 (pending empirical test)

   Squad cannot run from CCA unless we confirm CCA has access to sub-agent spawning tools. The architecture depends entirely on multi-agent orchestration — without it, Squad is just a single agent following instructions.

   ---

   What Works ✅

   1. Custom agent files — CCA reads .github/agents/*.agent.md exactly like CLI. CCA can load squad.agent.md as its governance.
   2. MCP server access — CCA supports MCP servers (GitHub MCP included). Issue management is feasible.
   3. Async session model — CCA operates asynchronously (no user in loop during execution). Compatible with Squad's batch execution model.
   4. File writes — CCA can write to .ai-team/ and commit changes back via PR.

   ---

   What's Missing ❌

   1. Sub-agent spawning (BLOCKER) — No documentation confirms CCA has access to ask,
      unSubagent, or equivalent tool. Squad's architecture requires spawning real sub-agents. Without this, Squad can't function as a multi-agent system.

   2. Background mode — CCA likely doesn't support \mode: "background"\ for fire-and-forget sub-agents. Could fall back to VS Code's parallel sync pattern, but requires sub-agent spawning first.

   3. .ai-team/ access (MAJOR CONSTRAINT) — .ai-team/ is gitignored on main (per Squad's architectural decision). CCA can't read Squad's memory (decisions, history, skills). All governance must be self-contained in squad.agent.md. This is a fundamental limitation.

   ---

   Research Findings (by question)

   Question	Answer	Status
   Can CCA use custom agent files?	YES — .github/agents/*.agent.md auto-discovered	✅ Confirmed
   Does CCA have \ ask\ tool access?	UNKNOWN — not documented	⚠️ BLOCKER
   Does CCA support background mode?	NO — likely not available	❌ Degraded but acceptable
   Does CCA have MCP server access?	YES — GitHub MCP included	✅ Confirmed
   Can CCA read .ai-team/ files?	NO — gitignored on main	❌ MAJOR CONSTRAINT
   Can CCA write .ai-team/ files?	YES — can commit via PR	✅ Confirmed
   How does async session model work?	Fire-and-forget batch execution, human gates at PR review	✅ Compatible
   What's the max complexity CCA can handle?	Best for single-issue work; multi-agent sprints risky without sub-agent spawning	⚠️ Limited

   ---

   Recommended Action

   Phase 1: Spike (2-4 hours) — Empirical Test

   Create a test custom agent in .github/agents/ that attempts to call the ask tool. Delegate an issue to it via CCA and observe:

   • Does CCA have access to ask?
   • Does it report tool not found?
   • Does it fall back to inline work?

   Outcome determines next steps:

   • If ask tool available: GO for v0.5.0. Design CCA integration (8-12h).
   • If ask tool NOT available: NO-GO. Select fallback:
     • Option A: CCA as Squad member (not coordinator) — Squad routes work TO CCA
     • Option B: Lightweight CCA mode (single-agent Squad) — CCA follows Squad conventions but works inline
     • Option C: Defer to v0.6.0 — wait for platform evolution

   ---

   Why This Matters

   Without sub-agent spawning, CCA-Squad is either:

   • A single agent pretending to be a team (not Squad's value prop), or
   • A member OF Squad (not Squad itself)

   The vision in #25 is "CCA boots Squad — spawns the team, uses shared memory, works as a full squad session." That vision requires sub-agent spawning. Without it, we're building a workaround, not the real thing.

   ---

   Full Analysis

   Complete findings, constraints, and fallback architectures documented in:
   .ai-team/decisions/inbox/kujan-cca-research.md

   ---

   Verdict: RUN THE SPIKE. Don't commit to v0.5.0 scope until we know if ask tool exists in CCA.

   — Kujan
5. bradygaster commented on Feb 18, 2026

   

   bradygaster

   on Feb 18, 2026

   CollaboratorAuthor

   More actions

   🧪 CCA Spike Test Plan — Empirical Capability Probing

   Author: Kujan (Copilot SDK Expert)
   Purpose: Step-by-step instructions to empirically test what tools and capabilities the Copilot Coding Agent (CCA) has access to — specifically sub-agent spawning (task tool), .ai-team/ file access, MCP tool availability, and context window limits.
   Time estimate: 2-4 hours (including CCA execution wait time)
   Prerequisite: Repository admin with CCA enabled (Settings → Copilot → Coding agent)

   ---

   1. Test Setup

   Branch & Files

   Create a dedicated test branch from dev (not main) so .ai-team/ files are present in the working tree:

   git checkout dev
   git checkout -b spike/cca-capability-probe

   Files to Create

   You will create 3 files on this branch:

   File	Purpose
   .github/agents/cca-probe.agent.md	Custom agent that probes for tool availability
   .ai-team/cca-probe-marker.md	Canary file to test .ai-team/ read access
   Test issue on GitHub	Trigger CCA via @copilot assignment

   Important: Push Before Testing

   CCA clones from GitHub — local-only files won't be visible. Push the branch:

   git push origin spike/cca-capability-probe

   ---

   2. Test Agent — cca-probe.agent.md

   Create .github/agents/cca-probe.agent.md with this exact content:

   ---
   name: CCA Capability Probe
   description: "Diagnostic agent that tests Copilot platform capabilities — tool availability, file access, and context limits."
   ---
   
   You are a diagnostic agent. Your ONLY job is to probe what capabilities are available in your current execution environment and report findings.
   
   ## Instructions
   
   Complete ALL of the following probes and write your findings to a NEW file called `cca-probe-results.md` in the repository root. Then open a PR with the results.
   
   ### Probe 1: Tool Inventory
   
   List EVERY tool you have access to. For each tool, report:
   - Tool name (exact identifier)
   - Brief description of what it does
   - Whether it accepts parameters
   
   Specifically check for and report on these tools:
   - `task` (sub-agent spawning)
   - `powershell` or `bash` or `shell` (command execution)
   - `edit` / `create` / `view` (file operations)
   - `grep` / `glob` (search)
   - `read_agent` / `list_agents` (agent management)
   - `web_search` / `web_fetch` (internet access)
   - `sql` (database)
   - `github-mcp-server-*` (GitHub MCP tools)
   
   Format as a markdown table.
   
   ### Probe 2: Sub-Agent Spawning Test
   
   **IF** you have access to the `task` tool:
   1. Attempt to spawn an `explore` agent with prompt: "What files exist in the .ai-team/ directory? List them all."
   2. Report the result (success/failure, output)
   3. Attempt to spawn a `general-purpose` agent with prompt: "Create a file called sub-agent-was-here.md containing the text 'Sub-agent spawning confirmed.'"
   4. Report whether the sub-agent was spawned, what mode it ran in, and whether the file was created
   
   **IF** you do NOT have access to the `task` tool:
   1. State clearly: "task tool is NOT available in this environment"
   2. List what tools you tried as alternatives (if any)
   3. Check if `runSubagent` or any other spawn mechanism exists
   
   ### Probe 3: `.ai-team/` File Access
   
   1. Attempt to read `.ai-team/cca-probe-marker.md` — report success or failure and the content if readable
   2. Attempt to read `.ai-team/team.md` — report success or failure
   3. Attempt to read `.ai-team/decisions.md` — report success or failure
   4. Attempt to list all files in `.ai-team/` directory — report the listing
   5. State which branch you are working on
   
   ### Probe 4: MCP Tool Test
   
   1. Attempt to use `github-mcp-server-list_issues` to list issues in this repository
   2. Report success or failure, and the first 3 issue titles if successful
   3. Attempt to use `github-mcp-server-issue_read` to read issue #25
   4. Report success or failure
   
   ### Probe 5: Context Window Assessment
   
   1. Report the total length of your system prompt (approximate word count or token count if available)
   2. Report whether you can see the full content of THIS agent file
   3. Attempt to read `.github/agents/squad.agent.md` and report:
      - Could you read it? (yes/no)
      - Approximate length (lines/words)
      - First 5 lines of content
      - Last 5 lines of content
      - Did it appear truncated?
   
   ### Probe 6: Environment Details
   
   Report:
   1. What operating system are you running on?
   2. What is your current working directory?
   3. What git branch are you on?
   4. Can you run shell commands? If so, run `node --version` and `git --version` and report output
   5. Do you have internet access?
   6. What model are you? (report your model identifier if you know it)
   
   ## Output Format
   
   Write ALL findings to `cca-probe-results.md` in this exact structure:
   
   ```markdown
   # CCA Capability Probe Results
   
   **Executed:** [timestamp]
   **Branch:** spike/cca-capability-probe
   **Issue:** [issue number that triggered this]
   
   ## Tool Inventory
   [table of all tools]
   
   ## Sub-Agent Spawning
   **Available:** YES / NO
   [details]
   
   ## .ai-team/ File Access
   **Readable:** YES / NO
   [details per file]
   
   ## MCP Tools
   **Available:** YES / NO
   [details]
   
   ## Context Window
   [details]
   
   ## Environment
   [details]
   
   ## Summary Verdict
   - Sub-agent spawning: ✅ / ❌
   - .ai-team/ access: ✅ / ❌
   - MCP tools: ✅ / ❌
   - Context sufficient for squad.agent.md: ✅ / ❌
   ```
   
   **CRITICAL:** Do NOT skip any probe. If a probe fails, report the failure and error message. The goal is data collection, not success.

   ---

   3. Canary File — .ai-team/cca-probe-marker.md

   Create .ai-team/cca-probe-marker.md:

   # CCA Probe Marker
   
   If you can read this file, CCA has access to .ai-team/ on this branch.
   
   Canary value: SQUAD-CCA-PROBE-2026
   Branch: spike/cca-capability-probe

   This gives us a unique string (SQUAD-CCA-PROBE-2026) to search for in CCA's output, confirming it actually read the file vs. hallucinating content.

   ---

   4. How to Trigger CCA

   Step 1: Push the branch

   git add .github/agents/cca-probe.agent.md .ai-team/cca-probe-marker.md
   git commit -m "spike: add CCA capability probe agent and marker file"
   git push origin spike/cca-capability-probe

   Step 2: Create the trigger issue

   Create an issue from the GitHub UI (not CLI, so you can assign @copilot):

   Title: [Spike] CCA Capability Probe — Tool Availability Test

   Body:

   ## Task
   
   Run the CCA Capability Probe diagnostic agent.
   
   Use the `cca-probe` agent (defined in `.github/agents/cca-probe.agent.md`) on branch `spike/cca-capability-probe`.
   
   Follow ALL instructions in the agent file. Complete every probe (1-6). Write results to `cca-probe-results.md` and open a PR.
   
   ## Branch
   
   Work on branch: `spike/cca-capability-probe`
   
   ## Context
   
   This is a diagnostic test to determine what tools and capabilities are available in the CCA environment. We need empirical data, not assumptions.
   
   Related: #25

   Labels: type:spike

   Step 3: Assign to @copilot

   1. In the issue sidebar, click "Assignees"
   2. Assign copilot (the Copilot Coding Agent)
   3. CCA will pick up the issue and begin working

   Step 4: Wait

   CCA typically starts within 1-5 minutes. Full execution may take 5-15 minutes. Monitor:

   • The issue comment thread (CCA posts progress updates)
   • The PR it opens (contains the probe results file)
   • The Actions tab (CCA runs may appear here)

   ---

   5. What to Observe

   Primary Signals

   Signal	Where to Look	What It Means
   cca-probe-results.md exists in PR	PR diff	CCA executed the probe agent
   Tool inventory table present	cca-probe-results.md	CCA reported its available tools
   task listed in tool inventory	cca-probe-results.md → Tool Inventory	Sub-agent spawning is available
   sub-agent-was-here.md created	PR diff	Sub-agent actually spawned and wrote a file
   Canary value SQUAD-CCA-PROBE-2026 appears	cca-probe-results.md → .ai-team/ section	CCA read .ai-team/ files on this branch
   MCP issue listing present	cca-probe-results.md → MCP section	CCA has GitHub MCP access
   squad.agent.md length reported	cca-probe-results.md → Context section	We know if context window can hold our coordinator

   Failure Signals

   Signal	Where to Look	What It Means
   CCA posts "I cannot complete this task"	Issue comments	Agent file wasn't loaded or tools missing
   No PR opened after 30 minutes	Issue + repo PRs	CCA failed silently
   task tool is NOT available in results	cca-probe-results.md	BLOCKER confirmed — no sub-agent spawning
   .ai-team/ files show "access denied" or "not found"	cca-probe-results.md	File system constraint confirmed
   squad.agent.md reported as truncated	cca-probe-results.md	Context window too small for full coordinator
   CCA ignores the custom agent and works as vanilla	PR content doesn't match probe format	CCA didn't load cca-probe.agent.md

   Secondary Signals

   • Did CCA create a branch from spike/cca-capability-probe or from main? This tells us if CCA respects the branch instruction.
   • How long did CCA take? Gives us baseline for multi-agent Sprint estimation.
   • What model does CCA report? Tells us the reasoning tier available.

   ---

   6. Decision Matrix

   Observation	Decision	Next Action
   task tool available + sub-agent spawned successfully	GO for CCA integration	Design CCA coordinator mode in squad.agent.md (8-12h). Target v0.5.0.
   task tool available but sub-agent failed	CONDITIONAL GO	Investigate failure mode. May need spawn pattern adaptation.
   task tool NOT available, but runSubagent or equivalent exists	GO with adaptation	Map CCA spawn API (like VS Code parity work). 4-8h additional design.
   task tool NOT available, no spawn mechanism	NO-GO for full Squad	Fallback: CCA as Squad member (not coordinator). Squad routes work TO CCA via issues.
   CCA didn't load cca-probe.agent.md at all	NO-GO	Custom agents don't work in CCA. Fundamental blocker.
   .ai-team/ not readable even on dev branch	Constraint confirmed	All CCA governance must be self-contained in squad.agent.md. No filesystem memory.
   squad.agent.md truncated by context window	Constraint confirmed	Need CCA-specific lightweight coordinator variant.
   MCP tools not available	Partial NO-GO	CCA can't manage issues directly. Less useful for async workflows.

   Combined Verdict Logic

   IF task_tool = YES AND ai_team_access = YES:
     → FULL GO: CCA can be Squad coordinator with memory access
   
   IF task_tool = YES AND ai_team_access = NO:
     → PARTIAL GO: CCA as coordinator but stateless (no memory/decisions)
     → Mitigation: embed critical decisions in squad.agent.md directly
   
   IF task_tool = NO AND ai_team_access = YES:
     → NO-GO as coordinator. CCA as Squad member only.
     → CCA follows Squad conventions, works inline, commits state
   
   IF task_tool = NO AND ai_team_access = NO:
     → FULL NO-GO. Defer CCA integration to v0.6.0+
     → CCA is vanilla Copilot with copilot-instructions.md only
   

   ---

   7. Secondary Tests

   Test A: .ai-team/ Visibility Under Gitignore

   Why: .ai-team/ is committed on dev but the repo's .gitignore comments say "blocked from main/preview by squad-main-guard.yml workflow." The gitignore itself does NOT have an .ai-team/ entry — it's a workflow guard, not an ignore rule. CCA clones the branch it works on, so if working on dev or a branch based on dev, .ai-team/ should be visible.

   How to verify (from probe results):

   • Check if CCA reports the branch it's working on
   • Check if the canary value SQUAD-CCA-PROBE-2026 appears in results
   • If CCA works on main despite being told spike/cca-capability-probe, the .ai-team/ test is invalid — file a follow-up

   Edge case to watch: CCA might clone main and then cherry-pick the agent file. In that case, .ai-team/ won't exist even though it does on the spike branch.

   Test B: MCP Tools in Practice

   Why: CCA docs say MCP is supported, but we need to confirm the GitHub MCP server is actually connected and functional in CCA's runtime, not just documented.

   How to verify (from probe results):

   • Probe 4 asks CCA to list issues and read issue Research: Run Squad from the Copilot Coding Agent (CCA) #25
   • If it returns real issue data (titles match our actual issues), MCP is functional
   • If it returns an error or hallucinated data, MCP is configured but broken or not connected

   Follow-up if MCP works: Can CCA use gh CLI directly? The probe's environment section asks CCA to run shell commands. If both MCP and gh work, CCA has two paths to GitHub API access.

   Test C: Token/Context Limit Under squad.agent.md

   Why: squad.agent.md is ~1,800 lines. CCA might truncate it, especially if combined with the custom agent instructions, system prompt, and issue body. If CCA can't hold the full coordinator in context, Squad-on-CCA requires a lightweight variant.

   How to verify (from probe results):

   • Probe 5 asks CCA to read squad.agent.md and report its length + whether it appears truncated
   • Compare reported line count against actual (wc -l .github/agents/squad.agent.md)
   • If first 5 and last 5 lines match, and line count is accurate, context window is sufficient
   • If last lines don't match or CCA reports truncation, measure how much was lost

   Contingency: If truncated, follow up with a second spike that tests a stripped-down squad-cca.agent.md (~500 lines, CCA-specific coordinator without Init Mode, ceremonies, etc.)

   ---

   8. Cleanup

   After collecting results, regardless of outcome:

   # Delete the spike branch (after merging probe results to dev if valuable)
   git push origin --delete spike/cca-capability-probe
   
   # Close the trigger issue (or leave open if follow-up spikes needed)
   gh issue close [ISSUE_NUMBER] --repo bradygaster/squad --comment "Spike complete. Results collected."
   
   # Remove test files if they were merged anywhere
   git rm .github/agents/cca-probe.agent.md
   git rm .ai-team/cca-probe-marker.md
   git rm -f cca-probe-results.md
   git rm -f sub-agent-was-here.md
   git commit -m "cleanup: remove CCA probe spike files"

   Do NOT delete: The probe results comment (this issue) or the PR that CCA opened — these are the evidence record.

   ---

   9. Timeline

   Step	Time	Who
   Create branch + files + push	15 min	Anyone
   Create issue + assign @copilot	5 min	Repo admin
   Wait for CCA execution	5-30 min	(automated)
   Analyze results + update decision matrix	30 min	Kujan or Brady
   Post go/no-go verdict on #25	15 min	Kujan
   Cleanup	10 min	Anyone
   Total	~1.5 hours

   ---

   This test plan is designed for a single execution. If results are ambiguous (e.g., CCA ignores the custom agent), re-run with modified instructions before declaring NO-GO. LLM non-determinism means one failure isn't conclusive — but one success IS.

   — Kujan, Copilot SDK Expert
6. bradygaster

   added a commit that references this issue on Feb 19, 2026

   docs(ai-team): orchestration log entries for design review and triage

   ...

   6e25578
7. bradygaster commented on Feb 20, 2026

   

   bradygaster

   on Feb 20, 2026

   CollaboratorAuthor

   More actions

   Deferred — CCA research not blocking v0.5.0. Filed as v0.5.x backlog item.
8. bradygaster

   closed this as completedon Feb 20, 2026
9. tamirdresher

   added a commit that references this issue on Mar 8, 2026

   docs: renumber ADO blog to github#25, add release notes blog github#26

   ...

   c9fa9bc
10. bradygaster

    added a commit that references this issue on Mar 8, 2026

    feat: Azure DevOps platform adapter — Squad for enterprise (#191)

    ...

    ea70504
11. joniba

    added 3 commits that reference this issue on Mar 26, 2026

    research(upstream-inheritance): comprehensive org-scale knowledge sha…

    ...

    58ac06c

    docs(elrond): Append upstream inheritance research to history

    ...

    0256bca

    docs: rate limiting research (github#26) + upstream inheritance (gith…

    ...

    9d27757
12. akubly

    added a commit that references this issue on May 28, 2026

    Scribe: merge piece-25 decisions and log workflow

    ...

    This issue will close once commit 919bbb3 is merged into the 'dev' branch.

    919bbb3