Gap analysis on done and not done work

I´m frequently hitting "we are done" on tasks that are not done at all.

My work flow is that I plan lot of issues in github and then point the team to it. I make sure that the issue is broken up into a phase and tasks

Phase 1 Do something

[ ] Something 1
[ ] Something 2

Phase 2 Something else

[ ] Something 1
[ ] Something 2

I ask the team to tick off what has been done and to automatically go through it again to make sure.

BUT this constantly fails. Its probably because the issues are to big and should be in separate issues but when I have dont that I have had issues with the overlapping code (things done in parallell and not merged correctly.. and just all kinds of issues..).

So I was wondering.. could there be a "gap analysis" loop that re-checsk if the work has really been done (or is this loop all ready there and my issue is steaming from something else?)

p.s
I see lot of work being done in the last 2 weeks (that might help with some of my issues) but no new version of Squad.. when can we expect a new version?

I love this idea. Squad. Does. Checklists.

🏗️ Squad Triage — Flight (Lead)

Issue: #472 — Gap analysis on done and not done work
Assigned to: Flight (Lead)
Reason: No specific domain match — assigned to Lead for further analysis

@copilot evaluation: No strong capability match — routed to squad member.

Team roster:

Flight (Lead) → label: squad:flight
Procedures (Prompt Engineer) → label: squad:procedures
EECOM (Core Dev) → label: squad:eecom
FIDO (Quality Owner) → label: squad:fido
PAO (DevRel) → label: squad:pao
CAPCOM (SDK Expert) → label: squad:capcom
CONTROL (TypeScript Engineer) → label: squad:control
Surgeon (Release Manager) → label: squad:surgeon
Booster (CI/CD Engineer) → label: squad:booster
GNC (Node.js Runtime) → label: squad:gnc
Network (Distribution) → label: squad:network
RETRO (Security) → label: squad:retro
INCO (CLI UX & Visual Design) → label: squad:inco
GUIDO (VS Code Extension) → label: squad:guido
Telemetry (Aspire & Observability) → label: squad:telemetry
VOX (REPL & Interactive Shell) → label: squad:vox
DSKY (TUI Engineer) → label: squad:dsky
Sims (E2E Test Engineer) → label: squad:sims
Handbook (SDK Usability) → label: squad:handbook
Ralph (Work Monitor) → label: squad:ralph
@copilot (Coding Agent) → label: squad:copilot

To reassign, remove the current squad:* label and add the correct one.

@sturlath

Hey @sturlath — thank you for this issue. You've nailed a real gap, and we love the workflow you described. Brady's reaction says it best: Squad. Does. Checklists. Let's make that real.

Here's a design proposal for how we'd tackle this.

🔍 Problem Statement

When an agent reports work as "done," the coordinator currently trusts that claim and moves on (collect results → log via Scribe → assess follow-ups). There's no step that independently verifies the agent actually completed what was asked — especially when the issue body contains a structured checklist (- [ ] items organized by phase).

This is a coordinator-level gap in the "After Agent Work" flow (the section between collecting results and spawning Scribe). The current flow has "silent success detection" for lost responses, but nothing for premature completion claims.

Your workflow — phased issues with checkboxes — is exactly the kind of structure that makes verification possible. The checkboxes are a contract. We should enforce it.

✅ Proposed Solution: Verification Phase

Add a new step to the coordinator's "After Agent Work" flow — Step 1b: Verification — that fires between collecting results and showing compact output.

How it works

Trigger condition: The coordinator checks whether the original issue body contains - [ ] checkboxes. If yes → run verification. If no checkboxes → skip (no overhead).
Spawn a verification agent — a lightweight, different agent than the one who did the work (fresh eyes catch what the doer missed):
- Model: claude-haiku-4.5 (cost-conscious — this is a checklist audit, not a creative task)
- Mode: sync (we need the answer before proceeding)
- No full ceremony needed — this is a focused, single-purpose spawn
The verifier checks three things:
- (a) Checklist coverage — Read the issue body, extract every - [ ] item. For each one, check whether the agent's work product (committed files, PR diff) actually addresses it.
- (b) Files exist & compile — Do the referenced/created files exist? Does the build pass?
- (c) Tests pass — If tests were part of the checklist or are affected by changes, do they pass?

Outcomes:

Verifier result	Coordinator action
✅ All items addressed	Mark done, proceed to Scribe & close issue
⚠️ Gaps found	Report gaps as a comment on the issue. Do NOT mark done. Route back to the original agent (or a different agent per lockout rules) with the gap list as context.
❌ Build/test failure	Same as gaps — report, don't close, re-route.

Re-check loop: After the re-routed agent completes its fix-up pass, verification runs again. This continues until either (a) all items pass or (b) a configurable retry limit is hit (default: 2 re-checks), at which point the coordinator escalates to the user.

🛠️ Implementation Sketch

Where it lives: New section in squad.agent.md under "After Agent Work", inserted as Step 1b — Verification (when issue has checklist) between the current Step 1 (collect results) and Step 2 (silent success detection).

Verification agent prompt structure:

You are a verification agent. Your ONLY job is to check completeness.

ISSUE BODY:
{issue_body}

AGENT WORK PRODUCT:
{summary of files changed, PR diff, or commit messages}

CHECKLIST ITEMS (extracted):
- [ ] Something 1
- [ ] Something 2
...

For EACH item: Does the work product address it? 
Report: ✅ addressed / ❌ not addressed (with explanation).
Final verdict: PASS or FAIL + gap list.

Key design choices:

Opt-in by structure — only triggers when - [ ] exists in the issue body. Zero overhead for issues without checklists.
Lightweight spawn — haiku model, sync mode, no Scribe logging needed for the verification step itself.
Independent verifier — a different agent context prevents the "I already think I did it" blind spot.
Retry cap — prevents infinite loops. Escalates to human after 2 re-verification attempts.

🚧 What This Doesn't Solve (Scope Boundaries)

Issues that are too big. You mentioned this yourself — and you're right. If an issue has 15+ checklist items across 5 phases, verification helps but the root cause is scope. We should document guidance on when to break into sub-issues. (Separate concern.)
Parallel work conflicts. When multiple agents work on overlapping code and merges go sideways — that's a coordination problem, not a verification problem. Different solution needed.
Cross-agent verification. This proposal covers single-agent "did you finish what you said you'd finish?" checks. Multi-agent orchestration verification (did agent A's work break agent B's?) is a bigger design space.

⏭️ Next Steps

Per our team's proposal-first workflow, this needs a formal spec in .squad/proposals/ before implementation. Here's the path:

Spec draft — Write the full proposal with edge cases, configuration options, and the exact prompt template
Team review — Flight + FIDO (quality) + Procedures (prompt engineering) weigh in
Implementation — Update squad.agent.md with the new verification step
Dog-food it — Test with phased checklist issues (like the pattern you described!) and measure completion accuracy before vs. after

We're tagging this go:spec-ready so it enters the proposal pipeline.

Thanks again for pushing on this, @sturlath. This is exactly the kind of feedback that makes Squad better. 🚀

🏷️ Triage verdict updated → go:spec-ready

feat: add gap analysis verification loop to After Agent Work flow

feat: add gap analysis verification loop to After Agent Work flow (#473)

fix: show buffered keystrokes during agent processing (github#457) (g…

Navigation Menu

Repository navigation

Gap analysis on done and not done work #472

Phase 1 Do something

Phase 2 Something else

🏗️ Squad Triage — Flight (Lead)

@copilot evaluation: No strong capability match — routed to squad member.

🔍 Problem Statement

✅ Proposed Solution: Verification Phase

How it works

🛠️ Implementation Sketch

🚧 What This Doesn't Solve (Scope Boundaries)

⏭️ Next Steps

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Gap analysis on done and not done work #472

Description

Phase 1 Do something

Phase 2 Something else

Activity

bradygaster commented on Mar 20, 2026

github-actions commented on Mar 20, 2026

🏗️ Squad Triage — Flight (Lead)

@copilot evaluation: No strong capability match — routed to squad member.

bradygaster commented on Mar 20, 2026

🔍 Problem Statement

✅ Proposed Solution: Verification Phase

How it works

🛠️ Implementation Sketch

🚧 What This Doesn't Solve (Scope Boundaries)

⏭️ Next Steps

github-actions commented on Mar 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions