Epic: WinCMux — voice-controlled multi-agent terminal multiplexer for Windows #1108

Overview

Build WinCMux — a Windows-native multiplexer for GitHub Copilot coding agent sessions with voice control, notification feed, and Teams chat integration. Think tmux-meets-voice-assistant for AI-assisted development workflows.

Problem Statement

When running multiple GitHub Copilot coding agents simultaneously, there is no unified way to:

• See what all agents are doing at a glance
• Switch between agent sessions hands-free
• Dictate instructions/context to a specific agent via voice
• Get notified when an agent needs input or completes a task
• Manage incoming Teams messages and respond without leaving the dev context

Goals

• Multi-agent workspace — run N GitHub Copilot coding agent sessions in a tiled/tabbed terminal layout (cmux-style)
• Voice control — wake-word or push-to-talk to switch focus to a pane and dictate instructions
• Notification feed — surfaced sidebar showing agent status updates, PR events, CI results, mentions
• Teams integration — read incoming chats, compose replies, route messages to the right agent context
• Windows-first — native Windows Terminal / WinUI or TUI (Spectre.Console / Terminal.Gui) experience

Proposed Architecture

Core Components

┌─────────────────────────────────────────────────────────┐
│  WinCMux Shell                                          │
│  ┌──────────────────┐  ┌──────────────────┐            │
│  │  Agent Pane 1    │  │  Agent Pane 2    │  ...       │
│  │  copilot #42     │  │  copilot #87     │            │
│  │  [working...]    │  │  [waiting...]    │            │
│  └──────────────────┘  └──────────────────┘            │
│  ┌────────────────────────────────────────────────────┐ │
│  │  Notification Feed  │  🎤 Voice Input  │  Teams    │ │
│  └────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘

Technology Candidates

User Experience

Voice Commands

"Hey Cmux, switch to agent two"          → focus pane 2
"Hey Cmux, tell agent one: [dictation]"  → type dictation into pane 1
"Hey Cmux, show notifications"           → expand notification sidebar
"Hey Cmux, reply to John in Teams: ..."  → compose Teams reply
"Hey Cmux, merge PR forty-two"           → run gh pr merge 42

Notification Feed Items

• 🤖 Agent completed task (chore: remove internal team-docs from main #42 — fix auth endpoint)
• 🔴 CI failed on PR squad- workflows seem to assume a particular project type? #87 (click to jump to agent)
• 💬 John in Teams: "Did you get a chance to look at…"
• ✅ PR chore: remove internal team-docs from main #42 approved — ready to merge
• 📋 New issue assigned: Workflow dual-path support for .squad/ migration #103

Acceptance Criteria

MVP (Phase 1)

• Launch 2+ gh copilot / copilot CLI sessions in split panes
• Keyboard switching between panes (Ctrl+1, Ctrl+2, ...)
• Notification sidebar polling GitHub events (issues, PRs, CI) via gh
• Voice wake-word detection + dictation-to-pane
• Windows 11 compatible, runs in Windows Terminal

Phase 2

• Teams chat integration (read + reply via Graph API)
• Voice commands for gh actions (merge, close, comment)
• Agent status inference (idle vs. working vs. waiting for input)
• Persist session layout across restarts

Phase 3

• Plugin model for other notification sources (GitHub MCP, Jira, Linear)
• Voice profile per agent pane (different wake phrases)
• Packaged installer (winget / MSIX)

Open Questions / Research Needed

• ConPTY viability — can we reliably multiplex copilot CLI sessions via ConPTY without PTY escape sequence conflicts?
• STT latency — is Windows Speech Recognition API fast enough, or do we need Azure Cognitive Services / local Whisper?
• Teams auth flow — OAuth device flow or service principal? Impact on consumer vs. enterprise accounts?
• TUI vs. WinUI — full GUI (WinUI 3) or stay terminal-native (Spectre.Console)? Trade-offs for voice overlay UX?
• Wake-word licensing — Picovoice Porcupine (MIT for open source), Azure custom keyword, or Windows built-in?

Suggested Squad Assignments

Labels

type:epic, type:feature, priority:p1, squad, go:needs-research

Related

• tmux — inspiration for pane model
• Windows Terminal — host shell
• Spectre.Console — TUI candidate
• Azure Cognitive Services Speech SDK
• Microsoft Graph API — Teams