diff --git a/.changeset/docs-state-backends.md b/.changeset/docs-state-backends.md index b391687af..5378b0cef 100644 --- a/.changeset/docs-state-backends.md +++ b/.changeset/docs-state-backends.md @@ -2,4 +2,4 @@ '@bradygaster/squad-cli': patch --- -Add user-facing documentation for state backends (git-notes, orphan-branch, worktree) +Add user-facing documentation for state backends (local, orphan-branch, two-layer) diff --git a/.changeset/git-state-backends.md b/.changeset/git-state-backends.md index 994e03465..c3a4582dd 100644 --- a/.changeset/git-state-backends.md +++ b/.changeset/git-state-backends.md @@ -5,13 +5,13 @@ feat(sdk): git-notes + orphan-branch state backends for .squad/ -Adds two git-native state storage backends as alternatives to the worktree -and external directory approaches: +Adds git-native state storage backends as alternatives to the local (disk) +approach: -- **git-notes** (`refs/notes/squad`): State stored in git notes ref. Survives - branch switches, invisible in diffs and PRs. - **orphan-branch** (`squad-state`): Dedicated orphan branch with no common ancestor. State files never appear in main. +- **two-layer** (notes + orphan): Git notes as best-effort commit annotations + plus orphan branch for durable state. Recommended for teams. -Configure via `.squad/config.json`: `{ "stateBackend": "git-notes" }` or +Configure via `.squad/config.json`: `{ "stateBackend": "two-layer" }` or the `--state-backend` CLI flag. \ No newline at end of file diff --git a/.changeset/state-backend-global.md b/.changeset/state-backend-global.md new file mode 100644 index 000000000..61906e8bf --- /dev/null +++ b/.changeset/state-backend-global.md @@ -0,0 +1,21 @@ +--- +'@bradygaster/squad-sdk': minor +'@bradygaster/squad-cli': patch +--- + +Add delete() and append() to StateBackend interface; add resolveSquadState() entry point; add StateBackendStorageAdapter; add git-notes agent protocol templates + +- Add `delete()` and `append()` methods to the `StateBackend` interface +- Implement delete/append for all three backends (local, orphan, two-layer) +- Fix orphan backend append to preserve trailing whitespace via readUntrimmed() +- Add `SquadStateContext` interface and `resolveSquadState()` factory in resolution module +- Add `StateBackendStorageAdapter` — bridges StateBackend as a StorageProvider so SDK modules that accept `storage: StorageProvider` work with git-notes and orphan backends +- Add `storage` field to `SquadStateContext` — local uses FSStorageProvider directly, orphan/two-layer use the adapter +- Export `StateBackendStorageAdapter` from SDK public API +- Wire `resolveSquadState()` into CLI entry so the state backend is resolved once at startup +- Pass pre-resolved `stateContext` through to watch config to avoid redundant resolution +- Add `notes-protocol.md` template — agent contract for git-notes state (namespaces, JSON schema, fetch/push, conflict handling) +- Add `scripts/notes/fetch.ps1` template — fetch notes + one-time refspec setup + merge after conflict +- Add `scripts/notes/write-note.ps1` template — agent helper for writing notes with JSON validation and push retry +- Update state-backends docs with "Using with Copilot CLI Sessions" section (copilot-instructions snippet, promotion flow, template index) +- This is the SDK foundation for making state backends squad-wide (Phase 1) diff --git a/.changeset/watch-p0-p1-fixes.md b/.changeset/watch-p0-p1-fixes.md index fce11b953..0e8379f94 100644 --- a/.changeset/watch-p0-p1-fixes.md +++ b/.changeset/watch-p0-p1-fixes.md @@ -5,6 +5,6 @@ fix(watch): wire missing CLI flags to config, validate --state-backend, fix auth stderr parsing (#834) - Wire --notify-level, --overnight-start, --overnight-end, --sentinel-file to WatchConfig and loadWatchConfig merge -- Add --state-backend flag with upfront validation against allowed values (worktree, git-notes, orphan, external) +- Add --state-backend flag with upfront validation against allowed values (local, orphan, two-layer, external) - Fix probeCurrentGhUser() to use `gh api user -q .login` (stdout) instead of parsing `gh auth status` stderr - Also wire authUser through loadWatchConfig merge logic (was accepted but silently dropped) \ No newline at end of file diff --git a/.github/agents/squad.agent.md b/.github/agents/squad.agent.md index 01e18dfad..cb2ed5d46 100644 --- a/.github/agents/squad.agent.md +++ b/.github/agents/squad.agent.md @@ -3,14 +3,14 @@ name: Squad description: "Your AI team. Describe what you're building, get a team of specialists that live in your repo." --- - + You are **Squad (Coordinator)** — the orchestrator for this project's AI team. ### Coordinator Identity - **Name:** Squad (Coordinator) -- **Version:** 0.0.0-source (see HTML comment above — this value is stamped during install/upgrade). Include it as `Squad v{version}` in your first response of each session (e.g., in the acknowledgment or greeting). +- **Version:** 0.9.1 (see HTML comment above — this value is stamped during install/upgrade). Include it as `Squad v0.9.1` in your first response of each session (e.g., in the acknowledgment or greeting). - **Role:** Agent orchestration, handoff enforcement, reviewer gating - **Inputs:** User request, repository state, `.squad/decisions.md` - **Outputs owned:** Final assembled artifacts, orchestration log (via Scribe) @@ -107,6 +107,8 @@ The `union` merge driver keeps all lines from both sides, which is correct for a **On every session start:** Run `git config user.name` to identify the current user, and **resolve the team root** (see Worktree Awareness). Store the team root — all `.squad/` paths must be resolved relative to it. Pass the team root and the current datetime (from `` in your system context) into every spawn prompt as `TEAM_ROOT` and `CURRENT_DATETIME` respectively. Pass the current user's name into every agent spawn prompt and Scribe log so the team always knows who requested the work. Check `.squad/identity/now.md` if it exists — it tells you what the team was last focused on. Update it if the focus has shifted. +**Resolve state backend:** Read `.squad/config.json` and check the `stateBackend` field. Valid values: `"worktree"` (default), `"git-notes"`, `"orphan"`, `"two-layer"`. Store as `STATE_BACKEND` and pass it into every spawn prompt. This determines how agents read and write mutable state (history, decisions, logs). Static config (charters, team.md, routing.md) always lives on disk regardless of backend. The `"two-layer"` option combines git-notes (commit-scoped annotations) with orphan branch (permanent state) — see the blog post for the full architecture. + **⚡ Context caching:** After the first message in a session, `team.md`, `routing.md`, and `registry.json` are already in your context. Do NOT re-read them on subsequent messages — you already have the roster, routing rules, and cast names. Only re-read if the user explicitly modifies the team (adds/removes members, changes routing). **Session catch-up (lazy — not on every start):** Do NOT scan logs on every session start. Only provide a catch-up summary when: @@ -223,13 +225,16 @@ The `name` parameter generates the human-readable agent ID shown in the tasks pa **When you detect a directive:** -1. Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: - ``` - ### {timestamp}: User directive - **By:** {user name} (via Copilot) - **What:** {the directive, verbatim or lightly paraphrased} - **Why:** User request — captured for team memory - ``` +1. Capture the directive: + - **worktree/orphan backend:** Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: + ``` + ### {timestamp}: User directive + **By:** {user name} (via Copilot) + **What:** {the directive, verbatim or lightly paraphrased} + **Why:** User request — captured for team memory + ``` + - **git-notes backend:** Persist via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/directives" -Content '{"timestamp": "{timestamp}", "by": "{user name}", "what": "...", "why": "User request"}'` 2. Acknowledge briefly: `"📌 Captured. {one-line summary of the directive}."` 3. If the message ALSO contains a work request, route that work normally after capturing. If it's directive-only, you're done — no agent spawn needed. @@ -344,7 +349,12 @@ prompt: | TARGET FILE(S): {exact file path(s)} Do the work. Keep it focused. + {% if STATE_BACKEND == "git-notes" %} + If you made a meaningful decision, persist it via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"decision": {"title": "...", "what": "...", "why": "..."}}'` + {% else %} If you made a meaningful decision, write to .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} ⚠️ OUTPUT: Report outcomes in human terms. Never expose tool internals or SQL. ⚠️ RESPONSE ORDER: After ALL tool calls, write a plain text summary as FINAL output. @@ -598,8 +608,9 @@ When the user gives any task, the Coordinator MUST: To enable full parallelism, shared writes use a drop-box pattern that eliminates file conflicts: **decisions.md** — Agents do NOT write directly to `decisions.md`. Instead: -- Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` -- Scribe merges inbox entries into the canonical `.squad/decisions.md` and clears the inbox +- **worktree/orphan backend:** Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` +- **git-notes backend:** Agents persist decisions via their git notes ref (see State Protocol). Scribe reads all agent notes and merges decisions. +- Scribe merges into the canonical `.squad/decisions.md` and clears the inbox (or note entries) - All agents READ from `.squad/decisions.md` at spawn time (last-merged snapshot) **orchestration-log/** — Scribe writes one entry per agent after each batch: @@ -799,7 +810,71 @@ prompt: | - Commit and push from the worktree {% endif %} + STATE_BACKEND: {state_backend} + + {% if STATE_BACKEND == "git-notes" %} + ## State Protocol — Git Notes + This project uses git-notes for mutable state. **DO NOT write to `.squad/` files for state.** + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading your state:** + Run: `powershell .squad/scripts/notes/fetch.ps1 -Setup` (first time per session) + Then: `git notes --ref=squad/{name} show $(git rev-list --max-parents=0 HEAD) 2>$null` + Falls back to empty if no note exists. + + **Writing state (history, decisions, learnings):** + Run: `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{json}'` + The helper handles JSON validation, conflict retry, and push. + + **Decisions:** Write decisions as JSON via your note ref. Scribe will merge them. + **Skills:** Skills are static config — write to `.squad/skills/` on disk as normal. + {% endif %} + + {% if STATE_BACKEND == "orphan" %} + ## State Protocol — Orphan Branch + This project uses an orphan branch (`squad-state`) for mutable state. + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading state:** Read `.squad/` files on disk — they are synced from the orphan branch. + **Writing state:** Write to `.squad/` files on disk as normal during your session. + Scribe will commit your changes to the orphan branch (not the working branch) and + ensure they persist across branch switches. + + **Important:** Do NOT commit `.squad/` state files to the working branch yourself. + Scribe handles the orphan branch commit workflow. + {% endif %} + + {% if STATE_BACKEND == "two-layer" %} + ## State Protocol — Two-Layer (Git Notes + Orphan Branch) + This project uses the two-layer architecture from Tamir's blog: + - **Layer 1 (git notes):** Commit-scoped "why" annotations — invisible in PRs + - **Layer 2 (orphan branch):** Permanent state store — decisions, histories, logs + + Static config (charters, team.md, routing.md) is on disk as normal. + + **During your session:** + 1. Write commit-scoped annotations as git notes on HEAD: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + 2. Write bulk state (history, logs) to `.squad/` files on disk — Scribe moves them to the orphan branch. + + **Note flags:** + - `"promote_to_permanent": true` — Ralph promotes this to decisions.md after PR merge + - `"archive_on_close": true` — Worth keeping even if PR is rejected (valuable research) + - Neither flag — silently ignored if PR is rejected (correct for branch-specific decisions) + + **Important:** Do NOT commit `.squad/` state files to the working branch. + Scribe handles orphan commits. Ralph handles note promotion. + {% endif %} + + {% if STATE_BACKEND == "worktree" or STATE_BACKEND is not defined %} Read .squad/agents/{name}/history.md (your project knowledge). + {% endif %} + {% if STATE_BACKEND == "git-notes" %} + Read your agent state from git notes (see State Protocol above). + {% endif %} + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + Read .squad/agents/{name}/history.md (your project knowledge — synced from orphan branch). + {% endif %} Read .squad/decisions.md (team decisions to respect). If .squad/identity/wisdom.md exists, read it before starting work. If .squad/identity/now.md exists, read it at spawn time. @@ -823,10 +898,27 @@ prompt: | ⚠️ DATES: When writing dates in any file (decisions, history, logs), use ONLY the CURRENT_DATETIME value above. Never infer or guess the date. AFTER work: + {% if STATE_BACKEND == "git-notes" %} + 1. Persist your learnings as JSON via the State Protocol: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"learnings": ["..."], "timestamp": "{current_datetime}"}'` + 2. If you made a team-relevant decision, include it in the JSON: + Add a `"decision"` field with `"title"`, `"what"`, and `"why"` keys. + Scribe will merge decisions into the canonical decisions.md. + {% elif STATE_BACKEND == "two-layer" %} + 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": + architecture decisions, patterns, user preferences, key file paths. + (Scribe commits this to the orphan branch.) + 2. If you made a team-relevant decision, write BOTH: + a. A git note on HEAD with promote flag: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + b. A drop file: .squad/decisions/inbox/{name}-{brief-slug}.md + (Scribe merges to orphan branch; Ralph promotes note after PR merge.) + {% else %} 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": architecture decisions, patterns, user preferences, key file paths. 2. If you made a team-relevant decision, write to: .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} 3. SKILL EXTRACTION: If you found a reusable pattern, write/update .squad/skills/{skill-name}/SKILL.md (read templates/skill.md for format). @@ -878,18 +970,47 @@ prompt: | You are the Scribe. Read .squad/agents/scribe/charter.md. TEAM ROOT: {team_root} CURRENT_DATETIME: {current_datetime} + STATE_BACKEND: {state_backend} SPAWN MANIFEST: {spawn_manifest} Tasks (in order): - 0. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "git-notes" or STATE_BACKEND == "two-layer" %} + 0. PRE-CHECK — STATE LEAK GUARD: Check if any agent accidentally committed or staged state files + (.squad/decisions.md, agents/*/history.md, log/*, orchestration-log/*, decisions/inbox/*) + to the working branch. If found: unstage with `git reset HEAD -- {file}`, restore with + `git checkout HEAD -- {file}`. If leaked in last commit, amend to remove. Log count. + {% endif %} + 0b. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. 1. DECISIONS ARCHIVE [HARD GATE]: If decisions.md >= 20480 bytes, archive entries older than 30 days NOW. If >= 51200 bytes, archive entries older than 7 days. Do not skip this step. + {% if STATE_BACKEND == "git-notes" %} + 2. DECISION MERGE (git-notes): For each agent ref `squad/{agent}`, read notes via `git notes --ref=squad/{agent} show $(git rev-list --max-parents=0 HEAD)`. Extract any `decision` entries. Merge into decisions.md. Clear the decision field by overwriting the note without it. + {% elif STATE_BACKEND == "two-layer" %} + 2. DECISION MERGE (two-layer): Merge .squad/decisions/inbox/ → decisions.md AND read agent note refs for any decisions with `promote_to_permanent`. Deduplicate. Push note refs: `git push origin 'refs/notes/squad/*'` + {% else %} 2. DECISION INBOX: Merge .squad/decisions/inbox/ → decisions.md, delete inbox files. Deduplicate. + {% endif %} 3. ORCHESTRATION LOG: Write .squad/orchestration-log/{timestamp}-{agent}.md per agent. Use ISO 8601 UTC timestamp. 4. SESSION LOG: Write .squad/log/{timestamp}-{topic}.md. Brief. Use ISO 8601 UTC timestamp. + {% if STATE_BACKEND == "git-notes" %} + 5. CROSS-AGENT (git-notes): For team updates, write to affected agents' note refs via `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{agent}" -Content '{json}'`. + {% else %} 5. CROSS-AGENT: Append team updates to affected agents' history.md. + {% endif %} 6. HISTORY SUMMARIZATION [HARD GATE]: If any history.md >= 15360 bytes (15KB), summarize now. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + 7. GIT COMMIT (orphan): Stage `.squad/` state files and commit to the `squad-state` orphan branch: + a. Identify changed `.squad/` state files via `git status --porcelain` (decisions.md, agents/*/history.md, log/*, orchestration-log/*). + b. For each file, use git plumbing to write to the orphan branch: + `git show squad-state:.squad/{path}` to check if file exists on orphan. + Use `git checkout squad-state -- .squad/{path}` + write + `git add` + `git commit` workflow, OR + use the SDK's OrphanBranchBackend if available. + c. Reset working tree state files: `git checkout HEAD -- .squad/` to avoid polluting the working branch. + d. Push orphan branch: `git push origin squad-state` + ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + {% else %} 7. GIT COMMIT: Stage only the exact `.squad/` files Scribe wrote in this session. Use `git status --porcelain` filtered to allowed paths (decisions.md, decisions-archive.md, agents/{name}/history.md, agents/{name}/history-archive.md, log/*, orchestration-log/*). Stage each file individually with `git add -- `. Handle renames by extracting destination path (`-replace '^.* -> ',''`). Commit with -F (write msg to temp file). Skip if nothing staged. ⚠️ NEVER use `git add .squad/` or broad globs. + {% endif %} 8. HEALTH REPORT: Log decisions.md before/after size, inbox count processed, history files summarized. Never speak to user. ⚠️ End with plain text summary after all tool calls. @@ -947,6 +1068,8 @@ If the user wants to remove someone: ## Source of Truth Hierarchy +> **State backend note:** Files below marked as "Derived / append-only" are **mutable state** — their storage location depends on the configured `STATE_BACKEND`. On `worktree` (default), they live on disk. On `git-notes`, they live in git notes refs. On `orphan`, they live on the `squad-state` orphan branch. On `two-layer`, commit-scoped annotations live in git notes AND permanent state lives on the orphan branch. Files marked as "Authoritative" are **static config** and always live on disk regardless of backend. + | File | Status | Who May Write | Who May Read | |------|--------|---------------|--------------| | `.github/agents/squad.agent.md` | **Authoritative governance.** All roles, handoffs, gates, and enforcement rules. | Repo maintainer (human) | Squad (Coordinator) | diff --git a/.squad-templates/notes-protocol.md b/.squad-templates/notes-protocol.md new file mode 100644 index 000000000..5a1d03a5f --- /dev/null +++ b/.squad-templates/notes-protocol.md @@ -0,0 +1,202 @@ +# Squad Notes Protocol + +> Contract for agent state via git notes. Agents write commit-scoped context +> here instead of modifying `.squad/` files in PRs. +> +> **Version:** 1.0 +> **Backends:** `git-notes`, `orphan` + +--- + +## Overview + +Squad state has two layers: + +1. **Git notes layer** (this document) — thin, commit-scoped annotations that + attach agent context to commits without appearing in PRs or diffs. +2. **Permanent state layer** — long-lived decisions, routing rules, and archives + stored via the configured state backend (`git-notes` or `orphan` branch). + +Agents write notes during their work rounds. Ralph promotes flagged notes to +permanent state after a PR merges. + +--- + +## Namespaces + +Each agent writes to its own namespace to prevent conflicts: + +| Namespace | Owner | Purpose | +|-----------|-------|---------| +| `refs/notes/squad/data` | Data | Architecture decisions, implementation choices | +| `refs/notes/squad/worf` | Worf | Security reviews, vulnerability assessments | +| `refs/notes/squad/seven` | Seven | Documentation quality, API contract decisions | +| `refs/notes/squad/ralph` | Ralph | Work-round progress, task-state annotations | +| `refs/notes/squad/q` | Q | Devil's advocate findings, risk assessments | +| `refs/notes/squad/research` | Any agent | Research notes that should survive branch deletion | +| `refs/notes/squad/review` | Any agent | Code review context (mirrors Gerrit's pattern) | + +**Rule**: Only write to your own namespace. The shared namespaces +(`research`, `review`) use `append` — never `add`. + +--- + +## Note JSON Schema + +All notes MUST be valid JSON. Minimum required fields: + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision | research | review | progress | security", + "content": "..." +} +``` + +### Decision notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision", + "decision": "Use JWT RS256 for auth middleware", + "reasoning": "Existing pattern in codebase — auth.go:47-89.", + "alternatives_considered": ["HS256", "session tokens"], + "confidence": "high", + "promote_to_permanent": true +} +``` + +Set `"promote_to_permanent": true` to signal Ralph to copy this to +`decisions.md` after the PR merges. + +### Research notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "research", + "topic": "JWT vs session tokens", + "findings": {}, + "effort_hours": 2.5, + "archive_on_close": true +} +``` + +Set `"archive_on_close": true` to signal Ralph to archive this to +`state/research/` even if the PR is rejected. + +--- + +## Write Commands + +```bash +# Write a decision note on the current commit +git notes --ref=squad/{your-agent} add \ + -m '{"agent":"{Agent}","timestamp":"...","type":"decision","decision":"..."}' \ + HEAD + +# Append to an existing note (multiple items on same commit) +git notes --ref=squad/{your-agent} append \ + -m '{"agent":"{Agent}","timestamp":"...","type":"progress","content":"..."}' \ + HEAD + +# Read your note +git notes --ref=squad/{your-agent} show HEAD + +# List all commits with notes in your namespace +git notes --ref=squad/{your-agent} list +``` + +Or use the helper script: + +```powershell +./scripts/notes/write-note.ps1 -Agent data -Type decision \ + -Content '{"decision":"Use JWT","reasoning":"..."}' \ + [-Commit HEAD] [-Promote] [-Archive] +``` + +--- + +## Fetch / Push + +**Notes are NOT fetched or pushed by default.** Every clone needs setup. + +### One-time setup + +```bash +git config --add remote.origin.fetch 'refs/notes/*:refs/notes/*' +git fetch origin 'refs/notes/*:refs/notes/*' +``` + +Or use the helper: + +```powershell +./scripts/notes/fetch.ps1 -Setup +``` + +### Every work round + +1. **Start**: `git fetch origin 'refs/notes/*:refs/notes/*'` +2. **End**: `git push origin 'refs/notes/*:refs/notes/*'` + +--- + +## Conflict Handling + +1. **Per-agent namespaces prevent 99% of conflicts.** Only one agent writes to + `refs/notes/squad/data`, so there are no write conflicts in normal use. + +2. **Same agent, two machines:** First push wins. Losing machine should fetch + and append: + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes --ref=squad/{agent} append -m '{...}' HEAD + git push origin 'refs/notes/*:refs/notes/*' + ``` + +3. **Shared namespaces** (`research`, `review`): Always use `git notes append`, + never `git notes add`. + +4. **Push conflict recovery:** + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes merge refs/notes/remotes/origin/squad/{namespace} + git push origin 'refs/notes/*:refs/notes/*' + ``` + +--- + +## When to Use Notes vs State Backend + +| Use git notes | Use state backend | +|---------------|-------------------| +| Why THIS choice on THIS commit | Universal routing rules, conventions | +| Decisions scoped to a feature | Long-lived decisions for all future work | +| Research for a specific investigation | Research archives (promoted from notes) | +| Security sign-offs per commit | Agent history persisting across features | +| Agent-to-agent context for current feature | Team agreements and policies | + +When in doubt: **notes first, promote to permanent state later.** Ralph handles +the promotion automatically when `promote_to_permanent` is set. + +--- + +## Ralph Promotion Rules + +**After PR merge:** + +1. Fetch all notes from remote +2. Traverse commits reachable from the default branch that have notes +3. For each note with `"promote_to_permanent": true` → append to `decisions.md` +4. Push state + +**After PR close/rejection:** + +1. List notes in `squad/research` on the closed branch's commits +2. For each note with `"archive_on_close": true` → archive to `research/` +3. Push state +4. Notes on rejected commits are NOT promoted — this is the desired behavior diff --git a/.squad-templates/scribe-charter.md b/.squad-templates/scribe-charter.md index 308b780e2..a2031d072 100644 --- a/.squad-templates/scribe-charter.md +++ b/.squad-templates/scribe-charter.md @@ -24,6 +24,59 @@ **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory). +**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. If it's `"orphan"` or `"git-notes"`, run the **State Leak Guard** before any other work. + +### State Leak Guard (orphan/git-notes backends only) + +Before logging or merging, check if any agent accidentally committed state files to the working branch: + +```powershell +# Check if state files are staged or committed but shouldn't be +$stateFiles = @( + '.squad/decisions.md', + '.squad/decisions-archive.md' +) +$statePatterns = @( + '.squad/agents/*/history.md', + '.squad/agents/*/history-archive.md', + '.squad/log/*', + '.squad/orchestration-log/*', + '.squad/decisions/inbox/*' +) + +# 1. Check git status for accidentally staged state files +$dirty = git status --porcelain | Where-Object { $_.Length -gt 3 } | ForEach-Object { + $_.Substring(3) -replace '^.* -> ','' +} | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($dirty) { + # Unstage any accidentally added state files + $dirty | ForEach-Object { git reset HEAD -- $_ 2>$null } + # Restore from HEAD (discard working tree changes for state files) + $dirty | ForEach-Object { git checkout HEAD -- $_ 2>$null } +} + +# 2. Check if the most recent commit on this branch has state files +$lastCommitFiles = git diff-tree --no-commit-id --name-only -r HEAD 2>$null +$leakedInCommit = $lastCommitFiles | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($leakedInCommit) { + # State files leaked into the last commit — amend to remove them + $leakedInCommit | ForEach-Object { git rm --cached -- $_ 2>$null } + git commit --amend --no-edit 2>$null +} +``` + +If any files were cleaned, log: `⚠️ State leak guard: removed {N} state file(s) from working branch.` + +After the guard, proceed with normal Scribe work (but persist state via the configured backend, not the working branch). + After every substantial work session: 1. **Log the session** to `.squad/log/{timestamp}-{topic}.md`: @@ -57,9 +110,50 @@ After every substantial work session: ``` 5. **Commit `.squad/` changes:** + **Check `STATE_BACKEND` from spawn prompt.** This determines WHERE state gets committed. + **IMPORTANT — Windows compatibility:** Do NOT use `git -C {path}` (unreliable with Windows paths). Do NOT embed newlines in `git commit -m` (backtick-n fails silently in PowerShell). - Instead: + + **If STATE_BACKEND is "orphan":** + State files must be committed to the `squad-state` orphan branch, NOT the working branch. + - Identify changed `.squad/` state files via `git status --porcelain` filtered to allowed paths. + - For each file, use git plumbing to write to the orphan branch: + ```powershell + # Create a temporary worktree for the orphan branch + $orphanWt = Join-Path ([System.IO.Path]::GetTempPath()) "squad-state-$(Get-Random)" + git worktree add $orphanWt squad-state 2>$null + if ($LASTEXITCODE -ne 0) { git worktree add --orphan $orphanWt squad-state } + # Copy state files to orphan worktree + $filesToSync | ForEach-Object { + $dest = Join-Path $orphanWt $_ + New-Item -ItemType Directory -Path (Split-Path $dest) -Force | Out-Null + Copy-Item $_ $dest -Force + } + # Commit in orphan worktree + Push-Location $orphanWt + git add .squad/ + git diff --cached --quiet + if ($LASTEXITCODE -ne 0) { + $msgFile = [System.IO.Path]::GetTempFileName() + Set-Content -Path $msgFile -Value "docs(ai-team): $summary" -Encoding utf8 + git commit -F $msgFile + Remove-Item $msgFile + git push origin squad-state + } + Pop-Location + git worktree remove $orphanWt --force + ``` + - After committing to orphan, reset working tree state files: `git checkout HEAD -- .squad/` + - ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + + **If STATE_BACKEND is "git-notes":** + State is already persisted in git notes refs by agents. Scribe only needs to: + - Push any locally created note refs: `git push origin 'refs/notes/squad/*'` + - Commit decisions.md (the merged canonical file) to the working branch as normal. + + **If STATE_BACKEND is "worktree" (default):** + Commit to the working branch as normal: - `cd` into the team root first. - Stage only files Scribe actually modified in this session. Use `git status --porcelain` to build an explicit file list filtered to allowed `.squad/` paths: diff --git a/.squad-templates/scripts/notes/fetch.ps1 b/.squad-templates/scripts/notes/fetch.ps1 new file mode 100644 index 000000000..5adc19480 --- /dev/null +++ b/.squad-templates/scripts/notes/fetch.ps1 @@ -0,0 +1,88 @@ +#!/usr/bin/env pwsh +# scripts/notes/fetch.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Fetch git notes from remote. Run on every Ralph-watch startup and before +# any agent reads or writes notes. +# +# Usage: +# ./scripts/notes/fetch.ps1 # fetch only +# ./scripts/notes/fetch.ps1 -Setup # first-time: add refspec + fetch +# ./scripts/notes/fetch.ps1 -Merge # fetch + merge (use after push conflict) +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [string]$Remote = "origin", + [string]$RepoPath = ".", + [switch]$Setup, + [switch]$Merge, + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/fetch] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath + +# ── One-time setup: add fetch refspec ────────────────────────────────────── +if ($Setup) { + $existing = git -C $repo config --get-all "remote.$Remote.fetch" 2>&1 | + Where-Object { $_ -match "refs/notes" } + if ($existing) { + Log "Notes refspec already configured." DarkGray + } else { + git -C $repo config --add "remote.$Remote.fetch" "refs/notes/*:refs/notes/*" + Log "Added notes refspec to remote.$Remote.fetch" Green + } +} + +# ── Fetch notes ───────────────────────────────────────────────────────────── +Log "Fetching notes from $Remote..." +$output = git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 +if ($LASTEXITCODE -ne 0) { + Log "Fetch warning: $output" DarkYellow +} else { + Log "Notes fetched." Green +} + +# ── Merge notes if requested (after push conflict) ────────────────────────── +if ($Merge) { + # Abort any stale merge-in-progress state + $mergeLock = Join-Path $repo ".git/NOTES_MERGE_PARTIAL" + if (Test-Path $mergeLock) { + Log "Stale notes merge in progress — aborting before retry" DarkYellow + git -C $repo notes merge --abort 2>&1 | Out-Null + } + + $namespaces = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + foreach ($ref in $namespaces) { + $ns = $ref -replace "refs/notes/", "" + $remoteRef = "refs/notes/remotes/$Remote/$ns" + $remoteExists = git -C $repo for-each-ref $remoteRef --format="%(refname)" 2>&1 + if ($remoteExists) { + Log "Merging notes: $ns (cat_sort_uniq)" + git -C $repo notes --ref=$ns merge -s cat_sort_uniq $remoteRef 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Log " Merge failed on $ns — aborting and continuing" Red + git -C $repo notes merge --abort 2>&1 | Out-Null + } + } + } + Log "Notes merge complete." Green +} + +# ── Show available namespaces ──────────────────────────────────────────────── +if (-not $Quiet) { + $refs = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + if ($refs) { + Log "Available namespaces:" + foreach ($r in $refs) { + $count = (git -C $repo notes --ref=($r -replace "refs/notes/","") list 2>&1 | + Where-Object { $_ -ne "" } | Measure-Object -Line).Lines + Log " $r ($count notes)" DarkGray + } + } else { + Log "No squad notes yet." DarkGray + } +} diff --git a/.squad-templates/scripts/notes/write-note.ps1 b/.squad-templates/scripts/notes/write-note.ps1 new file mode 100644 index 000000000..dbc719d35 --- /dev/null +++ b/.squad-templates/scripts/notes/write-note.ps1 @@ -0,0 +1,126 @@ +#!/usr/bin/env pwsh +# scripts/notes/write-note.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Helper for agents to write notes without wrestling with JSON escaping. +# Validates namespace ownership, handles conflicts, pushes automatically. +# +# Usage: +# ./scripts/notes/write-note.ps1 -Agent data -Type decision \ +# -Content '{"decision":"Use JWT","reasoning":"..."}' \ +# [-Commit HEAD] [-Promote] [-Archive] +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [Parameter(Mandatory)][string]$Agent, + + [Parameter(Mandatory)] + [ValidateSet("decision","research","review","security-review","progress", + "api-contract","risk-assessment","routing-discovery","counter-argument")] + [string]$Type, + + [Parameter(Mandatory)] + [string]$Content, # JSON object with type-specific fields + + [string]$Commit = "HEAD", + [string]$RepoPath = ".", + [string]$Remote = "origin", + [switch]$Promote, # set promote_to_permanent: true + [switch]$Archive, # set archive_on_close: true + [switch]$NoPush, # skip auto-push + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/write] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath +$namespace = "squad/$($Agent.ToLower())" + +# ── Validate JSON content ──────────────────────────────────────────────────── +try { + $parsed = $Content | ConvertFrom-Json -ErrorAction Stop +} catch { + Write-Error "Content must be valid JSON. Got: $Content" + exit 1 +} + +# ── Build full note object ──────────────────────────────────────────────────── +$note = [ordered]@{ + agent = (Get-Culture).TextInfo.ToTitleCase($Agent.ToLower()) + timestamp = [System.DateTime]::UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") + type = $Type +} + +# Merge content fields into note +$parsed.PSObject.Properties | ForEach-Object { $note[$_.Name] = $_.Value } + +# Add flag fields +if ($Promote) { $note["promote_to_permanent"] = $true } +if ($Archive) { $note["archive_on_close"] = $true } + +$noteJson = $note | ConvertTo-Json -Compress -Depth 10 + +# ── Fetch first to avoid conflicts ─────────────────────────────────────────── +Log "Fetching notes before write..." +git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 | Out-Null + +# ── Check if note already exists on this commit ───────────────────────────── +$existing = git -C $repo notes --ref=$namespace show $Commit 2>&1 +$useAppend = ($LASTEXITCODE -eq 0) + +if ($useAppend) { + Log "Note exists on $Commit — appending" DarkYellow + git -C $repo notes --ref=$namespace append -m $noteJson $Commit +} else { + git -C $repo notes --ref=$namespace add -m $noteJson $Commit +} + +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to write note to refs/notes/$namespace on $Commit" + exit 1 +} + +Log "Note written to refs/notes/$namespace on $($Commit.Substring(0,[Math]::Min(8,$Commit.Length)))" Green + +# ── Push with retry ────────────────────────────────────────────────────────── +if (-not $NoPush) { + $maxRetries = 5 + $nsRef = "refs/notes/$namespace" + + for ($i = 0; $i -lt $maxRetries; $i++) { + Log "Pushing notes (attempt $($i+1))..." + $pushOut = git -C $repo push $Remote "${nsRef}:${nsRef}" 2>&1 + if ($LASTEXITCODE -eq 0) { + Log "Notes pushed successfully." Green + break + } + + if ($pushOut -match "non-fast-forward|fetch first|rejected") { + Log "Push conflict — fetch-first retry..." DarkYellow + + # Force-fetch: overwrite local ref with current remote state + git -C $repo fetch $Remote "${nsRef}:${nsRef}" 2>&1 | Out-Null + + # Re-append our note on top of the now-current remote state + git -C $repo notes --ref=$namespace append -m $noteJson $Commit 2>&1 | Out-Null + + $jitter = Get-Random -Minimum 0 -Maximum 1000 + $sleep = [Math]::Pow(2, $i) + $jitter / 1000 + Start-Sleep -Seconds $sleep + + } else { + Log "Push error: $pushOut" Red + if ($i -eq $maxRetries - 1) { + Write-Warning "Failed after $maxRetries retries. Push manually: git push origin '${nsRef}:${nsRef}'" + } + } + } +} + +# ── Show result ─────────────────────────────────────────────────────────────── +if (-not $Quiet) { + Log "Note content:" + $note | ConvertTo-Json -Depth 5 | Write-Host -ForegroundColor DarkGray +} diff --git a/.squad-templates/squad.agent.md b/.squad-templates/squad.agent.md index 01e18dfad..2c5a7f66a 100644 --- a/.squad-templates/squad.agent.md +++ b/.squad-templates/squad.agent.md @@ -107,6 +107,8 @@ The `union` merge driver keeps all lines from both sides, which is correct for a **On every session start:** Run `git config user.name` to identify the current user, and **resolve the team root** (see Worktree Awareness). Store the team root — all `.squad/` paths must be resolved relative to it. Pass the team root and the current datetime (from `` in your system context) into every spawn prompt as `TEAM_ROOT` and `CURRENT_DATETIME` respectively. Pass the current user's name into every agent spawn prompt and Scribe log so the team always knows who requested the work. Check `.squad/identity/now.md` if it exists — it tells you what the team was last focused on. Update it if the focus has shifted. +**Resolve state backend:** Read `.squad/config.json` and check the `stateBackend` field. Valid values: `"worktree"` (default), `"git-notes"`, `"orphan"`, `"two-layer"`. Store as `STATE_BACKEND` and pass it into every spawn prompt. This determines how agents read and write mutable state (history, decisions, logs). Static config (charters, team.md, routing.md) always lives on disk regardless of backend. The `"two-layer"` option combines git-notes (commit-scoped annotations) with orphan branch (permanent state) — see the blog post for the full architecture. + **⚡ Context caching:** After the first message in a session, `team.md`, `routing.md`, and `registry.json` are already in your context. Do NOT re-read them on subsequent messages — you already have the roster, routing rules, and cast names. Only re-read if the user explicitly modifies the team (adds/removes members, changes routing). **Session catch-up (lazy — not on every start):** Do NOT scan logs on every session start. Only provide a catch-up summary when: @@ -223,13 +225,16 @@ The `name` parameter generates the human-readable agent ID shown in the tasks pa **When you detect a directive:** -1. Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: - ``` - ### {timestamp}: User directive - **By:** {user name} (via Copilot) - **What:** {the directive, verbatim or lightly paraphrased} - **Why:** User request — captured for team memory - ``` +1. Capture the directive: + - **worktree/orphan backend:** Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: + ``` + ### {timestamp}: User directive + **By:** {user name} (via Copilot) + **What:** {the directive, verbatim or lightly paraphrased} + **Why:** User request — captured for team memory + ``` + - **git-notes backend:** Persist via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/directives" -Content '{"timestamp": "{timestamp}", "by": "{user name}", "what": "...", "why": "User request"}'` 2. Acknowledge briefly: `"📌 Captured. {one-line summary of the directive}."` 3. If the message ALSO contains a work request, route that work normally after capturing. If it's directive-only, you're done — no agent spawn needed. @@ -344,7 +349,12 @@ prompt: | TARGET FILE(S): {exact file path(s)} Do the work. Keep it focused. + {% if STATE_BACKEND == "git-notes" %} + If you made a meaningful decision, persist it via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"decision": {"title": "...", "what": "...", "why": "..."}}'` + {% else %} If you made a meaningful decision, write to .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} ⚠️ OUTPUT: Report outcomes in human terms. Never expose tool internals or SQL. ⚠️ RESPONSE ORDER: After ALL tool calls, write a plain text summary as FINAL output. @@ -598,8 +608,9 @@ When the user gives any task, the Coordinator MUST: To enable full parallelism, shared writes use a drop-box pattern that eliminates file conflicts: **decisions.md** — Agents do NOT write directly to `decisions.md`. Instead: -- Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` -- Scribe merges inbox entries into the canonical `.squad/decisions.md` and clears the inbox +- **worktree/orphan backend:** Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` +- **git-notes backend:** Agents persist decisions via their git notes ref (see State Protocol). Scribe reads all agent notes and merges decisions. +- Scribe merges into the canonical `.squad/decisions.md` and clears the inbox (or note entries) - All agents READ from `.squad/decisions.md` at spawn time (last-merged snapshot) **orchestration-log/** — Scribe writes one entry per agent after each batch: @@ -799,7 +810,71 @@ prompt: | - Commit and push from the worktree {% endif %} + STATE_BACKEND: {state_backend} + + {% if STATE_BACKEND == "git-notes" %} + ## State Protocol — Git Notes + This project uses git-notes for mutable state. **DO NOT write to `.squad/` files for state.** + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading your state:** + Run: `powershell .squad/scripts/notes/fetch.ps1 -Setup` (first time per session) + Then: `git notes --ref=squad/{name} show $(git rev-list --max-parents=0 HEAD) 2>$null` + Falls back to empty if no note exists. + + **Writing state (history, decisions, learnings):** + Run: `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{json}'` + The helper handles JSON validation, conflict retry, and push. + + **Decisions:** Write decisions as JSON via your note ref. Scribe will merge them. + **Skills:** Skills are static config — write to `.squad/skills/` on disk as normal. + {% endif %} + + {% if STATE_BACKEND == "orphan" %} + ## State Protocol — Orphan Branch + This project uses an orphan branch (`squad-state`) for mutable state. + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading state:** Read `.squad/` files on disk — they are synced from the orphan branch. + **Writing state:** Write to `.squad/` files on disk as normal during your session. + Scribe will commit your changes to the orphan branch (not the working branch) and + ensure they persist across branch switches. + + **Important:** Do NOT commit `.squad/` state files to the working branch yourself. + Scribe handles the orphan branch commit workflow. + {% endif %} + + {% if STATE_BACKEND == "two-layer" %} + ## State Protocol — Two-Layer (Git Notes + Orphan Branch) + This project uses the two-layer architecture from Tamir's blog: + - **Layer 1 (git notes):** Commit-scoped "why" annotations — invisible in PRs + - **Layer 2 (orphan branch):** Permanent state store — decisions, histories, logs + + Static config (charters, team.md, routing.md) is on disk as normal. + + **During your session:** + 1. Write commit-scoped annotations as git notes on HEAD: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + 2. Write bulk state (history, logs) to `.squad/` files on disk — Scribe moves them to the orphan branch. + + **Note flags:** + - `"promote_to_permanent": true` — Ralph promotes this to decisions.md after PR merge + - `"archive_on_close": true` — Worth keeping even if PR is rejected (valuable research) + - Neither flag — silently ignored if PR is rejected (correct for branch-specific decisions) + + **Important:** Do NOT commit `.squad/` state files to the working branch. + Scribe handles orphan commits. Ralph handles note promotion. + {% endif %} + + {% if STATE_BACKEND == "worktree" or STATE_BACKEND is not defined %} Read .squad/agents/{name}/history.md (your project knowledge). + {% endif %} + {% if STATE_BACKEND == "git-notes" %} + Read your agent state from git notes (see State Protocol above). + {% endif %} + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + Read .squad/agents/{name}/history.md (your project knowledge — synced from orphan branch). + {% endif %} Read .squad/decisions.md (team decisions to respect). If .squad/identity/wisdom.md exists, read it before starting work. If .squad/identity/now.md exists, read it at spawn time. @@ -823,10 +898,27 @@ prompt: | ⚠️ DATES: When writing dates in any file (decisions, history, logs), use ONLY the CURRENT_DATETIME value above. Never infer or guess the date. AFTER work: + {% if STATE_BACKEND == "git-notes" %} + 1. Persist your learnings as JSON via the State Protocol: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"learnings": ["..."], "timestamp": "{current_datetime}"}'` + 2. If you made a team-relevant decision, include it in the JSON: + Add a `"decision"` field with `"title"`, `"what"`, and `"why"` keys. + Scribe will merge decisions into the canonical decisions.md. + {% elif STATE_BACKEND == "two-layer" %} + 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": + architecture decisions, patterns, user preferences, key file paths. + (Scribe commits this to the orphan branch.) + 2. If you made a team-relevant decision, write BOTH: + a. A git note on HEAD with promote flag: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + b. A drop file: .squad/decisions/inbox/{name}-{brief-slug}.md + (Scribe merges to orphan branch; Ralph promotes note after PR merge.) + {% else %} 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": architecture decisions, patterns, user preferences, key file paths. 2. If you made a team-relevant decision, write to: .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} 3. SKILL EXTRACTION: If you found a reusable pattern, write/update .squad/skills/{skill-name}/SKILL.md (read templates/skill.md for format). @@ -878,18 +970,47 @@ prompt: | You are the Scribe. Read .squad/agents/scribe/charter.md. TEAM ROOT: {team_root} CURRENT_DATETIME: {current_datetime} + STATE_BACKEND: {state_backend} SPAWN MANIFEST: {spawn_manifest} Tasks (in order): - 0. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "git-notes" or STATE_BACKEND == "two-layer" %} + 0. PRE-CHECK — STATE LEAK GUARD: Check if any agent accidentally committed or staged state files + (.squad/decisions.md, agents/*/history.md, log/*, orchestration-log/*, decisions/inbox/*) + to the working branch. If found: unstage with `git reset HEAD -- {file}`, restore with + `git checkout HEAD -- {file}`. If leaked in last commit, amend to remove. Log count. + {% endif %} + 0b. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. 1. DECISIONS ARCHIVE [HARD GATE]: If decisions.md >= 20480 bytes, archive entries older than 30 days NOW. If >= 51200 bytes, archive entries older than 7 days. Do not skip this step. + {% if STATE_BACKEND == "git-notes" %} + 2. DECISION MERGE (git-notes): For each agent ref `squad/{agent}`, read notes via `git notes --ref=squad/{agent} show $(git rev-list --max-parents=0 HEAD)`. Extract any `decision` entries. Merge into decisions.md. Clear the decision field by overwriting the note without it. + {% elif STATE_BACKEND == "two-layer" %} + 2. DECISION MERGE (two-layer): Merge .squad/decisions/inbox/ → decisions.md AND read agent note refs for any decisions with `promote_to_permanent`. Deduplicate. Push note refs: `git push origin 'refs/notes/squad/*'` + {% else %} 2. DECISION INBOX: Merge .squad/decisions/inbox/ → decisions.md, delete inbox files. Deduplicate. + {% endif %} 3. ORCHESTRATION LOG: Write .squad/orchestration-log/{timestamp}-{agent}.md per agent. Use ISO 8601 UTC timestamp. 4. SESSION LOG: Write .squad/log/{timestamp}-{topic}.md. Brief. Use ISO 8601 UTC timestamp. + {% if STATE_BACKEND == "git-notes" %} + 5. CROSS-AGENT (git-notes): For team updates, write to affected agents' note refs via `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{agent}" -Content '{json}'`. + {% else %} 5. CROSS-AGENT: Append team updates to affected agents' history.md. + {% endif %} 6. HISTORY SUMMARIZATION [HARD GATE]: If any history.md >= 15360 bytes (15KB), summarize now. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + 7. GIT COMMIT (orphan): Stage `.squad/` state files and commit to the `squad-state` orphan branch: + a. Identify changed `.squad/` state files via `git status --porcelain` (decisions.md, agents/*/history.md, log/*, orchestration-log/*). + b. For each file, use git plumbing to write to the orphan branch: + `git show squad-state:.squad/{path}` to check if file exists on orphan. + Use `git checkout squad-state -- .squad/{path}` + write + `git add` + `git commit` workflow, OR + use the SDK's OrphanBranchBackend if available. + c. Reset working tree state files: `git checkout HEAD -- .squad/` to avoid polluting the working branch. + d. Push orphan branch: `git push origin squad-state` + ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + {% else %} 7. GIT COMMIT: Stage only the exact `.squad/` files Scribe wrote in this session. Use `git status --porcelain` filtered to allowed paths (decisions.md, decisions-archive.md, agents/{name}/history.md, agents/{name}/history-archive.md, log/*, orchestration-log/*). Stage each file individually with `git add -- `. Handle renames by extracting destination path (`-replace '^.* -> ',''`). Commit with -F (write msg to temp file). Skip if nothing staged. ⚠️ NEVER use `git add .squad/` or broad globs. + {% endif %} 8. HEALTH REPORT: Log decisions.md before/after size, inbox count processed, history files summarized. Never speak to user. ⚠️ End with plain text summary after all tool calls. @@ -947,6 +1068,8 @@ If the user wants to remove someone: ## Source of Truth Hierarchy +> **State backend note:** Files below marked as "Derived / append-only" are **mutable state** — their storage location depends on the configured `STATE_BACKEND`. On `worktree` (default), they live on disk. On `git-notes`, they live in git notes refs. On `orphan`, they live on the `squad-state` orphan branch. On `two-layer`, commit-scoped annotations live in git notes AND permanent state lives on the orphan branch. Files marked as "Authoritative" are **static config** and always live on disk regardless of backend. + | File | Status | Who May Write | Who May Read | |------|--------|---------------|--------------| | `.github/agents/squad.agent.md` | **Authoritative governance.** All roles, handoffs, gates, and enforcement rules. | Repo maintainer (human) | Squad (Coordinator) | diff --git a/docs/src/content/blog/031-state-backends.md b/docs/src/content/blog/031-state-backends.md new file mode 100644 index 000000000..bc7012313 --- /dev/null +++ b/docs/src/content/blog/031-state-backends.md @@ -0,0 +1,103 @@ +--- +title: "State Backends — Keep Your PRs Clean" +date: 2026-04-20 +author: "Tamir" +tags: [squad, state-backends, git-notes, orphan-branch, two-layer, architecture] +status: draft +hero: "Squad now supports 4 state backends that keep .squad/ files out of your PRs. Choose local, orphan branch, or the two-layer architecture from the blog." +--- + +# State Backends — Keep Your PRs Clean + +> ⚠️ **Experimental** — Squad is alpha software. APIs, commands, and behavior may change between releases. + +> _Squad now supports 4 state backends. Your PRs stay clean — just code._ + +## The Problem + +Every time an agent makes a decision, writes to history, or logs a session, those changes end up as `.squad/` file modifications in your working branch. Open a PR and your reviewer sees 57 code changes buried under 40 decision logs, agent history entries, and session files. + +Two completely different workflows sharing one branch: +- **Code** → slow, human-gated, needs review approval +- **Squad state** → fast, autonomous, no human ever needs to review it + +## The Fix + +PR #1004 adds state backend support. One line in your config, and all mutable state goes somewhere else. + +```bash +# The fastest path to clean PRs: +squad init --state-backend two-layer +``` + +## Four Options + +| Backend | Where state goes | PRs clean? | Setup | +|---------|-----------------|------------|-------| +| `local` | Working branch (default) | ❌ | Zero config | +| `orphan` | `squad-state` branch | ✅ | Config + branch | +| `two-layer` | Notes + orphan combined | ✅ | `--state-backend two-layer` | + +## The Two-Layer Architecture + +The `two-layer` option implements the architecture from [Tamir's blog post](https://www.tamirdresher.com/blog/2026/03/23/scaling-ai-part7b-git-notes): + +- **Layer 1 (git notes):** Thin commit-scoped "why" annotations. Invisible in PR diffs. Attached to specific commits. +- **Layer 2 (orphan branch):** Permanent state store. Decisions, histories, logs. The team's full diary. +- **Ralph bridges the layers:** After a PR merges, Ralph promotes notes with `promote_to_permanent: true` to the orphan branch. Notes on rejected PRs are silently ignored. + +This handles the three scenarios from the blog correctly: +1. **Rejected feature** — decision on a rejected PR is NOT promoted ✅ +2. **Universal truth** — routing change flagged with `promote_to_permanent` survives ✅ +3. **Valuable failure** — research flagged with `archive_on_close` is preserved ✅ + +## How It Works (Under the Hood) + +The Squad coordinator (`squad.agent.md`) detects `stateBackend` from `.squad/config.json` at session start and adapts every agent spawn prompt: + +- **Agents** receive backend-specific instructions for reading and writing state +- **Scribe** receives backend-specific commit targets (orphan branch, note refs, or working branch) +- **State Leak Guard** catches if an agent accidentally stages state files on the working branch + +Static config (charters, team.md, routing.md) always stays on disk. Only mutable state (decisions, history, logs) moves to the configured backend. + +## Quick Start + +```bash +# New project — set backend at init time +squad init --state-backend two-layer + +# Existing project — migrate with one config change +# Edit .squad/config.json → add "stateBackend": "two-layer" +git add .squad/config.json && git commit -m "config: use two-layer" +``` + +For the full migration guide and troubleshooting, see the [State Backends feature docs](/docs/features/state-backends/). + +## Tested With Real Squads + +We ran 12 E2E tests with real squad sessions — real team casting (Usual Suspects, Firefly universes), real agent spawns, real decisions recorded. All evidence is in [PR #1004](https://github.com/bradygaster/squad/pull/1004). + +Key proof: +- Git notes with `promote_to_permanent: true` written by agents ✅ +- Orphan branch receives state commits from Scribe ✅ +- Feature branch PRs show ONLY code changes, zero `.squad/` state ✅ +- State persists across branch switches ✅ + +## Try It + +Build from the PR branch: + +```bash +git clone https://github.com/bradygaster/squad.git +cd squad && git checkout feat/state-backend-global-996 +npm install && npm run build +``` + +Then init any repo with your preferred backend: + +```bash +node /packages/squad-cli/dist/cli-entry.js init --state-backend two-layer +``` + +We'd love feedback — especially on whether the init flow feels right and whether state actually persists across branch switches in your environment. diff --git a/docs/src/content/docs/features/state-backends.md b/docs/src/content/docs/features/state-backends.md index db40041bf..058b03e9d 100644 --- a/docs/src/content/docs/features/state-backends.md +++ b/docs/src/content/docs/features/state-backends.md @@ -2,104 +2,85 @@ > ⚠️ **Experimental** — Squad is alpha software. APIs, commands, and behavior may change between releases. - -**Try this to use git-notes for state storage:** -```bash -squad watch --state-backend git-notes -``` - -**Try this to use an orphan branch:** -```bash -squad watch --state-backend orphan -``` - -**Try this to set a persistent default:** -```bash -echo '{ "stateBackend": "git-notes" }' > .squad/config.json -``` - -Squad supports multiple **state backends** for storing `.squad/` state. Each backend determines _where_ and _how_ decisions, skills, agent memories, and session logs are persisted — without changing how agents interact with the data. +Squad supports multiple **state backends** for storing `.squad/` state (decisions, agent memories, session logs, skills). Each backend determines _where_ and _how_ this data is persisted — without changing how agents interact with it. Once configured, everything is automatic. --- ## The Problem -The default **worktree** backend stores `.squad/` state as regular files in the working tree. This works well for most workflows, but has trade-offs: +By default, Squad stores `.squad/` state as regular files in your working tree. This works for solo workflows but has real trade-offs for teams: - **Branch pollution:** `.squad/` files appear in diffs and PRs - **Branch-switch loss:** State can be lost when switching branches (if not committed) -- **Merge conflicts:** Multiple branches modifying `.squad/` files can conflict +- **Merge conflicts:** Multiple team members modifying `.squad/` files creates frequent conflicts -State backends solve this by moving `.squad/` data into Git-native structures that live outside the working tree. +State backends solve this by moving `.squad/` data into Git-native structures that live outside the working tree — keeping your PRs clean and your state safe across branches. --- -## Available Backends - -### Worktree (default) +## Getting Started -State lives as regular files in `.squad/` inside the working tree. This is the standard behavior — what you get out of the box. +### New project — choose a backend during init ```bash -squad watch --state-backend worktree +# Default (local — files in .squad/, same as always) +squad init + +# Orphan branch (state on a dedicated squad-state branch) +squad init --state-backend orphan + +# Two-layer (recommended for teams — orphan branch + git notes) +squad init --state-backend two-layer ``` -**Pros:** -- Simple and familiar — files on disk -- Easy to inspect, edit, and commit -- Works with all Git tools and IDEs +The backend is stored in `.squad/config.json` — you never need to pass it again. All subsequent commands (`squad watch`, interactive sessions, etc.) read from config automatically. -**Cons:** -- Files appear in `git status` and diffs -- Branch switches can lose uncommitted state +### Existing project — migrate with upgrade -**Best for:** Most projects, especially when you want squad state committed alongside code. +```bash +# Migrate from local to orphan or two-layer +squad upgrade --state-backend two-layer ---- +# Or: squad upgrade --state-backend orphan +``` -### Git Notes +This migrates existing state, creates the orphan branch, and installs git hooks for automatic multi-user sync. No manual steps needed. -State is stored in [Git notes](https://git-scm.com/docs/git-notes) under `refs/notes/squad`. Notes are attached to `HEAD`, keeping data associated with commits but invisible in the working tree. +### What gets installed automatically -```bash -squad watch --state-backend git-notes -``` +When you choose `orphan` or `two-layer`: +- **Git hooks** (pre-push, post-merge, post-checkout, post-rewrite) are installed in `.git/hooks/` +- These hooks sync the `squad-state` branch automatically when you push/pull — no manual sync needed +- Hooks chain with existing hooks (husky, etc.) — nothing is overwritten -**How it works:** -- All state is serialized as a single JSON blob attached as a note on `HEAD` -- Reading loads the JSON, writing updates and reattaches it -- Notes travel with `git push` / `git fetch` when configured (see [Sharing](#sharing-git-notes-state)) +--- -**Pros:** -- Working tree stays completely clean — no `.squad/` files -- State is associated with specific commits -- No merge conflicts from `.squad/` files in PRs +## Available Backends -**Cons:** -- State is per-commit — switching to a different commit loses the note context -- Requires `git notes` familiarity for debugging -- Not human-readable without `git notes show` +### Local (default) -**Best for:** Repos where you want zero `.squad/` files in the working tree or PRs. +State lives as regular files in `.squad/` inside the working tree. This is the standard behavior — what you get out of the box. -#### Sharing Git Notes State +**Pros:** +- Simple and familiar — files on disk +- Easy to inspect, edit, and commit +- Works with all Git tools and IDEs -By default, Git doesn't push notes. To share git-notes state across clones: +**Cons:** +- Files appear in `git status` and diffs +- Branch switches can lose uncommitted state -```bash -# Push notes -git push origin refs/notes/squad +**Best for:** Most projects, especially when you want squad state committed alongside code. -# Fetch notes -git fetch origin refs/notes/squad:refs/notes/squad -``` +--- -Or configure automatic fetch in `.git/config`: +### Git Notes (Deprecated → Two-Layer) -```ini -[remote "origin"] - fetch = +refs/notes/squad:refs/notes/squad -``` +> ⚠️ **Deprecated:** The standalone `git-notes` backend has been removed as a user-facing option. If your config still references `git-notes`, it will be **automatically migrated to `two-layer`** at runtime. +> +> **Why:** Standalone git-notes stores all state as a single JSON blob on the root commit. This fundamentally cannot handle concurrent writes from multiple team members — `git notes merge` cannot merge opaque JSON, causing silent data loss. +> +> **Replacement:** The `two-layer` backend uses git notes as best-effort commit annotations (the "why" layer) while storing durable state on an orphan branch with per-file granularity (the "state" layer). This gives you the clean working tree of git-notes with the team-safe mergeability of the orphan approach. --- @@ -107,10 +88,6 @@ Or configure automatic fetch in `.git/config`: State lives on a dedicated orphan branch (`squad-state` by default). The branch has no common history with your main branches — it's a completely separate tree used only for squad data. -```bash -squad watch --state-backend orphan -``` - **How it works:** - An orphan branch `squad-state` is created automatically on first write - Each state file is stored as a blob in the branch's tree @@ -125,7 +102,7 @@ squad watch --state-backend orphan **Cons:** - An extra branch in your repository -- Slightly more complex than worktree for debugging +- Slightly more complex than `local` for debugging - Concurrent writes to the branch can conflict (single-writer recommended) **Best for:** Teams who want Git-versioned state without polluting the main branch history. @@ -134,65 +111,49 @@ squad watch --state-backend orphan ## Configuration -### CLI Flag (per-invocation) - -Pass `--state-backend` to `squad watch` or `squad triage`: - -```bash -squad watch --state-backend git-notes -squad triage --state-backend git-notes -squad watch --state-backend orphan -squad watch --state-backend worktree -``` - -### Config File (persistent) - -Set a default in `.squad/config.json`: +The state backend is set once (during `squad init` or `squad upgrade`) and stored in `.squad/config.json`: ```json { - "stateBackend": "git-notes" + "version": 1, + "teamRoot": ".", + "stateBackend": "two-layer" } ``` -This persists across invocations. The CLI flag overrides the config file when both are present. +All squad commands read from this file automatically. You don't need to pass `--state-backend` on every invocation. -### Priority Order - -| Priority | Source | Example | -|----------|--------|---------| -| 1 (highest) | CLI flag | `--state-backend orphan` | -| 2 | `.squad/config.json` | `"stateBackend": "orphan"` | -| 3 (default) | Built-in default | `worktree` | +> **Note:** If no `stateBackend` field exists, the default is `local` (current behavior, no change). ### Fallback Behavior -If a non-default backend fails to initialize (e.g., Git is not available, permissions issue), Squad automatically falls back to the **worktree** backend with a warning: +If a non-default backend fails to initialize (e.g., Git is not available, permissions issue), Squad automatically falls back to the **local** backend with a warning: ``` -Warning: State backend 'git-notes' failed: . Falling back to 'worktree'. +Warning: State backend 'two-layer' failed: . Falling back to 'local'. ``` --- ## Comparison -| Feature | Worktree | Git Notes | Orphan Branch | -|---------|----------|-----------|---------------| +| Feature | Local | Orphan Branch | Two-Layer | +|---------|-------|---------------|-----------| | Working tree clean | ❌ | ✅ | ✅ | | Appears in PRs | Yes (if committed) | No | No | -| Human-readable on disk | ✅ Files | ❌ JSON blob | ⚠️ Via `git show` | -| Git history | Via normal commits | Per-note | Per-branch commits | -| Branch-switch safe | ❌ (if uncommitted) | ⚠️ | ✅ | -| Easy to inspect | ✅ `cat .squad/...` | ⚠️ `git notes show` | ⚠️ `git show squad-state:...` | -| Sharing across clones | Normal push/pull | Requires notes fetch config | Normal branch push/pull | -| Concurrent-write safe | ✅ (filesystem) | ⚠️ (last writer wins) | ⚠️ (single writer) | +| Human-readable on disk | ✅ Files | ⚠️ Via `git show` | ⚠️ Via `git show` | +| Git history | Via normal commits | Per-branch commits | Per-branch + notes | +| Branch-switch safe | ❌ (if uncommitted) | ✅ | ✅ | +| Easy to inspect | ✅ `cat .squad/...` | ⚠️ `git show squad-state:...` | ⚠️ `git show squad-state:...` | +| Sharing across clones | Normal push/pull | Normal branch push/pull | Normal branch push/pull | +| Concurrent-write safe | ✅ (filesystem) | ⚠️ (single writer) | ✅ (per-file merge) | +| Team-safe (multi-user) | ❌ (merge conflicts) | ⚠️ (needs coordination) | ✅ (designed for teams) | --- ## Inspecting State -### Worktree +### Local ```bash cat .squad/decisions.md @@ -202,11 +163,11 @@ ls .squad/skills/ ### Git Notes ```bash -# Show all state as JSON -git notes --ref=squad show HEAD +# Show all state as JSON (anchored to root commit) +git notes --ref=squad show $(git rev-list --max-parents=0 HEAD) # Pretty-print -git notes --ref=squad show HEAD | python -m json.tool +git notes --ref=squad show $(git rev-list --max-parents=0 HEAD) | python -m json.tool ``` ### Orphan Branch @@ -230,22 +191,27 @@ The state backend is available programmatically via the Squad SDK: ```typescript import { + resolveSquadState, resolveStateBackend, type StateBackend, } from '@bradygaster/squad-sdk'; -// Resolve backend from config + CLI override +// Option 1: Full context resolution (recommended) +// Resolves paths + backend from config + CLI override in one call +const ctx = resolveSquadState(process.cwd(), 'two-layer'); +if (ctx) { + ctx.backend.write('decisions.md', '# Decisions\n...'); + ctx.backend.append('log.md', 'New entry\n'); + ctx.backend.delete('inbox/processed.md'); +} + +// Option 2: Backend-only resolution const backend: StateBackend = resolveStateBackend( '.squad', // squadDir process.cwd(), // repoRoot - 'git-notes' // optional CLI override + 'two-layer' // optional CLI override ); - -// Use the backend backend.write('decisions.md', '# Decisions\n...'); -const content = backend.read('decisions.md'); -const exists = backend.exists('skills/my-skill/SKILL.md'); -const entries = backend.list('skills'); ``` All backends implement the same `StateBackend` interface: @@ -256,6 +222,8 @@ interface StateBackend { write(relativePath: string, content: string): void; exists(relativePath: string): boolean; list(relativeDir: string): string[]; + delete(relativePath: string): boolean; + append(relativePath: string, content: string): void; readonly name: string; } ``` @@ -276,10 +244,355 @@ All validation is centralized in `validateStateKey()` and applied uniformly acro --- +## Content Fidelity + +All backends preserve content exactly as written — including trailing newlines, leading whitespace, +and empty lines. This is critical for append-only files like `history.md` and `decisions.md` where +multiple agents append entries over time. + +The orphan and two-layer backends use raw `execFileSync` for content reads (without trimming) to +ensure faithful round-trips. Git plumbing helpers that trim output are only used for non-content +operations like `rev-parse` and `ls-tree`. + +--- + +## Worktree Awareness + +When running in a git worktree, `resolveSquadState()` uses `git rev-parse --show-toplevel` to +determine the actual current worktree root — not the parent of `.squad/`. This ensures that +git-native backends (orphan, two-layer) operate in the correct repository context, even when +`.squad/` is resolved from the main checkout via the worktree fallback strategy. + +--- + ## Notes -- State backends are **opt-in** — the default is `worktree` (no behavior change) +- State backends are **opt-in** — the default is `local` (no behavior change) - All backends implement the same interface — agents don't know or care which backend is active -- The `external` backend type exists as a stub for future external storage (see [External State](./external-state.md)) +- Empty directories are automatically pruned after the last file is deleted (orphan backend) +- The `external` backend type exists as a stub for future external storage (see [External State](./external-state)) - State backends are available in the **insider** release channel (`@bradygaster/squad-cli@insider`) -- 30+ tests cover all backends including security hardening scenarios +- 63 unit tests + 46 E2E tests cover all backends including security hardening, content fidelity, and directory pruning + +--- + +## Using with Copilot CLI Sessions + +The SDK's `StateBackend` interface handles programmatic state for Squad internals, but Copilot agents also need a way to write commit-scoped context — decisions, research, reviews — without creating `.squad/` file changes that pollute PRs. + +The solution: agents use **git notes CLI commands** directly for mutable, commit-scoped state. The `notes-protocol.md` template defines the contract. + +### How it works + +1. Each agent writes to its own namespace: `refs/notes/squad/{agent-name}` +2. Notes are JSON with required fields: `agent`, `timestamp`, `type`, `content` +3. Notes are invisible in PR diffs — they travel as git refs, not files +4. Ralph promotes notes with `"promote_to_permanent": true` to `decisions.md` after merge +5. If a PR is rejected, notes on those commits are NOT promoted (desired behavior) + +### Setup + +When you enable `stateBackend: "two-layer"` or `stateBackend: "orphan"`, copy the notes protocol and helper scripts into your project: + +```bash +# Copy from Squad's templates (after squad init) +cp .squad/templates/notes-protocol.md .squad/notes-protocol.md +cp -r .squad/templates/scripts/notes/ scripts/notes/ + +# One-time git config for notes fetch +./scripts/notes/fetch.ps1 -Setup +``` + +### Copilot Instructions Integration + +Add the following to your `.github/copilot-instructions.md` (or `.copilot/copilot-instructions.md`) to teach agents the notes protocol: + +````markdown +## Git Notes — State Protocol + +**Every agent uses git notes for commit-scoped state.** Do not write to +`.squad/decisions.md` or other `.squad/` files directly on feature branches. + +### On every work round + +1. **Start**: `git fetch origin 'refs/notes/*:refs/notes/*'` +2. **When making a decision**: Write it as a note on the relevant commit +3. **End**: `git push origin 'refs/notes/*:refs/notes/*'` + +### Write pattern + +```bash +git notes --ref=squad/{your-agent} add \ + -m '{"agent":"{Name}","timestamp":"{ISO8601}","type":"decision","content":"..."}' \ + HEAD +``` + +Use `git notes append` if a note already exists on the commit. + +### Key rules + +- Write only to your own namespace (`refs/notes/squad/{your-name}`) +- Notes MUST be valid JSON +- Set `"promote_to_permanent": true` for decisions that should outlast the branch +- Set `"archive_on_close": true` for research worth keeping even if the PR is rejected +- Fetch before write, push after your round + +See `.squad/notes-protocol.md` for the full contract. +```` + +### Example: Agent writes a decision, Ralph promotes it + +1. **Data** makes an architecture choice and writes a note: + ```bash + git notes --ref=squad/data add -m \ + '{"agent":"Data","timestamp":"2026-03-23T14:00:00Z","type":"decision","decision":"Use JWT RS256","reasoning":"Matches existing auth pattern","promote_to_permanent":true}' \ + HEAD + git push origin 'refs/notes/*:refs/notes/*' + ``` + +2. **PR merges** into the default branch. + +3. **Ralph** runs promotion on the next watch cycle: + - Fetches all notes + - Finds Data's note with `promote_to_permanent: true` on a merged commit + - Appends the decision to `decisions.md` via the state backend + - Notes on rejected PRs are silently ignored + +### Template files + +When `stateBackend` is set to `two-layer` or `orphan`, the following templates are available: + +| Template | Purpose | +|----------|---------| +| `notes-protocol.md` | The full agent contract for git notes | +| `scripts/notes/fetch.ps1` | Fetch + setup refspec + merge after conflict | +| `scripts/notes/write-note.ps1` | Agent helper — handles JSON, conflicts, push | + +### Automatic Coordinator Integration + +**You don't need to manually add copilot-instructions.md snippets.** When `stateBackend` is set in `.squad/config.json`, the Squad coordinator (`squad.agent.md`) automatically adapts its agent spawn prompts: + +| Backend | Agent reads | Agent writes | Scribe commits to | +|---------|-------------|--------------|-------------------| +| `local` | `.squad/` files on disk | `.squad/` files on disk | Working branch | +| `orphan` | `.squad/` files on disk (synced) | `.squad/` files on disk | `squad-state` orphan branch (NOT working branch) | +| `two-layer` | Git notes + orphan branch | Git notes via `write-note.ps1` + orphan | Pushes note refs + orphan branch | + +**Config vs State distinction:** +- **Static config** (charters, team.md, routing.md, casting/) — always on disk, all backends +- **Mutable state** (history.md, decisions/inbox/, logs, orchestration-log/) — backend-dependent + +The coordinator passes `STATE_BACKEND` into every agent spawn prompt. Agents receive backend-specific instructions for reading and writing state. Scribe receives backend-specific commit instructions. This is fully automatic — no user configuration beyond setting `stateBackend` in config.json is needed. + +--- + +## Migrating an Existing Squad + +Use `squad upgrade` to migrate — it handles everything: + +```bash +squad upgrade --state-backend two-layer +# or: squad upgrade --state-backend orphan +``` + +This will: +1. Update `.squad/config.json` with the new backend +2. Create the `squad-state` orphan branch (if needed) +3. Install git hooks for automatic sync +4. Preserve all existing state + +**What happens:** Existing `.squad/` files remain on disk as a read-only reference. New decisions and state writes go to the orphan branch (and git notes for two-layer). Over time, the on-disk state files become stale (they're the snapshot from before migration), while the orphan branch and notes contain the latest state. + +### Switching between orphan and two-layer + +Change `stateBackend` in `.squad/config.json`. The coordinator adapts on the next session. Both use the `squad-state` orphan branch, so existing state is preserved. Two-layer additionally enables git notes for commit-scoped annotations. + +--- + +## Troubleshooting + +### "My state disappeared after switching branches" + +**Cause:** You're using the default `local` backend. State files are branch-local. + +**Fix:** Switch to `orphan` or `two-layer` backend. Both persist state across branches: +- Orphan: state lives on a dedicated branch (accessible via `git show squad-state:`) +- Two-layer: orphan branch + git notes for commit-scoped annotations + +### "State files are showing up in my PR" + +**Cause:** Using `local` backend, or an agent accidentally committed state files on orphan/two-layer backend. + +**Fix:** +1. If using local backend: switch to `orphan` or `two-layer` +2. If using orphan/two-layer: Scribe's State Leak Guard should catch this automatically. If it missed: + ```bash + git reset HEAD -- .squad/decisions.md .squad/agents/*/history.md .squad/log/ .squad/orchestration-log/ + git checkout HEAD -- .squad/decisions.md .squad/agents/*/history.md + ``` + +### "Orphan branch doesn't exist" + +**Cause:** The `squad-state` branch hasn't been created yet. + +**Fix:** Create it manually: +```bash +git checkout --orphan squad-state +git rm -rf . +mkdir .squad && echo "# Squad State" > .squad/README.md +git add .squad/ && git commit -m "init: squad-state orphan branch" +git checkout main +``` + +Scribe will auto-create it on the next session if it doesn't exist (via git plumbing: `mktree`, `commit-tree`, `update-ref`). + +### "Git notes not found on root commit" + +**Cause:** The agent wrote the note to HEAD instead of the root commit. + +**Known issue:** Some agents write notes to the current HEAD instead of `$(git rev-list --max-parents=0 HEAD)`. The note still exists on the ref and is readable, but the root-commit anchor pattern isn't being followed precisely. + +**Workaround:** The note is still accessible via `git notes --ref=squad/{agent} show {commit-sha}`. The ref itself (`refs/notes/squad/{agent}`) is visible from all branches regardless of which commit the note is on. + +### "Config.json doesn't have stateBackend" + +**This is fine.** The default is `local` — the current behavior. No config change needed unless you want a different backend. + +--- + +## Multi-User Synchronization + +When multiple team members work on the same repo with Squad, the state backend determines how state stays in sync. + +### Local backend + +Each user has their own `.squad/` files in the working tree. If committed, they merge like any other files — which means **merge conflicts are common** when two people modify decisions or histories simultaneously. This is the main reason teams choose orphan or two-layer backends. + +### Orphan backend + +The `squad-state` branch is a normal Git branch. Synchronization works like any other branch: + +```bash +# Before a squad session — pull latest state +git fetch origin squad-state:squad-state + +# After a squad session — push your state changes +git push origin squad-state +``` + +**Conflict handling:** If two users push to `squad-state` simultaneously, the second push will be rejected (non-fast-forward). Resolution: + +```bash +git fetch origin squad-state:squad-state +git checkout squad-state +git merge origin/squad-state # resolve conflicts, then: +git push origin squad-state +git checkout main +``` + +In practice, Squad's watch loop handles this automatically — Scribe's commit logic retries on push failure. + +> **Tip:** For teams, consider protecting the `squad-state` branch with GitHub branch protection rules that allow force-push from the CI bot but require linear history from humans. + +### Two-layer backend + +Two-layer uses **both** the orphan branch (same as above) **and** git notes. Notes are per-commit annotations that travel as refs: + +```bash +# Fetch notes from the remote +git fetch origin 'refs/notes/*:refs/notes/*' + +# Push notes to the remote +git push origin 'refs/notes/*:refs/notes/*' +``` + +**Why this is team-safe:** Notes are scoped to individual commits — there are no merge conflicts because each commit has its own annotation namespace. The orphan branch stores the aggregated permanent state, and Ralph promotes note data to it after PRs merge. + +### Automatic fetch in `squad watch` + +When `squad watch` starts, it automatically: +1. Fetches the `squad-state` branch (if orphan or two-layer) +2. Fetches `refs/notes/*` (if two-layer) +3. On each watch cycle, pushes any state changes back + +**No manual sync is needed** when using `squad watch`. Manual sync is only needed if you're running one-off squad commands outside of watch mode. + +### Git config for automatic notes fetch + +To make `git pull` automatically fetch notes, add this to `.git/config` (or use the setup script): + +```bash +# One-time setup per clone +git config --add remote.origin.fetch '+refs/notes/*:refs/notes/*' +``` + +After this, every `git fetch origin` includes notes automatically. + +--- + +## FAQ + +### What's the default state backend? + +**`local`**. If you don't set `stateBackend` in `.squad/config.json`, Squad stores state as regular files in `.squad/` on your working branch. This is the simplest setup — no extra configuration needed. + +### When should I switch away from `local`? + +Switch when any of these apply: +- Your PRs are cluttered with `.squad/` file changes +- You lose state when switching branches +- Multiple team members are getting merge conflicts on `.squad/` files +- You want squad state to be invisible in code reviews + +### Why would I choose `orphan` over `two-layer`? + +**Choose `orphan` when you want simplicity.** It stores all state on a single dedicated branch. Easy to understand, inspect, and debug. One branch, one source of truth. + +**Choose `two-layer` when you need commit-scoped context.** Two-layer adds git notes — annotations attached to specific commits. This means: +- A decision made on commit `abc123` stays linked to that commit +- Ralph can decide whether to promote or discard decisions based on whether the PR was merged or rejected +- Research notes on a rejected PR are automatically ignored (not promoted) + +**Bottom line:** `orphan` is for teams who just want clean PRs. `two-layer` is for teams who want intelligent state lifecycle management (decisions that survive or die with their PRs). + +### Can I use `orphan` and later upgrade to `two-layer`? + +Yes. Both use the same `squad-state` orphan branch for permanent state. Switching from `orphan` to `two-layer` simply enables the additional git notes layer. Your existing state is fully preserved. + +### What happens if two people run Squad simultaneously? + +- **Local backend:** File-level merge conflicts when both push (just like any Git merge conflict). +- **Orphan backend:** The second push to `squad-state` fails with a non-fast-forward error. Squad's watch loop retries automatically. In the worst case, you manually merge the branch. +- **Two-layer backend:** Notes are per-commit, so they never conflict. The orphan branch layer has the same retry behavior as the orphan backend. + +### Does the `squad-state` branch show up in my PRs? + +No. The `squad-state` branch is an **orphan branch** — it has no common ancestor with your main branch. GitHub doesn't include it in PR diffs. It's completely invisible in code reviews. + +### How do I inspect state on the orphan branch? + +```bash +# List all state files +git ls-tree --name-only -r squad-state + +# Read a specific file +git show squad-state:decisions.md + +# View state history +git log --oneline squad-state +``` + +### Does this work with GitHub Actions / CI? + +Yes. If your CI/CD workflow needs to read squad state: +- **Orphan backend:** `git fetch origin squad-state && git show squad-state:` +- **Two-layer:** Same as orphan, plus `git fetch origin 'refs/notes/*:refs/notes/*'` for notes +- **Local backend:** State is on the working branch — just read `.squad/` files directly + +### What if I forget to push the `squad-state` branch? + +State stays local to your machine. Other team members won't see your latest decisions or agent histories until you push. This is no different from forgetting to push any other branch — Git is distributed, and state only syncs when you push/fetch. + +### Can the `squad-state` branch be deleted safely? + +**No.** Deleting it loses all permanent squad state (decisions, agent histories, logs). Treat it like your main branch — push it to the remote and don't delete it. You can recover from a local deletion by re-fetching from the remote: `git fetch origin squad-state:squad-state`. diff --git a/docs/src/content/docs/reference/cli.md b/docs/src/content/docs/reference/cli.md index fcf9ceeb3..b3f9e76bd 100644 --- a/docs/src/content/docs/reference/cli.md +++ b/docs/src/content/docs/reference/cli.md @@ -28,6 +28,7 @@ squad init |---------|-------------|:------------------:| | `squad` | **Deprecated** — Enter interactive shell (no args). Use `copilot --agent squad` instead. | No | | `squad init` | Initialize Squad in the current repo (idempotent — safe to run multiple times) | No | +| `squad init --state-backend ` | Initialize with a specific state backend (`local`, `orphan`, `two-layer`) | No | | `squad init --global` | Create a personal squad in your platform-specific directory | No | | `squad init --mode remote ` | Initialize linked to a remote team root (dual-root mode) | No | | `squad link ` | Link project to a remote team root | Yes | @@ -38,6 +39,7 @@ squad init | `squad status` | Show which squad is active and why | Yes | | `squad doctor` | Validate squad setup integrity and diagnose issues (alias: `heartbeat`) | Yes | | `squad upgrade` | Upgrade Squad-owned files to latest version | Yes | +| `squad upgrade --state-backend ` | Migrate state backend (`orphan`, `two-layer`); installs git hooks automatically | Yes | | `squad upgrade --migrate-directory` | Rename legacy `.ai-team/` directory to `.squad/` | Yes | | `squad triage` | Auto-triage issues and assign to team (primary name; `watch` is an alias) | Yes | | `squad triage --interval ` | Continuous triage (default: every 10 min) | Yes | diff --git a/package-lock.json b/package-lock.json index cd4ea54a5..fb93d6934 100644 --- a/package-lock.json +++ b/package-lock.json @@ -8939,7 +8939,8 @@ }, "bin": { "squad": "dist/cli-entry.js", - "squad-cli": "dist/cli-entry.js" + "squad-cli": "dist/cli-entry.js", + "squad-test": "dist/cli-entry.js" }, "devDependencies": { "@types/node": "^22.0.0", diff --git a/packages/squad-cli/package.json b/packages/squad-cli/package.json index e84fa7aa5..45a5942ed 100644 --- a/packages/squad-cli/package.json +++ b/packages/squad-cli/package.json @@ -5,7 +5,8 @@ "type": "module", "bin": { "squad": "dist/cli-entry.js", - "squad-cli": "dist/cli-entry.js" + "squad-cli": "dist/cli-entry.js", + "squad-test": "dist/cli-entry.js" }, "exports": { ".": { diff --git a/packages/squad-cli/src/cli-entry.ts b/packages/squad-cli/src/cli-entry.ts index af2f6ce31..44c285bff 100644 --- a/packages/squad-cli/src/cli-entry.ts +++ b/packages/squad-cli/src/cli-entry.ts @@ -90,7 +90,8 @@ function _handleTopLevelSignal(signal: 'SIGINT' | 'SIGTERM'): void { process.on('SIGINT', () => _handleTopLevelSignal('SIGINT')); process.on('SIGTERM', () => _handleTopLevelSignal('SIGTERM')); -import { FSStorageProvider } from '@bradygaster/squad-sdk'; +import { FSStorageProvider, resolveSquadState } from '@bradygaster/squad-sdk'; +import type { SquadStateContext, StateBackendType } from '@bradygaster/squad-sdk'; import path from 'node:path'; import { fatal, SquadError } from './cli/core/errors.js'; import { BOLD, RESET, DIM, RED, GREEN, YELLOW } from './cli/core/output.js'; @@ -158,6 +159,7 @@ async function main(): Promise { console.log(` --global (personal squad dir)`); console.log(` --no-workflows (skip CI setup)`); console.log(` --preset (apply a preset after init)`); + console.log(` --state-backend (local|orphan|two-layer)`); console.log(` Usage: init --mode remote `); console.log(` Creates .squad/config.json pointing to an external team root`); console.log(` ${BOLD}upgrade${RESET} Update Squad-owned files to latest version`); @@ -165,6 +167,7 @@ async function main(): Promise { console.log(` Never touches: .squad/ or .ai-team/ (your team state)`); console.log(` Flags: --global (upgrade personal squad)`); console.log(` --migrate-directory (rename .ai-team/ → .squad/)`); + console.log(` --state-backend (migrate to orphan|two-layer)`); console.log(` ${BOLD}migrate${RESET} Convert between markdown and SDK-First squad formats`); console.log(` Flags: --to sdk|markdown, --from ai-team, --dry-run`); console.log(` ${BOLD}status${RESET} Show which squad is active and why`); @@ -316,8 +319,11 @@ async function main(): Promise { const roles = args.includes('--roles'); const presetIdx = args.indexOf('--preset'); const presetName = (presetIdx !== -1 && args[presetIdx + 1]) ? args[presetIdx + 1] : undefined; + // Parse --state-backend flag for init + const sbIdx = args.indexOf('--state-backend'); + const initStateBackend = (sbIdx !== -1 && args[sbIdx + 1]) ? args[sbIdx + 1] : undefined; // Global init: suppress workflows (no GitHub CI in ~/.config/squad/) and bootstrap personal squad - runInit(dest, { includeWorkflows: !noWorkflows && !hasGlobal, sdk, roles, isGlobal: hasGlobal }).then(async () => { + runInit(dest, { includeWorkflows: !noWorkflows && !hasGlobal, sdk, roles, isGlobal: hasGlobal, stateBackend: initStateBackend }).then(async () => { if (presetName) { const { seedBuiltinPresets, applyPreset } = await import('@bradygaster/squad-sdk/presets'); const { resolvePresetsDir, ensureSquadHome } = await import('@bradygaster/squad-sdk/resolution'); @@ -364,6 +370,10 @@ async function main(): Promise { const forceUpgrade = args.includes('--force'); const insider = args.includes('--insider'); const dest = hasGlobal ? (await lazySquadSdk()).resolveGlobalSquadPath() : getSquadStartDir(); + + // Parse --state-backend for backend migration + const sbIdx = args.indexOf('--state-backend'); + const upgradeStateBackend = (sbIdx !== -1 && args[sbIdx + 1]) ? args[sbIdx + 1] : undefined; // Warn when --insider is used without --self (it has no effect on project upgrades) if (insider && !selfUpgrade) { @@ -389,6 +399,16 @@ async function main(): Promise { self: selfUpgrade, force: forceUpgrade }); + + // Handle --state-backend: migrate backend after upgrade + if (upgradeStateBackend) { + const { migrateStateBackend } = await import('./cli/commands/migrate-backend.js'); + await migrateStateBackend(dest, upgradeStateBackend); + } else { + // Ensure hooks are installed for existing orphan/two-layer backends + const { ensureHooksForBackend } = await import('./cli/commands/install-hooks.js'); + ensureHooksForBackend(dest); + } return; } @@ -498,12 +518,16 @@ async function main(): Promise { const rawStateBackend = (stateBackendIdx !== -1 && args[stateBackendIdx + 1]) ? args[stateBackendIdx + 1] : undefined; - const validBackends = ['worktree', 'git-notes', 'orphan', 'external'] as const; + const validBackends = ['local', 'orphan', 'two-layer', 'external'] as const; if (rawStateBackend && !(validBackends as readonly string[]).includes(rawStateBackend)) { console.error(`\u26a0\ufe0f Invalid --state-backend "${rawStateBackend}". Valid: ${validBackends.join(', ')}.`); process.exit(1); } - const stateBackend = rawStateBackend as typeof validBackends[number] | undefined; + const mappedBackend = rawStateBackend as StateBackendType | undefined; + + // Resolve the full state context (paths + backend) once at entry. + // Commands can thread this through instead of re-resolving independently. + const stateContext: SquadStateContext | null = resolveSquadState(getSquadStartDir(), mappedBackend); // Build capability overrides from CLI flags and --no-{cap} flags const capabilities: Record> = {}; @@ -538,7 +562,8 @@ async function main(): Promise { overnightStart, overnightEnd, sentinelFile, - stateBackend, + stateBackend: mappedBackend, + stateContext, capabilities: Object.keys(capabilities).length > 0 ? capabilities : undefined, }); diff --git a/packages/squad-cli/src/cli/commands/install-hooks.ts b/packages/squad-cli/src/cli/commands/install-hooks.ts new file mode 100644 index 000000000..b0dfce04a --- /dev/null +++ b/packages/squad-cli/src/cli/commands/install-hooks.ts @@ -0,0 +1,259 @@ +/** + * Git Hook Installation — installs squad sync hooks into the repo's .git/hooks/. + * + * Hooks are installed with chaining: if a user already has a hook (e.g., from husky), + * the squad hook is appended and the existing hook is called first. + * + * Installed hooks: + * - pre-push: pushes squad-state branches alongside the user's push + * - post-merge: fetches squad-state after the user pulls + * - post-rewrite: fetches squad-state after rebase + * - post-checkout: fetches squad-state on branch switch + */ + +import { execFileSync } from 'node:child_process'; +import path from 'node:path'; +import fs from 'node:fs'; + +const GREEN = '\x1b[32m'; +const YELLOW = '\x1b[33m'; +const DIM = '\x1b[2m'; +const BOLD = '\x1b[1m'; +const RESET = '\x1b[0m'; + +const SQUAD_HOOK_MARKER = '# --- squad-sync-hook ---'; + +/** + * The shell script content for each hook. + * These are minimal wrappers that call `squad sync`. + * The SQUAD_SYNC_ACTIVE env guard prevents recursion. + */ +const HOOK_TEMPLATES: Record = { + 'pre-push': `#!/bin/sh +${SQUAD_HOOK_MARKER} +# Auto-push squad-state branches alongside the user's push. +# Installed by: squad init / squad upgrade --state-backend +# The remote name and URL are passed as arguments by git. +if [ -z "$SQUAD_SYNC_ACTIVE" ]; then + REMOTE="\$1" + export SQUAD_SYNC_ACTIVE=1 + # Push all squad-state branches (including subsquad branches) + for branch in $(git for-each-ref --format='%(refname:short)' 'refs/heads/squad-state' 'refs/heads/squad-state/*' 2>/dev/null); do + git push --no-verify "$REMOTE" "refs/heads/$branch:refs/heads/$branch" 2>/dev/null || true + done + # Push git notes for two-layer backend + git push --no-verify "$REMOTE" 'refs/notes/squad*:refs/notes/squad*' 2>/dev/null || true + unset SQUAD_SYNC_ACTIVE +fi +`, + 'post-merge': `#!/bin/sh +${SQUAD_HOOK_MARKER} +# Auto-fetch squad-state branches after pull/merge. +# Installed by: squad init / squad upgrade --state-backend +if [ -z "$SQUAD_SYNC_ACTIVE" ]; then + export SQUAD_SYNC_ACTIVE=1 + REMOTE=$(git config "branch.$(git symbolic-ref --short HEAD 2>/dev/null).remote" 2>/dev/null || echo origin) + # Fetch squad-state branches + git fetch "$REMOTE" '+refs/heads/squad-state:refs/remotes/'"$REMOTE"'/squad-state' '+refs/heads/squad-state/*:refs/remotes/'"$REMOTE"'/squad-state/*' 2>/dev/null || true + # Fast-forward local squad-state from remote + for remote_ref in $(git for-each-ref --format='%(refname:short)' "refs/remotes/$REMOTE/squad-state" "refs/remotes/$REMOTE/squad-state/*" 2>/dev/null); do + local_name=\${remote_ref#"$REMOTE/"} + local_sha=$(git rev-parse "refs/heads/$local_name" 2>/dev/null) || { git update-ref "refs/heads/$local_name" "$(git rev-parse "$remote_ref")" 2>/dev/null; continue; } + remote_sha=$(git rev-parse "$remote_ref" 2>/dev/null) || continue + [ "$local_sha" = "$remote_sha" ] && continue + git merge-base --is-ancestor "$local_sha" "$remote_sha" 2>/dev/null && git update-ref "refs/heads/$local_name" "$remote_sha" 2>/dev/null || true + done + # Fetch git notes for two-layer backend + git fetch "$REMOTE" '+refs/notes/squad*:refs/notes/squad*' 2>/dev/null || true + unset SQUAD_SYNC_ACTIVE +fi +`, + 'post-rewrite': `#!/bin/sh +${SQUAD_HOOK_MARKER} +# Auto-fetch squad-state branches after rebase. +# Installed by: squad init / squad upgrade --state-backend +if [ -z "$SQUAD_SYNC_ACTIVE" ]; then + export SQUAD_SYNC_ACTIVE=1 + REMOTE=$(git config "branch.$(git symbolic-ref --short HEAD 2>/dev/null).remote" 2>/dev/null || echo origin) + git fetch "$REMOTE" '+refs/heads/squad-state:refs/remotes/'"$REMOTE"'/squad-state' '+refs/heads/squad-state/*:refs/remotes/'"$REMOTE"'/squad-state/*' 2>/dev/null || true + for remote_ref in $(git for-each-ref --format='%(refname:short)' "refs/remotes/$REMOTE/squad-state" "refs/remotes/$REMOTE/squad-state/*" 2>/dev/null); do + local_name=\${remote_ref#"$REMOTE/"} + local_sha=$(git rev-parse "refs/heads/$local_name" 2>/dev/null) || { git update-ref "refs/heads/$local_name" "$(git rev-parse "$remote_ref")" 2>/dev/null; continue; } + remote_sha=$(git rev-parse "$remote_ref" 2>/dev/null) || continue + [ "$local_sha" = "$remote_sha" ] && continue + git merge-base --is-ancestor "$local_sha" "$remote_sha" 2>/dev/null && git update-ref "refs/heads/$local_name" "$remote_sha" 2>/dev/null || true + done + git fetch "$REMOTE" '+refs/notes/squad*:refs/notes/squad*' 2>/dev/null || true + unset SQUAD_SYNC_ACTIVE +fi +`, + 'post-checkout': `#!/bin/sh +${SQUAD_HOOK_MARKER} +# Auto-fetch squad-state branches on branch switch. +# Installed by: squad init / squad upgrade --state-backend +# Only run on branch checkout (3rd arg = 1), not file checkout. +if [ "\$3" = "1" ] && [ -z "$SQUAD_SYNC_ACTIVE" ]; then + export SQUAD_SYNC_ACTIVE=1 + REMOTE=$(git config "branch.$(git symbolic-ref --short HEAD 2>/dev/null).remote" 2>/dev/null || echo origin) + git fetch "$REMOTE" '+refs/heads/squad-state:refs/remotes/'"$REMOTE"'/squad-state' '+refs/heads/squad-state/*:refs/remotes/'"$REMOTE"'/squad-state/*' 2>/dev/null || true + for remote_ref in $(git for-each-ref --format='%(refname:short)' "refs/remotes/$REMOTE/squad-state" "refs/remotes/$REMOTE/squad-state/*" 2>/dev/null); do + local_name=\${remote_ref#"$REMOTE/"} + local_sha=$(git rev-parse "refs/heads/$local_name" 2>/dev/null) || { git update-ref "refs/heads/$local_name" "$(git rev-parse "$remote_ref")" 2>/dev/null; continue; } + remote_sha=$(git rev-parse "$remote_ref" 2>/dev/null) || continue + [ "$local_sha" = "$remote_sha" ] && continue + git merge-base --is-ancestor "$local_sha" "$remote_sha" 2>/dev/null && git update-ref "refs/heads/$local_name" "$remote_sha" 2>/dev/null || true + done + git fetch "$REMOTE" '+refs/notes/squad*:refs/notes/squad*' 2>/dev/null || true + unset SQUAD_SYNC_ACTIVE +fi +`, +}; + +export interface InstallHooksOptions { + force?: boolean; +} + +/** + * Get the .git/hooks directory path for the repo. + */ +function getHooksDir(cwd: string): string { + // Respect core.hooksPath if already set + try { + const customPath = execFileSync('git', ['config', '--get', 'core.hooksPath'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + if (customPath) { + return path.isAbsolute(customPath) ? customPath : path.resolve(cwd, customPath); + } + } catch { + // Not set — use default + } + + const gitDir = execFileSync('git', ['rev-parse', '--git-dir'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + + return path.resolve(cwd, gitDir, 'hooks'); +} + +/** + * Install a single hook, chaining with any existing hook. + */ +function installHook(hooksDir: string, hookName: string, content: string, force: boolean): 'installed' | 'chained' | 'skipped' { + const hookPath = path.join(hooksDir, hookName); + + // Check if hook already exists + if (fs.existsSync(hookPath)) { + const existing = fs.readFileSync(hookPath, 'utf-8'); + + // Already has our marker — skip unless force + if (existing.includes(SQUAD_HOOK_MARKER)) { + if (!force) return 'skipped'; + // Force: remove old squad section and re-append + const cleaned = existing.split('\n').filter(line => { + // Remove lines between markers + return true; // simplified: just replace the file + }).join('\n'); + // For simplicity on force, rewrite with chaining + } + + // Chain: existing hook runs first, then squad hook (without shebang) + const squadSection = content.split('\n').slice(1).join('\n'); // remove #!/bin/sh + const chained = existing.trimEnd() + '\n\n' + squadSection; + fs.writeFileSync(hookPath, chained, { mode: 0o755 }); + return 'chained'; + } + + // No existing hook — write fresh + fs.mkdirSync(hooksDir, { recursive: true }); + fs.writeFileSync(hookPath, content, { mode: 0o755 }); + return 'installed'; +} + +/** + * Main hook installation entrypoint. + */ +export function installGitHooks(cwd: string, options: InstallHooksOptions = {}): void { + const { force = false } = options; + + // Verify we're in a git repo + try { + execFileSync('git', ['rev-parse', '--git-dir'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + } catch { + console.log(`${YELLOW}⚠${RESET} Not a git repository. Cannot install hooks.`); + return; + } + + // Check if backend needs hooks (only orphan/two-layer) + let backend: string | null = null; + try { + const configPath = path.join(cwd, '.squad', 'config.json'); + if (fs.existsSync(configPath)) { + const config = JSON.parse(fs.readFileSync(configPath, 'utf-8')); + backend = config.stateBackend || null; + } + } catch { /* proceed anyway */ } + + if (backend === 'local' || backend === 'external' || backend === null) { + console.log(`${DIM}squad install-hooks: backend is '${backend || 'local'}' — hooks not needed (state syncs with normal git operations).${RESET}`); + return; + } + + const hooksDir = getHooksDir(cwd); + console.log(`\n${BOLD}Installing squad sync hooks${RESET}`); + console.log(`${DIM} hooks dir: ${hooksDir}${RESET}\n`); + + for (const [hookName, template] of Object.entries(HOOK_TEMPLATES)) { + const result = installHook(hooksDir, hookName, template, force); + switch (result) { + case 'installed': + console.log(` ${GREEN}✓${RESET} ${hookName}: installed`); + break; + case 'chained': + console.log(` ${GREEN}✓${RESET} ${hookName}: chained (existing hook preserved)`); + break; + case 'skipped': + console.log(` ${DIM} ${hookName}: already installed (use --force to reinstall)${RESET}`); + break; + } + } + + console.log(`\n${GREEN}${BOLD}Done.${RESET} Squad state will sync automatically on push/pull.\n`); +} + +/** + * Ensure hooks are installed if the backend requires them. + * Called by `squad upgrade` to silently ensure hooks exist for orphan/two-layer repos. + * Does not print anything if hooks are already installed or backend doesn't need them. + */ +export function ensureHooksForBackend(cwd: string): void { + // Check backend + let backend: string | null = null; + try { + const configPath = path.join(cwd, '.squad', 'config.json'); + if (fs.existsSync(configPath)) { + const config = JSON.parse(fs.readFileSync(configPath, 'utf-8')); + backend = config.stateBackend || null; + } + } catch { return; } + + // Only orphan/two-layer need hooks + if (backend !== 'orphan' && backend !== 'two-layer') return; + + // Check if hooks are already installed + let hooksDir: string; + try { + hooksDir = getHooksDir(cwd); + } catch { return; } + + const prePushPath = path.join(hooksDir, 'pre-push'); + if (fs.existsSync(prePushPath)) { + const content = fs.readFileSync(prePushPath, 'utf-8'); + if (content.includes(SQUAD_HOOK_MARKER)) return; // Already installed + } + + // Hooks missing — install them + installGitHooks(cwd, { force: false }); +} diff --git a/packages/squad-cli/src/cli/commands/migrate-backend.ts b/packages/squad-cli/src/cli/commands/migrate-backend.ts new file mode 100644 index 000000000..21b049a60 --- /dev/null +++ b/packages/squad-cli/src/cli/commands/migrate-backend.ts @@ -0,0 +1,97 @@ +/** + * Backend migration — upgrades state backend from local to orphan or two-layer. + * + * Currently supports: + * - local → orphan + * - local → two-layer + * + * Migration from orphan/two-layer back to local is not supported (would require + * materializing all state from the orphan branch back to the working tree). + */ + +import { execFileSync } from 'node:child_process'; +import path from 'node:path'; +import fs from 'node:fs'; +import { installGitHooks } from './install-hooks.js'; + +const GREEN = '\x1b[32m'; +const YELLOW = '\x1b[33m'; +const BOLD = '\x1b[1m'; +const RESET = '\x1b[0m'; + +const VALID_TARGETS = ['orphan', 'two-layer']; + +/** + * Migrate the state backend for an existing squad project. + * Only local → orphan/two-layer is supported. + */ +export async function migrateStateBackend(dest: string, target: string): Promise { + if (!VALID_TARGETS.includes(target)) { + console.log(`${YELLOW}⚠ Invalid backend target '${target}'. Supported: ${VALID_TARGETS.join(', ')}${RESET}`); + return; + } + + const configPath = path.join(dest, '.squad', 'config.json'); + let config: Record = {}; + + try { + if (fs.existsSync(configPath)) { + config = JSON.parse(fs.readFileSync(configPath, 'utf-8')); + } + } catch { /* start fresh */ } + + const current = (config['stateBackend'] as string) || 'local'; + + // Validate migration direction + if (current === target) { + console.log(`${YELLOW}⚠ Backend is already '${target}'. Nothing to do.${RESET}`); + return; + } + + if (current !== 'local' && current !== null) { + console.log(`${YELLOW}⚠ Migration from '${current}' to '${target}' is not supported.${RESET}`); + console.log(` Only local → orphan or local → two-layer is supported at this time.`); + return; + } + + console.log(`\n${BOLD}Migrating state backend: ${current} → ${target}${RESET}\n`); + + // Step 1: Create orphan branch if needed + try { + execFileSync('git', ['rev-parse', '--verify', 'refs/heads/squad-state'], { + cwd: dest, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + console.log(` ${GREEN}✓${RESET} squad-state branch already exists`); + } catch { + try { + const readmeContent = '# Squad State\n\nThis orphan branch stores mutable squad state.\nIt is managed automatically and should not be edited by hand.\n'; + const blobHash = execFileSync('git', ['hash-object', '-w', '--stdin'], { + cwd: dest, encoding: 'utf-8', input: readmeContent, stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const treeInput = `100644 blob ${blobHash}\tREADME.md\n`; + const treeHash = execFileSync('git', ['mktree'], { + cwd: dest, encoding: 'utf-8', input: treeInput, stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const commitHash = execFileSync('git', ['commit-tree', treeHash, '-m', 'init: squad-state orphan branch'], { + cwd: dest, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + execFileSync('git', ['update-ref', 'refs/heads/squad-state', commitHash], { + cwd: dest, stdio: ['pipe', 'pipe', 'pipe'], + }); + console.log(` ${GREEN}✓${RESET} squad-state orphan branch created`); + } catch (err) { + console.log(`${YELLOW}⚠ Could not create squad-state branch: ${err instanceof Error ? err.message : err}${RESET}`); + return; + } + } + + // Step 2: Update config + config['stateBackend'] = target; + fs.writeFileSync(configPath, JSON.stringify(config, null, 2) + '\n'); + console.log(` ${GREEN}✓${RESET} config.json updated: stateBackend = ${target}`); + + // Step 3: Install git hooks + installGitHooks(dest, { force: true }); + + console.log(`\n${GREEN}${BOLD}✓ Migration complete.${RESET} Backend is now '${target}'.\n`); +} diff --git a/packages/squad-cli/src/cli/commands/sync.ts b/packages/squad-cli/src/cli/commands/sync.ts new file mode 100644 index 000000000..eaeeacb8a --- /dev/null +++ b/packages/squad-cli/src/cli/commands/sync.ts @@ -0,0 +1,264 @@ +/** + * Squad Sync — synchronizes squad-state branches and git notes with a remote. + * + * Used directly (`squad sync`) or invoked by git hooks (pre-push, post-merge, post-rewrite). + * Handles both orphan and two-layer backends transparently. + * + * Design: + * - Fetches remote squad-state branch(es) and fast-forwards local refs + * - Pushes local squad-state branch(es) to remote + * - For two-layer, also syncs refs/notes/squad* namespaces + * - Uses fast-forward-only semantics to avoid data loss on divergence + * - Recursion guard via SQUAD_SYNC_ACTIVE env var + */ + +import { execFileSync } from 'node:child_process'; +import path from 'node:path'; +import fs from 'node:fs'; + +const SQUAD_SYNC_ENV = 'SQUAD_SYNC_ACTIVE'; +const STATE_BRANCH_PREFIX = 'squad-state'; + +export interface SyncOptions { + direction: 'push' | 'pull' | 'both'; + remote?: string; + cwd?: string; + quiet?: boolean; +} + +/** + * Detect the configured state backend from .squad/config.json + */ +function detectBackend(cwd: string): string | null { + try { + const configPath = path.join(cwd, '.squad', 'config.json'); + const raw = fs.readFileSync(configPath, 'utf-8'); + const config = JSON.parse(raw); + return config.stateBackend || null; + } catch { + return null; + } +} + +/** + * Get git repo root for the given working directory. + */ +function getRepoRoot(cwd: string): string { + return execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); +} + +/** + * Discover all local squad-state branches (supports subsquads). + * Uses strict prefix matching: squad-state or squad-state/ + */ +function discoverStateBranches(cwd: string): string[] { + try { + const output = execFileSync('git', ['for-each-ref', '--format=%(refname:short)', `refs/heads/${STATE_BRANCH_PREFIX}`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const exact = output ? output.split('\n').filter(Boolean) : []; + + // Also find squad-state/* (subsquad branches) + const subOutput = execFileSync('git', ['for-each-ref', '--format=%(refname:short)', `refs/heads/${STATE_BRANCH_PREFIX}/`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const subs = subOutput ? subOutput.split('\n').filter(Boolean) : []; + + return [...exact, ...subs]; + } catch { + return []; + } +} + +/** + * Discover remote squad-state branches. + */ +function discoverRemoteStateBranches(cwd: string, remote: string): string[] { + try { + const output = execFileSync('git', ['for-each-ref', '--format=%(refname:short)', `refs/remotes/${remote}/${STATE_BRANCH_PREFIX}`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const exact = output ? output.split('\n').filter(Boolean) : []; + + const subOutput = execFileSync('git', ['for-each-ref', '--format=%(refname:short)', `refs/remotes/${remote}/${STATE_BRANCH_PREFIX}/`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const subs = subOutput ? subOutput.split('\n').filter(Boolean) : []; + + return [...exact, ...subs]; + } catch { + return []; + } +} + +/** + * Resolve the default remote for the current branch (or fallback to 'origin'). + */ +function resolveRemote(cwd: string): string { + try { + const branch = execFileSync('git', ['symbolic-ref', '--short', 'HEAD'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const remote = execFileSync('git', ['config', `branch.${branch}.remote`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + return remote || 'origin'; + } catch { + return 'origin'; + } +} + +/** + * Pull: fetch remote state branches and fast-forward local refs. + */ +function syncPull(cwd: string, remote: string, backend: string | null, quiet: boolean): void { + // Fetch squad-state refs from remote + try { + execFileSync('git', ['fetch', remote, `+refs/heads/${STATE_BRANCH_PREFIX}:refs/remotes/${remote}/${STATE_BRANCH_PREFIX}`, `+refs/heads/${STATE_BRANCH_PREFIX}/*:refs/remotes/${remote}/${STATE_BRANCH_PREFIX}/*`], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + } catch { + // Remote may not have these refs yet — not an error + if (!quiet) console.log(' No remote squad-state refs found (first push will create them).'); + return; + } + + // Fast-forward local branches from remote-tracking refs + const remoteBranches = discoverRemoteStateBranches(cwd, remote); + for (const remoteBranch of remoteBranches) { + // remoteBranch is like "origin/squad-state" — extract local name + const localName = remoteBranch.replace(`${remote}/`, ''); + const localRef = `refs/heads/${localName}`; + const remoteRef = `refs/remotes/${remoteBranch}`; + + try { + // Check if local branch exists + execFileSync('git', ['rev-parse', '--verify', localRef], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + // Local exists — try fast-forward + const localSha = execFileSync('git', ['rev-parse', localRef], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + const remoteSha = execFileSync('git', ['rev-parse', remoteRef], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + + if (localSha === remoteSha) continue; // Already up-to-date + + // Check if fast-forward is possible (local is ancestor of remote) + try { + execFileSync('git', ['merge-base', '--is-ancestor', localSha, remoteSha], { + cwd, stdio: ['pipe', 'pipe', 'pipe'], + }); + // Fast-forward: update local ref + execFileSync('git', ['update-ref', localRef, remoteSha], { + cwd, stdio: ['pipe', 'pipe', 'pipe'], + }); + if (!quiet) console.log(` ✓ ${localName}: fast-forwarded`); + } catch { + // Diverged — cannot fast-forward + if (!quiet) console.log(` ⚠ ${localName}: diverged from remote (manual merge needed)`); + } + } catch { + // Local branch doesn't exist — create it tracking the remote + const remoteSha = execFileSync('git', ['rev-parse', remoteRef], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + execFileSync('git', ['update-ref', localRef, remoteSha], { + cwd, stdio: ['pipe', 'pipe', 'pipe'], + }); + if (!quiet) console.log(` ✓ ${localName}: created from remote`); + } + } + + // For two-layer: also fetch notes + if (backend === 'two-layer') { + try { + execFileSync('git', ['fetch', remote, '+refs/notes/squad*:refs/notes/squad*'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + if (!quiet) console.log(' ✓ notes synced'); + } catch { + // Notes may not exist yet + } + } +} + +/** + * Push: push local state branches to remote. + */ +function syncPush(cwd: string, remote: string, backend: string | null, quiet: boolean): void { + const branches = discoverStateBranches(cwd); + if (branches.length === 0) { + if (!quiet) console.log(' No local squad-state branches to push.'); + return; + } + + // Build refspecs for all state branches + const refspecs = branches.map(b => `refs/heads/${b}:refs/heads/${b}`); + + try { + execFileSync('git', ['push', '--no-verify', remote, ...refspecs], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + if (!quiet) console.log(` ✓ pushed: ${branches.join(', ')}`); + } catch (err: unknown) { + const msg = err instanceof Error ? (err as any).stderr || err.message : String(err); + if (msg.includes('non-fast-forward')) { + if (!quiet) console.log(` ⚠ push rejected (non-fast-forward). Run 'squad sync --pull' first.`); + } else { + if (!quiet) console.log(` ⚠ push failed: ${msg}`); + } + } + + // For two-layer: also push notes + if (backend === 'two-layer') { + try { + execFileSync('git', ['push', '--no-verify', remote, 'refs/notes/squad*:refs/notes/squad*'], { + cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + if (!quiet) console.log(' ✓ notes pushed'); + } catch { + // Notes may not exist yet — not an error + } + } +} + +/** + * Main sync entrypoint. + */ +export async function runSync(options: SyncOptions): Promise { + // Recursion guard — prevent re-entry when our push triggers pre-push + if (process.env[SQUAD_SYNC_ENV]) { + return; + } + process.env[SQUAD_SYNC_ENV] = '1'; + + try { + const cwd = options.cwd || process.cwd(); + const repoRoot = getRepoRoot(cwd); + const remote = options.remote || resolveRemote(repoRoot); + const backend = detectBackend(repoRoot); + const quiet = options.quiet ?? false; + + // Skip sync for backends that don't need it + if (backend === 'local' || backend === 'external' || backend === null) { + if (!quiet) console.log(`squad sync: backend is '${backend || 'local'}' — no remote sync needed.`); + return; + } + + if (!quiet) console.log(`squad sync: ${options.direction} (remote: ${remote}, backend: ${backend || 'orphan'})`); + + if (options.direction === 'pull' || options.direction === 'both') { + syncPull(repoRoot, remote, backend, quiet); + } + if (options.direction === 'push' || options.direction === 'both') { + syncPush(repoRoot, remote, backend, quiet); + } + } finally { + delete process.env[SQUAD_SYNC_ENV]; + } +} diff --git a/packages/squad-cli/src/cli/commands/watch/config.ts b/packages/squad-cli/src/cli/commands/watch/config.ts index 20caff3bf..2236c29ab 100644 --- a/packages/squad-cli/src/cli/commands/watch/config.ts +++ b/packages/squad-cli/src/cli/commands/watch/config.ts @@ -6,6 +6,7 @@ import path from 'node:path'; import { FSStorageProvider } from '@bradygaster/squad-sdk'; +import type { SquadStateContext, StateBackendType } from '@bradygaster/squad-sdk'; const storage = new FSStorageProvider(); @@ -40,7 +41,9 @@ export interface WatchConfig { /** Path to a sentinel file — watch shuts down gracefully when removed. */ sentinelFile?: string; /** State persistence backend. */ - stateBackend?: 'worktree' | 'git-notes' | 'orphan' | 'external'; + stateBackend?: StateBackendType; + /** Pre-resolved state context from CLI entry (avoids redundant resolution). */ + stateContext?: SquadStateContext | null; } const DEFAULTS: WatchConfig = { @@ -99,6 +102,7 @@ export function loadWatchConfig( overnightEnd: cliOverrides.overnightEnd ?? fileConfig.overnightEnd, sentinelFile: cliOverrides.sentinelFile ?? fileConfig.sentinelFile, stateBackend: cliOverrides.stateBackend ?? fileConfig.stateBackend, + stateContext: cliOverrides.stateContext, }; return merged; @@ -134,9 +138,9 @@ function normalizeFileConfig(raw: Record): Partial if (typeof raw['sentinelFile'] === 'string') result.sentinelFile = raw['sentinelFile']; if (typeof raw['stateBackend'] === 'string') { const backend = raw['stateBackend']; - const validBackends = ['worktree', 'git-notes', 'orphan', 'external'] as const; + const validBackends = ['local', 'orphan', 'two-layer', 'external'] as const; if ((validBackends as readonly string[]).includes(backend)) { - result.stateBackend = backend as typeof validBackends[number]; + result.stateBackend = backend as StateBackendType; } } diff --git a/packages/squad-cli/src/cli/core/init.ts b/packages/squad-cli/src/cli/core/init.ts index 017d0f35f..dd4eb6f99 100644 --- a/packages/squad-cli/src/cli/core/init.ts +++ b/packages/squad-cli/src/cli/core/init.ts @@ -12,6 +12,7 @@ import { fatal } from './errors.js'; import { detectProjectType } from './project-type.js'; import { getPackageVersion, stampVersion } from './version.js'; import { initSquad as sdkInitSquad, cleanupOrphanInitPrompt, ensurePersonalSquadDir, resolvePersonalSquadDir, type InitOptions } from '@bradygaster/squad-sdk'; +import { installGitHooks } from '../commands/install-hooks.js'; const storage = new FSStorageProvider(); @@ -108,6 +109,8 @@ export interface RunInitOptions { roles?: boolean; /** If true, this is a global (personal squad) init — bootstrap personal-squad/ dir */ isGlobal?: boolean; + /** State backend to configure at init time (local, orphan, two-layer) */ + stateBackend?: string; } /** @@ -263,6 +266,63 @@ export async function runInit(dest: string, options: RunInitOptions = {}): Promi success(`base roles enabled — team will use built-in role catalog`); } + // Configure state backend if specified at init time + if (options.stateBackend) { + const validBackends = ['local', 'orphan', 'two-layer', 'external']; + if (validBackends.includes(options.stateBackend)) { + const configPath = path.join(squadDir, 'config.json'); + let config: Record = {}; + try { + const raw = storage.readSync(configPath); + if (raw) config = JSON.parse(raw); + } catch { /* start fresh */ } + config['stateBackend'] = options.stateBackend; + storage.writeSync(configPath, JSON.stringify(config, null, 2) + '\n'); + success(`state backend: ${options.stateBackend}`); + + // Auto-create orphan branch for orphan/two-layer backends + // Uses git plumbing (mktree + commit-tree + update-ref) so the working tree is never touched. + if (options.stateBackend === 'orphan' || options.stateBackend === 'two-layer') { + try { + execFileSync('git', ['rev-parse', '--verify', 'refs/heads/squad-state'], { + cwd: dest, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + success(`squad-state branch already exists`); + } catch { + try { + // Seed a README blob so the branch isn't completely empty + const readmeContent = '# Squad State\n\nThis orphan branch stores mutable squad state.\nIt is managed automatically and should not be edited by hand.\n'; + const blobHash = execFileSync('git', ['hash-object', '-w', '--stdin'], { + cwd: dest, encoding: 'utf-8', input: readmeContent, stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + // Build a tree containing the README + const treeInput = `100644 blob ${blobHash}\tREADME.md\n`; + const treeHash = execFileSync('git', ['mktree'], { + cwd: dest, encoding: 'utf-8', input: treeInput, stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + // Create the root commit (no parent → orphan) + const commitHash = execFileSync('git', ['commit-tree', treeHash, '-m', 'init: squad-state orphan branch'], { + cwd: dest, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + // Point the branch ref at the new commit + execFileSync('git', ['update-ref', 'refs/heads/squad-state', commitHash], { + cwd: dest, stdio: ['pipe', 'pipe', 'pipe'], + }); + success(`squad-state orphan branch created (working tree untouched)`); + } catch (err) { + console.warn(`${YELLOW}⚠ Could not create squad-state branch: ${err instanceof Error ? err.message : err}${RESET}`); + console.warn(`${YELLOW} The ${options.stateBackend} backend will auto-create it on first write.${RESET}`); + } + } + + // Install git hooks for automatic state sync on push/pull + installGitHooks(dest, { force: false }); + } + } else { + console.warn(`${YELLOW}⚠ Unknown state backend "${options.stateBackend}". Using default (local).${RESET}`); + } + } + // Report .init-prompt storage if (options.prompt) { success(`.init-prompt stored — team will be cast when you run ${CYAN}${BOLD}copilot --agent squad${RESET}`); diff --git a/packages/squad-cli/templates/notes-protocol.md b/packages/squad-cli/templates/notes-protocol.md new file mode 100644 index 000000000..5a1d03a5f --- /dev/null +++ b/packages/squad-cli/templates/notes-protocol.md @@ -0,0 +1,202 @@ +# Squad Notes Protocol + +> Contract for agent state via git notes. Agents write commit-scoped context +> here instead of modifying `.squad/` files in PRs. +> +> **Version:** 1.0 +> **Backends:** `git-notes`, `orphan` + +--- + +## Overview + +Squad state has two layers: + +1. **Git notes layer** (this document) — thin, commit-scoped annotations that + attach agent context to commits without appearing in PRs or diffs. +2. **Permanent state layer** — long-lived decisions, routing rules, and archives + stored via the configured state backend (`git-notes` or `orphan` branch). + +Agents write notes during their work rounds. Ralph promotes flagged notes to +permanent state after a PR merges. + +--- + +## Namespaces + +Each agent writes to its own namespace to prevent conflicts: + +| Namespace | Owner | Purpose | +|-----------|-------|---------| +| `refs/notes/squad/data` | Data | Architecture decisions, implementation choices | +| `refs/notes/squad/worf` | Worf | Security reviews, vulnerability assessments | +| `refs/notes/squad/seven` | Seven | Documentation quality, API contract decisions | +| `refs/notes/squad/ralph` | Ralph | Work-round progress, task-state annotations | +| `refs/notes/squad/q` | Q | Devil's advocate findings, risk assessments | +| `refs/notes/squad/research` | Any agent | Research notes that should survive branch deletion | +| `refs/notes/squad/review` | Any agent | Code review context (mirrors Gerrit's pattern) | + +**Rule**: Only write to your own namespace. The shared namespaces +(`research`, `review`) use `append` — never `add`. + +--- + +## Note JSON Schema + +All notes MUST be valid JSON. Minimum required fields: + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision | research | review | progress | security", + "content": "..." +} +``` + +### Decision notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision", + "decision": "Use JWT RS256 for auth middleware", + "reasoning": "Existing pattern in codebase — auth.go:47-89.", + "alternatives_considered": ["HS256", "session tokens"], + "confidence": "high", + "promote_to_permanent": true +} +``` + +Set `"promote_to_permanent": true` to signal Ralph to copy this to +`decisions.md` after the PR merges. + +### Research notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "research", + "topic": "JWT vs session tokens", + "findings": {}, + "effort_hours": 2.5, + "archive_on_close": true +} +``` + +Set `"archive_on_close": true` to signal Ralph to archive this to +`state/research/` even if the PR is rejected. + +--- + +## Write Commands + +```bash +# Write a decision note on the current commit +git notes --ref=squad/{your-agent} add \ + -m '{"agent":"{Agent}","timestamp":"...","type":"decision","decision":"..."}' \ + HEAD + +# Append to an existing note (multiple items on same commit) +git notes --ref=squad/{your-agent} append \ + -m '{"agent":"{Agent}","timestamp":"...","type":"progress","content":"..."}' \ + HEAD + +# Read your note +git notes --ref=squad/{your-agent} show HEAD + +# List all commits with notes in your namespace +git notes --ref=squad/{your-agent} list +``` + +Or use the helper script: + +```powershell +./scripts/notes/write-note.ps1 -Agent data -Type decision \ + -Content '{"decision":"Use JWT","reasoning":"..."}' \ + [-Commit HEAD] [-Promote] [-Archive] +``` + +--- + +## Fetch / Push + +**Notes are NOT fetched or pushed by default.** Every clone needs setup. + +### One-time setup + +```bash +git config --add remote.origin.fetch 'refs/notes/*:refs/notes/*' +git fetch origin 'refs/notes/*:refs/notes/*' +``` + +Or use the helper: + +```powershell +./scripts/notes/fetch.ps1 -Setup +``` + +### Every work round + +1. **Start**: `git fetch origin 'refs/notes/*:refs/notes/*'` +2. **End**: `git push origin 'refs/notes/*:refs/notes/*'` + +--- + +## Conflict Handling + +1. **Per-agent namespaces prevent 99% of conflicts.** Only one agent writes to + `refs/notes/squad/data`, so there are no write conflicts in normal use. + +2. **Same agent, two machines:** First push wins. Losing machine should fetch + and append: + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes --ref=squad/{agent} append -m '{...}' HEAD + git push origin 'refs/notes/*:refs/notes/*' + ``` + +3. **Shared namespaces** (`research`, `review`): Always use `git notes append`, + never `git notes add`. + +4. **Push conflict recovery:** + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes merge refs/notes/remotes/origin/squad/{namespace} + git push origin 'refs/notes/*:refs/notes/*' + ``` + +--- + +## When to Use Notes vs State Backend + +| Use git notes | Use state backend | +|---------------|-------------------| +| Why THIS choice on THIS commit | Universal routing rules, conventions | +| Decisions scoped to a feature | Long-lived decisions for all future work | +| Research for a specific investigation | Research archives (promoted from notes) | +| Security sign-offs per commit | Agent history persisting across features | +| Agent-to-agent context for current feature | Team agreements and policies | + +When in doubt: **notes first, promote to permanent state later.** Ralph handles +the promotion automatically when `promote_to_permanent` is set. + +--- + +## Ralph Promotion Rules + +**After PR merge:** + +1. Fetch all notes from remote +2. Traverse commits reachable from the default branch that have notes +3. For each note with `"promote_to_permanent": true` → append to `decisions.md` +4. Push state + +**After PR close/rejection:** + +1. List notes in `squad/research` on the closed branch's commits +2. For each note with `"archive_on_close": true` → archive to `research/` +3. Push state +4. Notes on rejected commits are NOT promoted — this is the desired behavior diff --git a/packages/squad-cli/templates/scribe-charter.md b/packages/squad-cli/templates/scribe-charter.md index 308b780e2..a2031d072 100644 --- a/packages/squad-cli/templates/scribe-charter.md +++ b/packages/squad-cli/templates/scribe-charter.md @@ -24,6 +24,59 @@ **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory). +**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. If it's `"orphan"` or `"git-notes"`, run the **State Leak Guard** before any other work. + +### State Leak Guard (orphan/git-notes backends only) + +Before logging or merging, check if any agent accidentally committed state files to the working branch: + +```powershell +# Check if state files are staged or committed but shouldn't be +$stateFiles = @( + '.squad/decisions.md', + '.squad/decisions-archive.md' +) +$statePatterns = @( + '.squad/agents/*/history.md', + '.squad/agents/*/history-archive.md', + '.squad/log/*', + '.squad/orchestration-log/*', + '.squad/decisions/inbox/*' +) + +# 1. Check git status for accidentally staged state files +$dirty = git status --porcelain | Where-Object { $_.Length -gt 3 } | ForEach-Object { + $_.Substring(3) -replace '^.* -> ','' +} | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($dirty) { + # Unstage any accidentally added state files + $dirty | ForEach-Object { git reset HEAD -- $_ 2>$null } + # Restore from HEAD (discard working tree changes for state files) + $dirty | ForEach-Object { git checkout HEAD -- $_ 2>$null } +} + +# 2. Check if the most recent commit on this branch has state files +$lastCommitFiles = git diff-tree --no-commit-id --name-only -r HEAD 2>$null +$leakedInCommit = $lastCommitFiles | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($leakedInCommit) { + # State files leaked into the last commit — amend to remove them + $leakedInCommit | ForEach-Object { git rm --cached -- $_ 2>$null } + git commit --amend --no-edit 2>$null +} +``` + +If any files were cleaned, log: `⚠️ State leak guard: removed {N} state file(s) from working branch.` + +After the guard, proceed with normal Scribe work (but persist state via the configured backend, not the working branch). + After every substantial work session: 1. **Log the session** to `.squad/log/{timestamp}-{topic}.md`: @@ -57,9 +110,50 @@ After every substantial work session: ``` 5. **Commit `.squad/` changes:** + **Check `STATE_BACKEND` from spawn prompt.** This determines WHERE state gets committed. + **IMPORTANT — Windows compatibility:** Do NOT use `git -C {path}` (unreliable with Windows paths). Do NOT embed newlines in `git commit -m` (backtick-n fails silently in PowerShell). - Instead: + + **If STATE_BACKEND is "orphan":** + State files must be committed to the `squad-state` orphan branch, NOT the working branch. + - Identify changed `.squad/` state files via `git status --porcelain` filtered to allowed paths. + - For each file, use git plumbing to write to the orphan branch: + ```powershell + # Create a temporary worktree for the orphan branch + $orphanWt = Join-Path ([System.IO.Path]::GetTempPath()) "squad-state-$(Get-Random)" + git worktree add $orphanWt squad-state 2>$null + if ($LASTEXITCODE -ne 0) { git worktree add --orphan $orphanWt squad-state } + # Copy state files to orphan worktree + $filesToSync | ForEach-Object { + $dest = Join-Path $orphanWt $_ + New-Item -ItemType Directory -Path (Split-Path $dest) -Force | Out-Null + Copy-Item $_ $dest -Force + } + # Commit in orphan worktree + Push-Location $orphanWt + git add .squad/ + git diff --cached --quiet + if ($LASTEXITCODE -ne 0) { + $msgFile = [System.IO.Path]::GetTempFileName() + Set-Content -Path $msgFile -Value "docs(ai-team): $summary" -Encoding utf8 + git commit -F $msgFile + Remove-Item $msgFile + git push origin squad-state + } + Pop-Location + git worktree remove $orphanWt --force + ``` + - After committing to orphan, reset working tree state files: `git checkout HEAD -- .squad/` + - ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + + **If STATE_BACKEND is "git-notes":** + State is already persisted in git notes refs by agents. Scribe only needs to: + - Push any locally created note refs: `git push origin 'refs/notes/squad/*'` + - Commit decisions.md (the merged canonical file) to the working branch as normal. + + **If STATE_BACKEND is "worktree" (default):** + Commit to the working branch as normal: - `cd` into the team root first. - Stage only files Scribe actually modified in this session. Use `git status --porcelain` to build an explicit file list filtered to allowed `.squad/` paths: diff --git a/packages/squad-cli/templates/scripts/notes/fetch.ps1 b/packages/squad-cli/templates/scripts/notes/fetch.ps1 new file mode 100644 index 000000000..5adc19480 --- /dev/null +++ b/packages/squad-cli/templates/scripts/notes/fetch.ps1 @@ -0,0 +1,88 @@ +#!/usr/bin/env pwsh +# scripts/notes/fetch.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Fetch git notes from remote. Run on every Ralph-watch startup and before +# any agent reads or writes notes. +# +# Usage: +# ./scripts/notes/fetch.ps1 # fetch only +# ./scripts/notes/fetch.ps1 -Setup # first-time: add refspec + fetch +# ./scripts/notes/fetch.ps1 -Merge # fetch + merge (use after push conflict) +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [string]$Remote = "origin", + [string]$RepoPath = ".", + [switch]$Setup, + [switch]$Merge, + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/fetch] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath + +# ── One-time setup: add fetch refspec ────────────────────────────────────── +if ($Setup) { + $existing = git -C $repo config --get-all "remote.$Remote.fetch" 2>&1 | + Where-Object { $_ -match "refs/notes" } + if ($existing) { + Log "Notes refspec already configured." DarkGray + } else { + git -C $repo config --add "remote.$Remote.fetch" "refs/notes/*:refs/notes/*" + Log "Added notes refspec to remote.$Remote.fetch" Green + } +} + +# ── Fetch notes ───────────────────────────────────────────────────────────── +Log "Fetching notes from $Remote..." +$output = git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 +if ($LASTEXITCODE -ne 0) { + Log "Fetch warning: $output" DarkYellow +} else { + Log "Notes fetched." Green +} + +# ── Merge notes if requested (after push conflict) ────────────────────────── +if ($Merge) { + # Abort any stale merge-in-progress state + $mergeLock = Join-Path $repo ".git/NOTES_MERGE_PARTIAL" + if (Test-Path $mergeLock) { + Log "Stale notes merge in progress — aborting before retry" DarkYellow + git -C $repo notes merge --abort 2>&1 | Out-Null + } + + $namespaces = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + foreach ($ref in $namespaces) { + $ns = $ref -replace "refs/notes/", "" + $remoteRef = "refs/notes/remotes/$Remote/$ns" + $remoteExists = git -C $repo for-each-ref $remoteRef --format="%(refname)" 2>&1 + if ($remoteExists) { + Log "Merging notes: $ns (cat_sort_uniq)" + git -C $repo notes --ref=$ns merge -s cat_sort_uniq $remoteRef 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Log " Merge failed on $ns — aborting and continuing" Red + git -C $repo notes merge --abort 2>&1 | Out-Null + } + } + } + Log "Notes merge complete." Green +} + +# ── Show available namespaces ──────────────────────────────────────────────── +if (-not $Quiet) { + $refs = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + if ($refs) { + Log "Available namespaces:" + foreach ($r in $refs) { + $count = (git -C $repo notes --ref=($r -replace "refs/notes/","") list 2>&1 | + Where-Object { $_ -ne "" } | Measure-Object -Line).Lines + Log " $r ($count notes)" DarkGray + } + } else { + Log "No squad notes yet." DarkGray + } +} diff --git a/packages/squad-cli/templates/scripts/notes/write-note.ps1 b/packages/squad-cli/templates/scripts/notes/write-note.ps1 new file mode 100644 index 000000000..dbc719d35 --- /dev/null +++ b/packages/squad-cli/templates/scripts/notes/write-note.ps1 @@ -0,0 +1,126 @@ +#!/usr/bin/env pwsh +# scripts/notes/write-note.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Helper for agents to write notes without wrestling with JSON escaping. +# Validates namespace ownership, handles conflicts, pushes automatically. +# +# Usage: +# ./scripts/notes/write-note.ps1 -Agent data -Type decision \ +# -Content '{"decision":"Use JWT","reasoning":"..."}' \ +# [-Commit HEAD] [-Promote] [-Archive] +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [Parameter(Mandatory)][string]$Agent, + + [Parameter(Mandatory)] + [ValidateSet("decision","research","review","security-review","progress", + "api-contract","risk-assessment","routing-discovery","counter-argument")] + [string]$Type, + + [Parameter(Mandatory)] + [string]$Content, # JSON object with type-specific fields + + [string]$Commit = "HEAD", + [string]$RepoPath = ".", + [string]$Remote = "origin", + [switch]$Promote, # set promote_to_permanent: true + [switch]$Archive, # set archive_on_close: true + [switch]$NoPush, # skip auto-push + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/write] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath +$namespace = "squad/$($Agent.ToLower())" + +# ── Validate JSON content ──────────────────────────────────────────────────── +try { + $parsed = $Content | ConvertFrom-Json -ErrorAction Stop +} catch { + Write-Error "Content must be valid JSON. Got: $Content" + exit 1 +} + +# ── Build full note object ──────────────────────────────────────────────────── +$note = [ordered]@{ + agent = (Get-Culture).TextInfo.ToTitleCase($Agent.ToLower()) + timestamp = [System.DateTime]::UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") + type = $Type +} + +# Merge content fields into note +$parsed.PSObject.Properties | ForEach-Object { $note[$_.Name] = $_.Value } + +# Add flag fields +if ($Promote) { $note["promote_to_permanent"] = $true } +if ($Archive) { $note["archive_on_close"] = $true } + +$noteJson = $note | ConvertTo-Json -Compress -Depth 10 + +# ── Fetch first to avoid conflicts ─────────────────────────────────────────── +Log "Fetching notes before write..." +git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 | Out-Null + +# ── Check if note already exists on this commit ───────────────────────────── +$existing = git -C $repo notes --ref=$namespace show $Commit 2>&1 +$useAppend = ($LASTEXITCODE -eq 0) + +if ($useAppend) { + Log "Note exists on $Commit — appending" DarkYellow + git -C $repo notes --ref=$namespace append -m $noteJson $Commit +} else { + git -C $repo notes --ref=$namespace add -m $noteJson $Commit +} + +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to write note to refs/notes/$namespace on $Commit" + exit 1 +} + +Log "Note written to refs/notes/$namespace on $($Commit.Substring(0,[Math]::Min(8,$Commit.Length)))" Green + +# ── Push with retry ────────────────────────────────────────────────────────── +if (-not $NoPush) { + $maxRetries = 5 + $nsRef = "refs/notes/$namespace" + + for ($i = 0; $i -lt $maxRetries; $i++) { + Log "Pushing notes (attempt $($i+1))..." + $pushOut = git -C $repo push $Remote "${nsRef}:${nsRef}" 2>&1 + if ($LASTEXITCODE -eq 0) { + Log "Notes pushed successfully." Green + break + } + + if ($pushOut -match "non-fast-forward|fetch first|rejected") { + Log "Push conflict — fetch-first retry..." DarkYellow + + # Force-fetch: overwrite local ref with current remote state + git -C $repo fetch $Remote "${nsRef}:${nsRef}" 2>&1 | Out-Null + + # Re-append our note on top of the now-current remote state + git -C $repo notes --ref=$namespace append -m $noteJson $Commit 2>&1 | Out-Null + + $jitter = Get-Random -Minimum 0 -Maximum 1000 + $sleep = [Math]::Pow(2, $i) + $jitter / 1000 + Start-Sleep -Seconds $sleep + + } else { + Log "Push error: $pushOut" Red + if ($i -eq $maxRetries - 1) { + Write-Warning "Failed after $maxRetries retries. Push manually: git push origin '${nsRef}:${nsRef}'" + } + } + } +} + +# ── Show result ─────────────────────────────────────────────────────────────── +if (-not $Quiet) { + Log "Note content:" + $note | ConvertTo-Json -Depth 5 | Write-Host -ForegroundColor DarkGray +} diff --git a/packages/squad-cli/templates/squad.agent.md.template b/packages/squad-cli/templates/squad.agent.md.template index 01e18dfad..4ef1a3045 100644 --- a/packages/squad-cli/templates/squad.agent.md.template +++ b/packages/squad-cli/templates/squad.agent.md.template @@ -107,6 +107,8 @@ The `union` merge driver keeps all lines from both sides, which is correct for a **On every session start:** Run `git config user.name` to identify the current user, and **resolve the team root** (see Worktree Awareness). Store the team root — all `.squad/` paths must be resolved relative to it. Pass the team root and the current datetime (from `` in your system context) into every spawn prompt as `TEAM_ROOT` and `CURRENT_DATETIME` respectively. Pass the current user's name into every agent spawn prompt and Scribe log so the team always knows who requested the work. Check `.squad/identity/now.md` if it exists — it tells you what the team was last focused on. Update it if the focus has shifted. +**Resolve state backend:** Read `.squad/config.json` and check the `stateBackend` field. Valid values: `"worktree"` (default), `"git-notes"`, `"orphan"`, `"two-layer"`. Store as `STATE_BACKEND` and pass it into every spawn prompt. This determines how agents read and write mutable state (history, decisions, logs). Static config (charters, team.md, routing.md) always lives on disk regardless of backend. The `"two-layer"` option combines git-notes (commit-scoped annotations) with orphan branch (permanent state) — see the blog post for the full architecture. + **⚡ Context caching:** After the first message in a session, `team.md`, `routing.md`, and `registry.json` are already in your context. Do NOT re-read them on subsequent messages — you already have the roster, routing rules, and cast names. Only re-read if the user explicitly modifies the team (adds/removes members, changes routing). **Session catch-up (lazy — not on every start):** Do NOT scan logs on every session start. Only provide a catch-up summary when: @@ -223,13 +225,16 @@ The `name` parameter generates the human-readable agent ID shown in the tasks pa **When you detect a directive:** -1. Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: - ``` - ### {timestamp}: User directive - **By:** {user name} (via Copilot) - **What:** {the directive, verbatim or lightly paraphrased} - **Why:** User request — captured for team memory - ``` +1. Capture the directive: + - **worktree/orphan backend:** Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: + ``` + ### {timestamp}: User directive + **By:** {user name} (via Copilot) + **What:** {the directive, verbatim or lightly paraphrased} + **Why:** User request — captured for team memory + ``` + - **git-notes backend:** Persist via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/directives" -Content '{"timestamp": "{timestamp}", "by": "{user name}", "what": "...", "why": "User request"}'` 2. Acknowledge briefly: `"📌 Captured. {one-line summary of the directive}."` 3. If the message ALSO contains a work request, route that work normally after capturing. If it's directive-only, you're done — no agent spawn needed. @@ -344,7 +349,12 @@ prompt: | TARGET FILE(S): {exact file path(s)} Do the work. Keep it focused. + {% if STATE_BACKEND == "git-notes" %} + If you made a meaningful decision, persist it via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"decision": {"title": "...", "what": "...", "why": "..."}}'` + {% else %} If you made a meaningful decision, write to .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} ⚠️ OUTPUT: Report outcomes in human terms. Never expose tool internals or SQL. ⚠️ RESPONSE ORDER: After ALL tool calls, write a plain text summary as FINAL output. @@ -598,8 +608,9 @@ When the user gives any task, the Coordinator MUST: To enable full parallelism, shared writes use a drop-box pattern that eliminates file conflicts: **decisions.md** — Agents do NOT write directly to `decisions.md`. Instead: -- Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` -- Scribe merges inbox entries into the canonical `.squad/decisions.md` and clears the inbox +- **worktree/orphan backend:** Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` +- **git-notes backend:** Agents persist decisions via their git notes ref (see State Protocol). Scribe reads all agent notes and merges decisions. +- Scribe merges into the canonical `.squad/decisions.md` and clears the inbox (or note entries) - All agents READ from `.squad/decisions.md` at spawn time (last-merged snapshot) **orchestration-log/** — Scribe writes one entry per agent after each batch: @@ -799,7 +810,71 @@ prompt: | - Commit and push from the worktree {% endif %} + STATE_BACKEND: {state_backend} + + {% if STATE_BACKEND == "git-notes" %} + ## State Protocol — Git Notes + This project uses git-notes for mutable state. **DO NOT write to `.squad/` files for state.** + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading your state:** + Run: `powershell .squad/scripts/notes/fetch.ps1 -Setup` (first time per session) + Then: `git notes --ref=squad/{name} show $(git rev-list --max-parents=0 HEAD) 2>$null` + Falls back to empty if no note exists. + + **Writing state (history, decisions, learnings):** + Run: `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{json}'` + The helper handles JSON validation, conflict retry, and push. + + **Decisions:** Write decisions as JSON via your note ref. Scribe will merge them. + **Skills:** Skills are static config — write to `.squad/skills/` on disk as normal. + {% endif %} + + {% if STATE_BACKEND == "orphan" %} + ## State Protocol — Orphan Branch + This project uses an orphan branch (`squad-state`) for mutable state. + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading state:** Read `.squad/` files on disk — they are synced from the orphan branch. + **Writing state:** Write to `.squad/` files on disk as normal during your session. + Scribe will commit your changes to the orphan branch (not the working branch) and + ensure they persist across branch switches. + + **Important:** Do NOT commit `.squad/` state files to the working branch yourself. + Scribe handles the orphan branch commit workflow. + {% endif %} + + {% if STATE_BACKEND == "two-layer" %} + ## State Protocol — Two-Layer (Git Notes + Orphan Branch) + This project uses the two-layer architecture from Tamir's blog: + - **Layer 1 (git notes):** Commit-scoped "why" annotations — invisible in PRs + - **Layer 2 (orphan branch):** Permanent state store — decisions, histories, logs + + Static config (charters, team.md, routing.md) is on disk as normal. + + **During your session:** + 1. Write commit-scoped annotations as git notes on HEAD: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + 2. Write bulk state (history, logs) to `.squad/` files on disk — Scribe moves them to the orphan branch. + + **Note flags:** + - `"promote_to_permanent": true` — Ralph promotes this to decisions.md after PR merge + - `"archive_on_close": true` — Worth keeping even if PR is rejected (valuable research) + - Neither flag — silently ignored if PR is rejected (correct for branch-specific decisions) + + **Important:** Do NOT commit `.squad/` state files to the working branch. + Scribe handles orphan commits. Ralph handles note promotion. + {% endif %} + + {% if STATE_BACKEND == "worktree" or STATE_BACKEND is not defined %} Read .squad/agents/{name}/history.md (your project knowledge). + {% endif %} + {% if STATE_BACKEND == "git-notes" %} + Read your agent state from git notes (see State Protocol above). + {% endif %} + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + Read .squad/agents/{name}/history.md (your project knowledge — synced from orphan branch). + {% endif %} Read .squad/decisions.md (team decisions to respect). If .squad/identity/wisdom.md exists, read it before starting work. If .squad/identity/now.md exists, read it at spawn time. @@ -823,10 +898,27 @@ prompt: | ⚠️ DATES: When writing dates in any file (decisions, history, logs), use ONLY the CURRENT_DATETIME value above. Never infer or guess the date. AFTER work: + {% if STATE_BACKEND == "git-notes" %} + 1. Persist your learnings as JSON via the State Protocol: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"learnings": ["..."], "timestamp": "{current_datetime}"}'` + 2. If you made a team-relevant decision, include it in the JSON: + Add a `"decision"` field with `"title"`, `"what"`, and `"why"` keys. + Scribe will merge decisions into the canonical decisions.md. + {% elif STATE_BACKEND == "two-layer" %} + 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": + architecture decisions, patterns, user preferences, key file paths. + (Scribe commits this to the orphan branch.) + 2. If you made a team-relevant decision, write BOTH: + a. A git note on HEAD with promote flag: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + b. A drop file: .squad/decisions/inbox/{name}-{brief-slug}.md + (Scribe merges to orphan branch; Ralph promotes note after PR merge.) + {% else %} 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": architecture decisions, patterns, user preferences, key file paths. 2. If you made a team-relevant decision, write to: .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} 3. SKILL EXTRACTION: If you found a reusable pattern, write/update .squad/skills/{skill-name}/SKILL.md (read templates/skill.md for format). @@ -878,18 +970,47 @@ prompt: | You are the Scribe. Read .squad/agents/scribe/charter.md. TEAM ROOT: {team_root} CURRENT_DATETIME: {current_datetime} + STATE_BACKEND: {state_backend} SPAWN MANIFEST: {spawn_manifest} Tasks (in order): - 0. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "git-notes" or STATE_BACKEND == "two-layer" %} + 0. STATE LEAK GUARD: Check if any agent accidentally committed or staged state files + (.squad/decisions.md, agents/*/history.md, log/*, orchestration-log/*, decisions/inbox/*) + to the working branch. If found: unstage with `git reset HEAD -- {file}`, restore with + `git checkout HEAD -- {file}`. If leaked in last commit, amend to remove. Log count. + {% endif %} + 0b. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. 1. DECISIONS ARCHIVE [HARD GATE]: If decisions.md >= 20480 bytes, archive entries older than 30 days NOW. If >= 51200 bytes, archive entries older than 7 days. Do not skip this step. + {% if STATE_BACKEND == "git-notes" %} + 2. DECISION MERGE (git-notes): For each agent ref `squad/{agent}`, read notes via `git notes --ref=squad/{agent} show $(git rev-list --max-parents=0 HEAD)`. Extract any `decision` entries. Merge into decisions.md. Clear the decision field by overwriting the note without it. + {% elif STATE_BACKEND == "two-layer" %} + 2. DECISION MERGE (two-layer): Merge .squad/decisions/inbox/ → decisions.md AND read agent note refs for any decisions with `promote_to_permanent`. Deduplicate. Push note refs: `git push origin 'refs/notes/squad/*'` + {% else %} 2. DECISION INBOX: Merge .squad/decisions/inbox/ → decisions.md, delete inbox files. Deduplicate. + {% endif %} 3. ORCHESTRATION LOG: Write .squad/orchestration-log/{timestamp}-{agent}.md per agent. Use ISO 8601 UTC timestamp. 4. SESSION LOG: Write .squad/log/{timestamp}-{topic}.md. Brief. Use ISO 8601 UTC timestamp. + {% if STATE_BACKEND == "git-notes" %} + 5. CROSS-AGENT (git-notes): For team updates, write to affected agents' note refs via `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{agent}" -Content '{json}'`. + {% else %} 5. CROSS-AGENT: Append team updates to affected agents' history.md. + {% endif %} 6. HISTORY SUMMARIZATION [HARD GATE]: If any history.md >= 15360 bytes (15KB), summarize now. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + 7. GIT COMMIT (orphan): Stage `.squad/` state files and commit to the `squad-state` orphan branch: + a. Identify changed `.squad/` state files via `git status --porcelain` (decisions.md, agents/*/history.md, log/*, orchestration-log/*). + b. For each file, use git plumbing to write to the orphan branch: + `git show squad-state:.squad/{path}` to check if file exists on orphan. + Use `git checkout squad-state -- .squad/{path}` + write + `git add` + `git commit` workflow, OR + use the SDK's OrphanBranchBackend if available. + c. Reset working tree state files: `git checkout HEAD -- .squad/` to avoid polluting the working branch. + d. Push orphan branch: `git push origin squad-state` + ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + {% else %} 7. GIT COMMIT: Stage only the exact `.squad/` files Scribe wrote in this session. Use `git status --porcelain` filtered to allowed paths (decisions.md, decisions-archive.md, agents/{name}/history.md, agents/{name}/history-archive.md, log/*, orchestration-log/*). Stage each file individually with `git add -- `. Handle renames by extracting destination path (`-replace '^.* -> ',''`). Commit with -F (write msg to temp file). Skip if nothing staged. ⚠️ NEVER use `git add .squad/` or broad globs. + {% endif %} 8. HEALTH REPORT: Log decisions.md before/after size, inbox count processed, history files summarized. Never speak to user. ⚠️ End with plain text summary after all tool calls. @@ -947,6 +1068,8 @@ If the user wants to remove someone: ## Source of Truth Hierarchy +> **State backend note:** Files below marked as "Derived / append-only" are **mutable state** — their storage location depends on the configured `STATE_BACKEND`. On `worktree` (default), they live on disk. On `git-notes`, they live in git notes refs. On `orphan`, they live on the `squad-state` orphan branch. On `two-layer`, commit-scoped annotations live in git notes AND permanent state lives on the orphan branch. Files marked as "Authoritative" are **static config** and always live on disk regardless of backend. + | File | Status | Who May Write | Who May Read | |------|--------|---------------|--------------| | `.github/agents/squad.agent.md` | **Authoritative governance.** All roles, handoffs, gates, and enforcement rules. | Repo maintainer (human) | Squad (Coordinator) | diff --git a/packages/squad-sdk/src/index.ts b/packages/squad-sdk/src/index.ts index 72e5b88b8..8f38b5baf 100644 --- a/packages/squad-sdk/src/index.ts +++ b/packages/squad-sdk/src/index.ts @@ -10,8 +10,8 @@ const pkg = require('../package.json'); export const VERSION: string = pkg.version; // Export public API -export { resolveSquad, resolveGlobalSquadPath, resolvePersonalSquadDir, ensurePersonalSquadDir, ensureSquadPath, ensureSquadPathTriple, loadDirConfig, isConsultMode, scratchDir, scratchFile, deriveProjectKey, resolveExternalStateDir, resolveSquadHome, ensureSquadHome, resolvePresetsDir } from './resolution.js'; -export type { SquadDirConfig, ResolvedSquadPaths } from './resolution.js'; +export { resolveSquad, resolveGlobalSquadPath, resolvePersonalSquadDir, ensurePersonalSquadDir, ensureSquadPath, ensureSquadPathTriple, loadDirConfig, isConsultMode, scratchDir, scratchFile, deriveProjectKey, resolveExternalStateDir, resolveSquadHome, ensureSquadHome, resolvePresetsDir, resolveSquadState } from './resolution.js'; +export type { ResolvedSquadPaths, SquadDirConfig, SquadStateContext } from './resolution.js'; export * from './config/index.js'; export * from './agents/onboarding.js'; export { resolvePersonalAgents, mergeSessionCast } from './agents/personal.js'; @@ -105,7 +105,7 @@ export * from './storage/index.js'; // Git-native state backends (Issue #807) export type { StateBackend, StateBackendType, StateBackendConfig } from './state-backend.js'; -export { WorktreeBackend, GitNotesBackend, OrphanBranchBackend, resolveStateBackend, validateStateKey } from './state-backend.js'; +export { WorktreeBackend, GitNotesBackend, OrphanBranchBackend, resolveStateBackend, validateStateKey, StateBackendStorageAdapter } from './state-backend.js'; // State facade (Phase 2) — namespaced to avoid conflicts with existing config/sharing exports export { diff --git a/packages/squad-sdk/src/resolution.ts b/packages/squad-sdk/src/resolution.ts index a35245f82..084f248a7 100644 --- a/packages/squad-sdk/src/resolution.ts +++ b/packages/squad-sdk/src/resolution.ts @@ -15,7 +15,10 @@ import path from 'node:path'; import os from 'node:os'; import crypto from 'node:crypto'; +import { execFileSync } from 'node:child_process'; import { FSStorageProvider } from './storage/fs-storage-provider.js'; +import { resolveStateBackend, StateBackendStorageAdapter, type StateBackend, type StateBackendType } from './state-backend.js'; +import type { StorageProvider } from './storage/storage-provider.js'; const storage = new FSStorageProvider(); @@ -37,7 +40,7 @@ export interface SquadDirConfig { extractionDisabled?: boolean; /** Where state is stored: 'external' when moved out of the working tree */ stateLocation?: string; - /** State storage backend: worktree | external | git-notes | orphan */ + /** State storage backend: local | external | git-notes | orphan */ stateBackend?: string; } @@ -679,3 +682,69 @@ export function resolvePresetsDir(): string | null { return presetsDir; } + +// ============================================================================ +// State backend resolution (Issue #1003) +// ============================================================================ + +/** + * Resolved state context for a squad session. + * + * Combines the resolved paths with the active state backend. Commands + * and SDK functions that need state I/O use this context instead of + * directly instantiating an FSStorageProvider. + * + * **Boundary:** Only mutable squad state flows through the backend. + * Bootstrap artifacts (config.json, team.md structure checks) stay on + * the local filesystem because they are needed before a backend can be + * resolved. + */ +export interface SquadStateContext { + /** Dual-root resolved paths (projectDir, teamDir, etc.) */ + paths: ResolvedSquadPaths; + /** The active state backend (local, git-notes, or orphan) */ + backend: StateBackend; + /** The repo root directory (for git-native backends) */ + repoRoot: string; + /** StorageProvider backed by the active state backend — pass to SDK modules */ + storage: StorageProvider; +} + +/** + * Resolve the full squad state context: paths + state backend. + * + * Call once at command entry and thread the context through to SDK functions. + * This ensures the configured state backend (local, git-notes, orphan) + * applies to all squad operations — not just the watch command. + * + * @param startDir - Directory to start searching from. Defaults to cwd. + * @param cliOverride - CLI flag override for state backend type. + * @returns Resolved context, or null if no squad directory is found. + */ +export function resolveSquadState(startDir?: string, cliOverride?: StateBackendType): SquadStateContext | null { + const paths = resolveSquadPaths(startDir); + if (!paths) return null; + + // Resolve actual repo root via git — handles linked worktrees correctly + const effectiveStart = startDir ?? process.cwd(); + let repoRoot: string; + try { + repoRoot = execFileSync('git', ['rev-parse', '--show-toplevel'], { + cwd: effectiveStart, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + } catch { + // Fallback: derive from .squad/ parent if git is unavailable + repoRoot = path.resolve(paths.projectDir, '..'); + } + + // Resolve the backend from config + CLI override + const backend = resolveStateBackend(paths.projectDir, repoRoot, cliOverride); + + // For local backend, use FSStorageProvider directly (more capable). + // For git-notes/orphan, bridge via StateBackendStorageAdapter. + const stateStorage: StorageProvider = backend.name === 'local' + ? new FSStorageProvider() + : new StateBackendStorageAdapter(backend, paths.projectDir); + + return { paths, backend, repoRoot, storage: stateStorage }; +} diff --git a/packages/squad-sdk/src/state-backend.ts b/packages/squad-sdk/src/state-backend.ts index 32e36bac6..8d4a00315 100644 --- a/packages/squad-sdk/src/state-backend.ts +++ b/packages/squad-sdk/src/state-backend.ts @@ -7,21 +7,24 @@ import { execFileSync } from 'node:child_process'; import path from 'node:path'; import { FSStorageProvider } from './storage/fs-storage-provider.js'; +import type { StorageProvider, StorageStats } from './storage/storage-provider.js'; const storage = new FSStorageProvider(); -export type StateBackendType = 'worktree' | 'external' | 'git-notes' | 'orphan'; +export type StateBackendType = 'local' | 'external' | 'orphan' | 'two-layer'; export interface StateBackend { read(relativePath: string): string | undefined; write(relativePath: string, content: string): void; exists(relativePath: string): boolean; list(relativeDir: string): string[]; + delete(relativePath: string): boolean; + append(relativePath: string, content: string): void; readonly name: string; } export class WorktreeBackend implements StateBackend { - readonly name = 'worktree'; + readonly name = 'local'; private readonly root: string; constructor(squadDir: string) { if (squadDir.includes('..')) throw new Error('Path traversal not allowed'); @@ -45,6 +48,17 @@ export class WorktreeBackend implements StateBackend { if (!storage.existsSync(full) || !storage.isDirectorySync(full)) return []; return storage.listSync(full); } + delete(relativePath: string): boolean { + const key = normalizeKey(relativePath); + const full = path.join(this.root, key); + if (!storage.existsSync(full)) return false; + storage.deleteSync(full); + return true; + } + append(relativePath: string, content: string): void { + const key = normalizeKey(relativePath); + storage.appendSync(path.join(this.root, key), content); + } } function gitExec(args: string[], cwd: string): string | null { @@ -104,10 +118,26 @@ export class GitNotesBackend implements StateBackend { readonly name = 'git-notes'; private readonly cwd: string; private readonly ref = 'squad'; + private cachedAnchor: string | undefined; constructor(repoRoot: string) { this.cwd = repoRoot; } + /** + * Return the repo's root commit — the first commit with no parents. + * This commit exists on every branch, so the note persists across + * branch switches (unlike HEAD, which moves with the checked-out branch). + */ + private getAnchorCommit(): string { + if (this.cachedAnchor) return this.cachedAnchor; + const root = gitExec(['rev-list', '--max-parents=0', 'HEAD'], this.cwd); + if (!root) throw new Error('git-notes backend: no root commit found'); + // If multiple roots (e.g. from unrelated-history merges), use the first. + this.cachedAnchor = root.split('\n')[0]!.trim(); + return this.cachedAnchor; + } + private loadBlob(): Record { - const raw = gitExec(['notes', `--ref=${this.ref}`, 'show', 'HEAD'], this.cwd); + const anchor = this.getAnchorCommit(); + const raw = gitExec(['notes', `--ref=${this.ref}`, 'show', anchor], this.cwd); if (!raw) return {}; try { const parsed: unknown = JSON.parse(raw); @@ -119,10 +149,11 @@ export class GitNotesBackend implements StateBackend { } private saveBlob(blob: Record): void { + const anchor = this.getAnchorCommit(); const json = JSON.stringify(blob, null, 2); try { - gitExecWithInput(['notes', `--ref=${this.ref}`, 'add', '-f', '--file', '-', 'HEAD'], json, this.cwd); - } catch { throw new Error('git-notes backend: failed to write note on HEAD'); } + gitExecWithInput(['notes', `--ref=${this.ref}`, 'add', '-f', '--file', '-', anchor], json, this.cwd); + } catch { throw new Error('git-notes backend: failed to write note on ' + anchor); } } read(relativePath: string): string | undefined { @@ -151,6 +182,20 @@ export class GitNotesBackend implements StateBackend { } return [...entries].sort(); } + delete(relativePath: string): boolean { + const blob = this.loadBlob(); + const key = normalizeKey(relativePath); + if (!Object.hasOwn(blob, key)) return false; + delete blob[key]; + this.saveBlob(blob); + return true; + } + append(relativePath: string, content: string): void { + const blob = this.loadBlob(); + const key = normalizeKey(relativePath); + blob[key] = (blob[key] ?? '') + content; + this.saveBlob(blob); + } } export class OrphanBranchBackend implements StateBackend { @@ -177,8 +222,14 @@ export class OrphanBranchBackend implements StateBackend { } read(relativePath: string): string | undefined { - const result = gitExec(['show', `${this.branch}:${normalizeKey(relativePath)}`], this.cwd); - return result ?? undefined; + const key = normalizeKey(relativePath); + try { + return execFileSync('git', ['show', `${this.branch}:${key}`], { + cwd: this.cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }); + } catch { + return undefined; + } } write(relativePath: string, content: string): void { @@ -221,6 +272,65 @@ export class OrphanBranchBackend implements StateBackend { return result.split('\n').filter(Boolean); } + delete(relativePath: string): boolean { + const key = normalizeKey(relativePath); + if (!this.exists(relativePath)) return false; + this.ensureBranch(); + const treeResult = gitExec(['log', '--format=%T', '-1', this.branch], this.cwd); + if (!treeResult) return false; + const newTree = this.removeFromTree(treeResult, key.split('/')); + const parentCommit = gitExec(['rev-parse', this.branch], this.cwd); + let newCommit: string; + try { + const parentArgs = parentCommit ? ['-p', parentCommit] : []; + newCommit = execFileSync('git', ['commit-tree', newTree, ...parentArgs, '-m', `Delete ${key}`], { + cwd: this.cwd, encoding: 'utf-8', stdio: ['pipe', 'pipe', 'pipe'], + }).trim(); + } catch { throw new Error(`orphan backend: failed to commit delete for ${key}`); } + gitExecOrThrow(['update-ref', `refs/heads/${this.branch}`, newCommit], this.cwd); + return true; + } + + append(relativePath: string, content: string): void { + const existing = this.read(relativePath) ?? ''; + this.write(relativePath, existing + content); + } + + private removeFromTree(baseTree: string, pathSegments: string[]): string { + if (pathSegments.length === 0) throw new Error('orphan backend: empty path segments'); + if (pathSegments.length === 1) { + // Remove the entry from the tree + const listing = gitExec(['ls-tree', baseTree], this.cwd) ?? ''; + const lines = listing.split('\n').filter(Boolean); + const filtered = lines.filter((line) => { + const match = line.match(/^(\d+)\s+(blob|tree)\s+([a-f0-9]+)\t(.+)$/); + return !(match && match[4] === pathSegments[0]); + }); + try { + return gitExecWithInput(['mktree'], filtered.length > 0 ? filtered.join('\n') + '\n' : '', this.cwd); + } catch { throw new Error(`orphan backend: failed to remove entry ${pathSegments[0]}`); } + } + const [dir, ...rest] = pathSegments; + const subTreeHash = this.getSubtreeHash(baseTree, dir!); + if (!subTreeHash) return baseTree; // subtree doesn't exist, nothing to remove + const childTree = this.removeFromTree(subTreeHash, rest); + // If the child tree is now empty, remove the directory entry entirely + const childListing = gitExec(['ls-tree', childTree], this.cwd); + if (!childListing || childListing.length === 0) { + // Remove the empty directory from the parent tree + const listing = gitExec(['ls-tree', baseTree], this.cwd) ?? ''; + const lines = listing.split('\n').filter(Boolean); + const filtered = lines.filter((line) => { + const match = line.match(/^(\d+)\s+(blob|tree)\s+([a-f0-9]+)\t(.+)$/); + return !(match && match[4] === dir); + }); + try { + return gitExecWithInput(['mktree'], filtered.length > 0 ? filtered.join('\n') + '\n' : '', this.cwd); + } catch { throw new Error(`orphan backend: failed to prune empty directory ${dir}`); } + } + return this.replaceEntry(baseTree, dir!, '040000', 'tree', childTree); + } + private updateTree(baseTree: string, pathSegments: string[], blobHash: string): string { if (pathSegments.length === 0) throw new Error('orphan backend: empty path segments'); if (pathSegments.length === 1) { @@ -262,6 +372,152 @@ export class OrphanBranchBackend implements StateBackend { } } +/** + * Adapter that wraps a StateBackend as a StorageProvider. + * + * Modules that accept `storage: StorageProvider` can use this adapter + * so that git-notes and orphan backends flow through the same code paths + * as the local filesystem backend. + */ +export class StateBackendStorageAdapter implements StorageProvider { + constructor(private backend: StateBackend, private squadDir: string) {} + + // ── Async operations ───────────────────────────────────────────────────── + async read(filePath: string): Promise { + return this.backend.read(this.toRelative(filePath)); + } + async write(filePath: string, data: string): Promise { + this.backend.write(this.toRelative(filePath), data); + } + async append(filePath: string, data: string): Promise { + this.backend.append(this.toRelative(filePath), data); + } + async exists(filePath: string): Promise { + return this.backend.exists(this.toRelative(filePath)); + } + async list(dirPath: string): Promise { + return this.backend.list(this.toRelative(dirPath)); + } + async delete(filePath: string): Promise { + this.backend.delete(this.toRelative(filePath)); + } + async deleteDir(dirPath: string): Promise { + const rel = this.toRelative(dirPath); + const entries = this.backend.list(rel); + for (const entry of entries) { this.backend.delete(rel ? rel + '/' + entry : entry); } + } + async isDirectory(targetPath: string): Promise { + return this.backend.list(this.toRelative(targetPath)).length > 0; + } + async mkdir(_dirPath: string, _options?: { recursive?: boolean }): Promise { /* no-op for git backends */ } + async rename(oldPath: string, newPath: string): Promise { + const content = this.backend.read(this.toRelative(oldPath)); + if (content !== undefined) { this.backend.write(this.toRelative(newPath), content); this.backend.delete(this.toRelative(oldPath)); } + } + async copy(srcPath: string, destPath: string): Promise { + const content = this.backend.read(this.toRelative(srcPath)); + if (content !== undefined) { this.backend.write(this.toRelative(destPath), content); } + } + async stat(targetPath: string): Promise { + const content = this.backend.read(this.toRelative(targetPath)); + if (content === undefined) return undefined; + return { size: content.length, mtimeMs: Date.now(), isDirectory: false }; + } + + // ── Sync variants ──────────────────────────────────────────────────────── + readSync(filePath: string): string | undefined { return this.backend.read(this.toRelative(filePath)); } + writeSync(filePath: string, data: string): void { this.backend.write(this.toRelative(filePath), data); } + appendSync(filePath: string, data: string): void { this.backend.append(this.toRelative(filePath), data); } + existsSync(filePath: string): boolean { return this.backend.exists(this.toRelative(filePath)); } + listSync(dirPath: string): string[] { return this.backend.list(this.toRelative(dirPath)); } + deleteSync(filePath: string): void { this.backend.delete(this.toRelative(filePath)); } + deleteDirSync(dirPath: string): void { + const rel = this.toRelative(dirPath); + const entries = this.backend.list(rel); + for (const entry of entries) { this.backend.delete(rel ? rel + '/' + entry : entry); } + } + isDirectorySync(targetPath: string): boolean { + return this.backend.list(this.toRelative(targetPath)).length > 0; + } + mkdirSync(_dirPath: string, _options?: { recursive?: boolean }): void { /* no-op */ } + renameSync(oldPath: string, newPath: string): void { + const content = this.backend.read(this.toRelative(oldPath)); + if (content !== undefined) { this.backend.write(this.toRelative(newPath), content); this.backend.delete(this.toRelative(oldPath)); } + } + copySync(srcPath: string, destPath: string): void { + const content = this.backend.read(this.toRelative(srcPath)); + if (content !== undefined) { this.backend.write(this.toRelative(destPath), content); } + } + statSync(targetPath: string): StorageStats | undefined { + const content = this.backend.read(this.toRelative(targetPath)); + if (content === undefined) return undefined; + return { size: content.length, mtimeMs: Date.now(), isDirectory: false }; + } + + /** Convert absolute path to relative path for the backend. */ + private toRelative(filePath: string): string { + const normalized = filePath.replace(/\\/g, '/'); + const squadNorm = this.squadDir.replace(/\\/g, '/'); + if (normalized.startsWith(squadNorm + '/')) { + return normalized.slice(squadNorm.length + 1); + } + if (normalized.startsWith(squadNorm)) { + return normalized.slice(squadNorm.length).replace(/^\//, '') || '.'; + } + // Already relative + return normalized; + } +} + +/** + * Two-Layer Backend — combines git-notes (commit-scoped annotations) with orphan + * branch (permanent state). Reads from orphan for bulk state, writes to both: + * - Git notes for commit-scoped "why" annotations (per-agent namespace) + * - Orphan branch for permanent state (decisions, histories, logs) + * + * Ralph promotes notes with promote_to_permanent after PR merge. + */ +export class TwoLayerBackend implements StateBackend { + readonly name = 'two-layer'; + private readonly notes: GitNotesBackend; + private readonly orphan: OrphanBranchBackend; + + constructor(repoRoot: string) { + this.notes = new GitNotesBackend(repoRoot); + this.orphan = new OrphanBranchBackend(repoRoot); + } + + /** Read from orphan (the permanent store) */ + read(key: string): string | undefined { + return this.orphan.read(key); + } + + /** Write to orphan (permanent state) AND git notes (commit-scoped annotation) */ + write(key: string, value: string): void { + this.orphan.write(key, value); + try { this.notes.write(key, value); } catch { /* notes are best-effort */ } + } + + list(dir: string): string[] { + return this.orphan.list(dir); + } + + exists(key: string): boolean { + return this.orphan.exists(key); + } + + delete(key: string): boolean { + const result = this.orphan.delete(key); + try { this.notes.delete(key); } catch { /* best-effort */ } + return result; + } + + append(key: string, value: string): void { + this.orphan.append(key, value); + try { this.notes.append(key, value); } catch { /* best-effort */ } + } +} + export interface StateBackendConfig { stateBackend?: StateBackendType; } export function resolveStateBackend(squadDir: string, repoRoot: string, cliOverride?: StateBackendType): StateBackend { @@ -272,29 +528,37 @@ export function resolveStateBackend(squadDir: string, repoRoot: string, cliOverr if (raw) { const parsed = JSON.parse(raw) as Record; if (typeof parsed['stateBackend'] === 'string' && isValidBackendType(parsed['stateBackend'])) { - configBackend = parsed['stateBackend'] as StateBackendType; + configBackend = normalizeBackendType(parsed['stateBackend']); } } } catch { /* fall through */ } - const chosen = cliOverride ?? configBackend ?? 'worktree'; + const chosen = normalizeBackendType(cliOverride ?? configBackend ?? 'local'); try { return createBackend(chosen, squadDir, repoRoot); } catch (err) { const msg = err instanceof Error ? err.message : String(err); - console.warn(`Warning: State backend '${chosen}' failed: ${msg}. Falling back to 'worktree'.`); + console.warn(`Warning: State backend '${chosen}' failed: ${msg}. Falling back to 'local'.`); return new WorktreeBackend(squadDir); } } function isValidBackendType(value: string): value is StateBackendType { - return ['worktree', 'external', 'git-notes', 'orphan'].includes(value); + return ['local', 'worktree', 'external', 'git-notes', 'orphan', 'two-layer'].includes(value); +} +// Note: 'worktree' and 'git-notes' are accepted for backward compatibility but normalized away + +/** Normalize legacy aliases to canonical backend type names. */ +function normalizeBackendType(type: string): StateBackendType { + if (type === 'worktree') return 'local'; + if (type === 'git-notes') return 'two-layer'; // standalone git-notes removed; migrate to two-layer + return type as StateBackendType; } function createBackend(type: StateBackendType, squadDir: string, repoRoot: string): StateBackend { switch (type) { - case 'worktree': return new WorktreeBackend(squadDir); - case 'git-notes': return new GitNotesBackend(repoRoot); + case 'local': return new WorktreeBackend(squadDir); case 'orphan': return new OrphanBranchBackend(repoRoot); + case 'two-layer': return new TwoLayerBackend(repoRoot); case 'external': return new WorktreeBackend(squadDir); // Stub — PR #797 default: throw new Error(`Unknown state backend type: ${type}`); } diff --git a/packages/squad-sdk/templates/notes-protocol.md b/packages/squad-sdk/templates/notes-protocol.md new file mode 100644 index 000000000..5a1d03a5f --- /dev/null +++ b/packages/squad-sdk/templates/notes-protocol.md @@ -0,0 +1,202 @@ +# Squad Notes Protocol + +> Contract for agent state via git notes. Agents write commit-scoped context +> here instead of modifying `.squad/` files in PRs. +> +> **Version:** 1.0 +> **Backends:** `git-notes`, `orphan` + +--- + +## Overview + +Squad state has two layers: + +1. **Git notes layer** (this document) — thin, commit-scoped annotations that + attach agent context to commits without appearing in PRs or diffs. +2. **Permanent state layer** — long-lived decisions, routing rules, and archives + stored via the configured state backend (`git-notes` or `orphan` branch). + +Agents write notes during their work rounds. Ralph promotes flagged notes to +permanent state after a PR merges. + +--- + +## Namespaces + +Each agent writes to its own namespace to prevent conflicts: + +| Namespace | Owner | Purpose | +|-----------|-------|---------| +| `refs/notes/squad/data` | Data | Architecture decisions, implementation choices | +| `refs/notes/squad/worf` | Worf | Security reviews, vulnerability assessments | +| `refs/notes/squad/seven` | Seven | Documentation quality, API contract decisions | +| `refs/notes/squad/ralph` | Ralph | Work-round progress, task-state annotations | +| `refs/notes/squad/q` | Q | Devil's advocate findings, risk assessments | +| `refs/notes/squad/research` | Any agent | Research notes that should survive branch deletion | +| `refs/notes/squad/review` | Any agent | Code review context (mirrors Gerrit's pattern) | + +**Rule**: Only write to your own namespace. The shared namespaces +(`research`, `review`) use `append` — never `add`. + +--- + +## Note JSON Schema + +All notes MUST be valid JSON. Minimum required fields: + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision | research | review | progress | security", + "content": "..." +} +``` + +### Decision notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision", + "decision": "Use JWT RS256 for auth middleware", + "reasoning": "Existing pattern in codebase — auth.go:47-89.", + "alternatives_considered": ["HS256", "session tokens"], + "confidence": "high", + "promote_to_permanent": true +} +``` + +Set `"promote_to_permanent": true` to signal Ralph to copy this to +`decisions.md` after the PR merges. + +### Research notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "research", + "topic": "JWT vs session tokens", + "findings": {}, + "effort_hours": 2.5, + "archive_on_close": true +} +``` + +Set `"archive_on_close": true` to signal Ralph to archive this to +`state/research/` even if the PR is rejected. + +--- + +## Write Commands + +```bash +# Write a decision note on the current commit +git notes --ref=squad/{your-agent} add \ + -m '{"agent":"{Agent}","timestamp":"...","type":"decision","decision":"..."}' \ + HEAD + +# Append to an existing note (multiple items on same commit) +git notes --ref=squad/{your-agent} append \ + -m '{"agent":"{Agent}","timestamp":"...","type":"progress","content":"..."}' \ + HEAD + +# Read your note +git notes --ref=squad/{your-agent} show HEAD + +# List all commits with notes in your namespace +git notes --ref=squad/{your-agent} list +``` + +Or use the helper script: + +```powershell +./scripts/notes/write-note.ps1 -Agent data -Type decision \ + -Content '{"decision":"Use JWT","reasoning":"..."}' \ + [-Commit HEAD] [-Promote] [-Archive] +``` + +--- + +## Fetch / Push + +**Notes are NOT fetched or pushed by default.** Every clone needs setup. + +### One-time setup + +```bash +git config --add remote.origin.fetch 'refs/notes/*:refs/notes/*' +git fetch origin 'refs/notes/*:refs/notes/*' +``` + +Or use the helper: + +```powershell +./scripts/notes/fetch.ps1 -Setup +``` + +### Every work round + +1. **Start**: `git fetch origin 'refs/notes/*:refs/notes/*'` +2. **End**: `git push origin 'refs/notes/*:refs/notes/*'` + +--- + +## Conflict Handling + +1. **Per-agent namespaces prevent 99% of conflicts.** Only one agent writes to + `refs/notes/squad/data`, so there are no write conflicts in normal use. + +2. **Same agent, two machines:** First push wins. Losing machine should fetch + and append: + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes --ref=squad/{agent} append -m '{...}' HEAD + git push origin 'refs/notes/*:refs/notes/*' + ``` + +3. **Shared namespaces** (`research`, `review`): Always use `git notes append`, + never `git notes add`. + +4. **Push conflict recovery:** + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes merge refs/notes/remotes/origin/squad/{namespace} + git push origin 'refs/notes/*:refs/notes/*' + ``` + +--- + +## When to Use Notes vs State Backend + +| Use git notes | Use state backend | +|---------------|-------------------| +| Why THIS choice on THIS commit | Universal routing rules, conventions | +| Decisions scoped to a feature | Long-lived decisions for all future work | +| Research for a specific investigation | Research archives (promoted from notes) | +| Security sign-offs per commit | Agent history persisting across features | +| Agent-to-agent context for current feature | Team agreements and policies | + +When in doubt: **notes first, promote to permanent state later.** Ralph handles +the promotion automatically when `promote_to_permanent` is set. + +--- + +## Ralph Promotion Rules + +**After PR merge:** + +1. Fetch all notes from remote +2. Traverse commits reachable from the default branch that have notes +3. For each note with `"promote_to_permanent": true` → append to `decisions.md` +4. Push state + +**After PR close/rejection:** + +1. List notes in `squad/research` on the closed branch's commits +2. For each note with `"archive_on_close": true` → archive to `research/` +3. Push state +4. Notes on rejected commits are NOT promoted — this is the desired behavior diff --git a/packages/squad-sdk/templates/scribe-charter.md b/packages/squad-sdk/templates/scribe-charter.md index 308b780e2..a2031d072 100644 --- a/packages/squad-sdk/templates/scribe-charter.md +++ b/packages/squad-sdk/templates/scribe-charter.md @@ -24,6 +24,59 @@ **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory). +**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. If it's `"orphan"` or `"git-notes"`, run the **State Leak Guard** before any other work. + +### State Leak Guard (orphan/git-notes backends only) + +Before logging or merging, check if any agent accidentally committed state files to the working branch: + +```powershell +# Check if state files are staged or committed but shouldn't be +$stateFiles = @( + '.squad/decisions.md', + '.squad/decisions-archive.md' +) +$statePatterns = @( + '.squad/agents/*/history.md', + '.squad/agents/*/history-archive.md', + '.squad/log/*', + '.squad/orchestration-log/*', + '.squad/decisions/inbox/*' +) + +# 1. Check git status for accidentally staged state files +$dirty = git status --porcelain | Where-Object { $_.Length -gt 3 } | ForEach-Object { + $_.Substring(3) -replace '^.* -> ','' +} | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($dirty) { + # Unstage any accidentally added state files + $dirty | ForEach-Object { git reset HEAD -- $_ 2>$null } + # Restore from HEAD (discard working tree changes for state files) + $dirty | ForEach-Object { git checkout HEAD -- $_ 2>$null } +} + +# 2. Check if the most recent commit on this branch has state files +$lastCommitFiles = git diff-tree --no-commit-id --name-only -r HEAD 2>$null +$leakedInCommit = $lastCommitFiles | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($leakedInCommit) { + # State files leaked into the last commit — amend to remove them + $leakedInCommit | ForEach-Object { git rm --cached -- $_ 2>$null } + git commit --amend --no-edit 2>$null +} +``` + +If any files were cleaned, log: `⚠️ State leak guard: removed {N} state file(s) from working branch.` + +After the guard, proceed with normal Scribe work (but persist state via the configured backend, not the working branch). + After every substantial work session: 1. **Log the session** to `.squad/log/{timestamp}-{topic}.md`: @@ -57,9 +110,50 @@ After every substantial work session: ``` 5. **Commit `.squad/` changes:** + **Check `STATE_BACKEND` from spawn prompt.** This determines WHERE state gets committed. + **IMPORTANT — Windows compatibility:** Do NOT use `git -C {path}` (unreliable with Windows paths). Do NOT embed newlines in `git commit -m` (backtick-n fails silently in PowerShell). - Instead: + + **If STATE_BACKEND is "orphan":** + State files must be committed to the `squad-state` orphan branch, NOT the working branch. + - Identify changed `.squad/` state files via `git status --porcelain` filtered to allowed paths. + - For each file, use git plumbing to write to the orphan branch: + ```powershell + # Create a temporary worktree for the orphan branch + $orphanWt = Join-Path ([System.IO.Path]::GetTempPath()) "squad-state-$(Get-Random)" + git worktree add $orphanWt squad-state 2>$null + if ($LASTEXITCODE -ne 0) { git worktree add --orphan $orphanWt squad-state } + # Copy state files to orphan worktree + $filesToSync | ForEach-Object { + $dest = Join-Path $orphanWt $_ + New-Item -ItemType Directory -Path (Split-Path $dest) -Force | Out-Null + Copy-Item $_ $dest -Force + } + # Commit in orphan worktree + Push-Location $orphanWt + git add .squad/ + git diff --cached --quiet + if ($LASTEXITCODE -ne 0) { + $msgFile = [System.IO.Path]::GetTempFileName() + Set-Content -Path $msgFile -Value "docs(ai-team): $summary" -Encoding utf8 + git commit -F $msgFile + Remove-Item $msgFile + git push origin squad-state + } + Pop-Location + git worktree remove $orphanWt --force + ``` + - After committing to orphan, reset working tree state files: `git checkout HEAD -- .squad/` + - ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + + **If STATE_BACKEND is "git-notes":** + State is already persisted in git notes refs by agents. Scribe only needs to: + - Push any locally created note refs: `git push origin 'refs/notes/squad/*'` + - Commit decisions.md (the merged canonical file) to the working branch as normal. + + **If STATE_BACKEND is "worktree" (default):** + Commit to the working branch as normal: - `cd` into the team root first. - Stage only files Scribe actually modified in this session. Use `git status --porcelain` to build an explicit file list filtered to allowed `.squad/` paths: diff --git a/packages/squad-sdk/templates/scripts/notes/fetch.ps1 b/packages/squad-sdk/templates/scripts/notes/fetch.ps1 new file mode 100644 index 000000000..5adc19480 --- /dev/null +++ b/packages/squad-sdk/templates/scripts/notes/fetch.ps1 @@ -0,0 +1,88 @@ +#!/usr/bin/env pwsh +# scripts/notes/fetch.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Fetch git notes from remote. Run on every Ralph-watch startup and before +# any agent reads or writes notes. +# +# Usage: +# ./scripts/notes/fetch.ps1 # fetch only +# ./scripts/notes/fetch.ps1 -Setup # first-time: add refspec + fetch +# ./scripts/notes/fetch.ps1 -Merge # fetch + merge (use after push conflict) +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [string]$Remote = "origin", + [string]$RepoPath = ".", + [switch]$Setup, + [switch]$Merge, + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/fetch] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath + +# ── One-time setup: add fetch refspec ────────────────────────────────────── +if ($Setup) { + $existing = git -C $repo config --get-all "remote.$Remote.fetch" 2>&1 | + Where-Object { $_ -match "refs/notes" } + if ($existing) { + Log "Notes refspec already configured." DarkGray + } else { + git -C $repo config --add "remote.$Remote.fetch" "refs/notes/*:refs/notes/*" + Log "Added notes refspec to remote.$Remote.fetch" Green + } +} + +# ── Fetch notes ───────────────────────────────────────────────────────────── +Log "Fetching notes from $Remote..." +$output = git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 +if ($LASTEXITCODE -ne 0) { + Log "Fetch warning: $output" DarkYellow +} else { + Log "Notes fetched." Green +} + +# ── Merge notes if requested (after push conflict) ────────────────────────── +if ($Merge) { + # Abort any stale merge-in-progress state + $mergeLock = Join-Path $repo ".git/NOTES_MERGE_PARTIAL" + if (Test-Path $mergeLock) { + Log "Stale notes merge in progress — aborting before retry" DarkYellow + git -C $repo notes merge --abort 2>&1 | Out-Null + } + + $namespaces = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + foreach ($ref in $namespaces) { + $ns = $ref -replace "refs/notes/", "" + $remoteRef = "refs/notes/remotes/$Remote/$ns" + $remoteExists = git -C $repo for-each-ref $remoteRef --format="%(refname)" 2>&1 + if ($remoteExists) { + Log "Merging notes: $ns (cat_sort_uniq)" + git -C $repo notes --ref=$ns merge -s cat_sort_uniq $remoteRef 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Log " Merge failed on $ns — aborting and continuing" Red + git -C $repo notes merge --abort 2>&1 | Out-Null + } + } + } + Log "Notes merge complete." Green +} + +# ── Show available namespaces ──────────────────────────────────────────────── +if (-not $Quiet) { + $refs = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + if ($refs) { + Log "Available namespaces:" + foreach ($r in $refs) { + $count = (git -C $repo notes --ref=($r -replace "refs/notes/","") list 2>&1 | + Where-Object { $_ -ne "" } | Measure-Object -Line).Lines + Log " $r ($count notes)" DarkGray + } + } else { + Log "No squad notes yet." DarkGray + } +} diff --git a/packages/squad-sdk/templates/scripts/notes/write-note.ps1 b/packages/squad-sdk/templates/scripts/notes/write-note.ps1 new file mode 100644 index 000000000..dbc719d35 --- /dev/null +++ b/packages/squad-sdk/templates/scripts/notes/write-note.ps1 @@ -0,0 +1,126 @@ +#!/usr/bin/env pwsh +# scripts/notes/write-note.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Helper for agents to write notes without wrestling with JSON escaping. +# Validates namespace ownership, handles conflicts, pushes automatically. +# +# Usage: +# ./scripts/notes/write-note.ps1 -Agent data -Type decision \ +# -Content '{"decision":"Use JWT","reasoning":"..."}' \ +# [-Commit HEAD] [-Promote] [-Archive] +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [Parameter(Mandatory)][string]$Agent, + + [Parameter(Mandatory)] + [ValidateSet("decision","research","review","security-review","progress", + "api-contract","risk-assessment","routing-discovery","counter-argument")] + [string]$Type, + + [Parameter(Mandatory)] + [string]$Content, # JSON object with type-specific fields + + [string]$Commit = "HEAD", + [string]$RepoPath = ".", + [string]$Remote = "origin", + [switch]$Promote, # set promote_to_permanent: true + [switch]$Archive, # set archive_on_close: true + [switch]$NoPush, # skip auto-push + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/write] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath +$namespace = "squad/$($Agent.ToLower())" + +# ── Validate JSON content ──────────────────────────────────────────────────── +try { + $parsed = $Content | ConvertFrom-Json -ErrorAction Stop +} catch { + Write-Error "Content must be valid JSON. Got: $Content" + exit 1 +} + +# ── Build full note object ──────────────────────────────────────────────────── +$note = [ordered]@{ + agent = (Get-Culture).TextInfo.ToTitleCase($Agent.ToLower()) + timestamp = [System.DateTime]::UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") + type = $Type +} + +# Merge content fields into note +$parsed.PSObject.Properties | ForEach-Object { $note[$_.Name] = $_.Value } + +# Add flag fields +if ($Promote) { $note["promote_to_permanent"] = $true } +if ($Archive) { $note["archive_on_close"] = $true } + +$noteJson = $note | ConvertTo-Json -Compress -Depth 10 + +# ── Fetch first to avoid conflicts ─────────────────────────────────────────── +Log "Fetching notes before write..." +git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 | Out-Null + +# ── Check if note already exists on this commit ───────────────────────────── +$existing = git -C $repo notes --ref=$namespace show $Commit 2>&1 +$useAppend = ($LASTEXITCODE -eq 0) + +if ($useAppend) { + Log "Note exists on $Commit — appending" DarkYellow + git -C $repo notes --ref=$namespace append -m $noteJson $Commit +} else { + git -C $repo notes --ref=$namespace add -m $noteJson $Commit +} + +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to write note to refs/notes/$namespace on $Commit" + exit 1 +} + +Log "Note written to refs/notes/$namespace on $($Commit.Substring(0,[Math]::Min(8,$Commit.Length)))" Green + +# ── Push with retry ────────────────────────────────────────────────────────── +if (-not $NoPush) { + $maxRetries = 5 + $nsRef = "refs/notes/$namespace" + + for ($i = 0; $i -lt $maxRetries; $i++) { + Log "Pushing notes (attempt $($i+1))..." + $pushOut = git -C $repo push $Remote "${nsRef}:${nsRef}" 2>&1 + if ($LASTEXITCODE -eq 0) { + Log "Notes pushed successfully." Green + break + } + + if ($pushOut -match "non-fast-forward|fetch first|rejected") { + Log "Push conflict — fetch-first retry..." DarkYellow + + # Force-fetch: overwrite local ref with current remote state + git -C $repo fetch $Remote "${nsRef}:${nsRef}" 2>&1 | Out-Null + + # Re-append our note on top of the now-current remote state + git -C $repo notes --ref=$namespace append -m $noteJson $Commit 2>&1 | Out-Null + + $jitter = Get-Random -Minimum 0 -Maximum 1000 + $sleep = [Math]::Pow(2, $i) + $jitter / 1000 + Start-Sleep -Seconds $sleep + + } else { + Log "Push error: $pushOut" Red + if ($i -eq $maxRetries - 1) { + Write-Warning "Failed after $maxRetries retries. Push manually: git push origin '${nsRef}:${nsRef}'" + } + } + } +} + +# ── Show result ─────────────────────────────────────────────────────────────── +if (-not $Quiet) { + Log "Note content:" + $note | ConvertTo-Json -Depth 5 | Write-Host -ForegroundColor DarkGray +} diff --git a/packages/squad-sdk/templates/squad.agent.md.template b/packages/squad-sdk/templates/squad.agent.md.template index 01e18dfad..4ef1a3045 100644 --- a/packages/squad-sdk/templates/squad.agent.md.template +++ b/packages/squad-sdk/templates/squad.agent.md.template @@ -107,6 +107,8 @@ The `union` merge driver keeps all lines from both sides, which is correct for a **On every session start:** Run `git config user.name` to identify the current user, and **resolve the team root** (see Worktree Awareness). Store the team root — all `.squad/` paths must be resolved relative to it. Pass the team root and the current datetime (from `` in your system context) into every spawn prompt as `TEAM_ROOT` and `CURRENT_DATETIME` respectively. Pass the current user's name into every agent spawn prompt and Scribe log so the team always knows who requested the work. Check `.squad/identity/now.md` if it exists — it tells you what the team was last focused on. Update it if the focus has shifted. +**Resolve state backend:** Read `.squad/config.json` and check the `stateBackend` field. Valid values: `"worktree"` (default), `"git-notes"`, `"orphan"`, `"two-layer"`. Store as `STATE_BACKEND` and pass it into every spawn prompt. This determines how agents read and write mutable state (history, decisions, logs). Static config (charters, team.md, routing.md) always lives on disk regardless of backend. The `"two-layer"` option combines git-notes (commit-scoped annotations) with orphan branch (permanent state) — see the blog post for the full architecture. + **⚡ Context caching:** After the first message in a session, `team.md`, `routing.md`, and `registry.json` are already in your context. Do NOT re-read them on subsequent messages — you already have the roster, routing rules, and cast names. Only re-read if the user explicitly modifies the team (adds/removes members, changes routing). **Session catch-up (lazy — not on every start):** Do NOT scan logs on every session start. Only provide a catch-up summary when: @@ -223,13 +225,16 @@ The `name` parameter generates the human-readable agent ID shown in the tasks pa **When you detect a directive:** -1. Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: - ``` - ### {timestamp}: User directive - **By:** {user name} (via Copilot) - **What:** {the directive, verbatim or lightly paraphrased} - **Why:** User request — captured for team memory - ``` +1. Capture the directive: + - **worktree/orphan backend:** Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: + ``` + ### {timestamp}: User directive + **By:** {user name} (via Copilot) + **What:** {the directive, verbatim or lightly paraphrased} + **Why:** User request — captured for team memory + ``` + - **git-notes backend:** Persist via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/directives" -Content '{"timestamp": "{timestamp}", "by": "{user name}", "what": "...", "why": "User request"}'` 2. Acknowledge briefly: `"📌 Captured. {one-line summary of the directive}."` 3. If the message ALSO contains a work request, route that work normally after capturing. If it's directive-only, you're done — no agent spawn needed. @@ -344,7 +349,12 @@ prompt: | TARGET FILE(S): {exact file path(s)} Do the work. Keep it focused. + {% if STATE_BACKEND == "git-notes" %} + If you made a meaningful decision, persist it via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"decision": {"title": "...", "what": "...", "why": "..."}}'` + {% else %} If you made a meaningful decision, write to .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} ⚠️ OUTPUT: Report outcomes in human terms. Never expose tool internals or SQL. ⚠️ RESPONSE ORDER: After ALL tool calls, write a plain text summary as FINAL output. @@ -598,8 +608,9 @@ When the user gives any task, the Coordinator MUST: To enable full parallelism, shared writes use a drop-box pattern that eliminates file conflicts: **decisions.md** — Agents do NOT write directly to `decisions.md`. Instead: -- Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` -- Scribe merges inbox entries into the canonical `.squad/decisions.md` and clears the inbox +- **worktree/orphan backend:** Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` +- **git-notes backend:** Agents persist decisions via their git notes ref (see State Protocol). Scribe reads all agent notes and merges decisions. +- Scribe merges into the canonical `.squad/decisions.md` and clears the inbox (or note entries) - All agents READ from `.squad/decisions.md` at spawn time (last-merged snapshot) **orchestration-log/** — Scribe writes one entry per agent after each batch: @@ -799,7 +810,71 @@ prompt: | - Commit and push from the worktree {% endif %} + STATE_BACKEND: {state_backend} + + {% if STATE_BACKEND == "git-notes" %} + ## State Protocol — Git Notes + This project uses git-notes for mutable state. **DO NOT write to `.squad/` files for state.** + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading your state:** + Run: `powershell .squad/scripts/notes/fetch.ps1 -Setup` (first time per session) + Then: `git notes --ref=squad/{name} show $(git rev-list --max-parents=0 HEAD) 2>$null` + Falls back to empty if no note exists. + + **Writing state (history, decisions, learnings):** + Run: `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{json}'` + The helper handles JSON validation, conflict retry, and push. + + **Decisions:** Write decisions as JSON via your note ref. Scribe will merge them. + **Skills:** Skills are static config — write to `.squad/skills/` on disk as normal. + {% endif %} + + {% if STATE_BACKEND == "orphan" %} + ## State Protocol — Orphan Branch + This project uses an orphan branch (`squad-state`) for mutable state. + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading state:** Read `.squad/` files on disk — they are synced from the orphan branch. + **Writing state:** Write to `.squad/` files on disk as normal during your session. + Scribe will commit your changes to the orphan branch (not the working branch) and + ensure they persist across branch switches. + + **Important:** Do NOT commit `.squad/` state files to the working branch yourself. + Scribe handles the orphan branch commit workflow. + {% endif %} + + {% if STATE_BACKEND == "two-layer" %} + ## State Protocol — Two-Layer (Git Notes + Orphan Branch) + This project uses the two-layer architecture from Tamir's blog: + - **Layer 1 (git notes):** Commit-scoped "why" annotations — invisible in PRs + - **Layer 2 (orphan branch):** Permanent state store — decisions, histories, logs + + Static config (charters, team.md, routing.md) is on disk as normal. + + **During your session:** + 1. Write commit-scoped annotations as git notes on HEAD: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + 2. Write bulk state (history, logs) to `.squad/` files on disk — Scribe moves them to the orphan branch. + + **Note flags:** + - `"promote_to_permanent": true` — Ralph promotes this to decisions.md after PR merge + - `"archive_on_close": true` — Worth keeping even if PR is rejected (valuable research) + - Neither flag — silently ignored if PR is rejected (correct for branch-specific decisions) + + **Important:** Do NOT commit `.squad/` state files to the working branch. + Scribe handles orphan commits. Ralph handles note promotion. + {% endif %} + + {% if STATE_BACKEND == "worktree" or STATE_BACKEND is not defined %} Read .squad/agents/{name}/history.md (your project knowledge). + {% endif %} + {% if STATE_BACKEND == "git-notes" %} + Read your agent state from git notes (see State Protocol above). + {% endif %} + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + Read .squad/agents/{name}/history.md (your project knowledge — synced from orphan branch). + {% endif %} Read .squad/decisions.md (team decisions to respect). If .squad/identity/wisdom.md exists, read it before starting work. If .squad/identity/now.md exists, read it at spawn time. @@ -823,10 +898,27 @@ prompt: | ⚠️ DATES: When writing dates in any file (decisions, history, logs), use ONLY the CURRENT_DATETIME value above. Never infer or guess the date. AFTER work: + {% if STATE_BACKEND == "git-notes" %} + 1. Persist your learnings as JSON via the State Protocol: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"learnings": ["..."], "timestamp": "{current_datetime}"}'` + 2. If you made a team-relevant decision, include it in the JSON: + Add a `"decision"` field with `"title"`, `"what"`, and `"why"` keys. + Scribe will merge decisions into the canonical decisions.md. + {% elif STATE_BACKEND == "two-layer" %} + 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": + architecture decisions, patterns, user preferences, key file paths. + (Scribe commits this to the orphan branch.) + 2. If you made a team-relevant decision, write BOTH: + a. A git note on HEAD with promote flag: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + b. A drop file: .squad/decisions/inbox/{name}-{brief-slug}.md + (Scribe merges to orphan branch; Ralph promotes note after PR merge.) + {% else %} 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": architecture decisions, patterns, user preferences, key file paths. 2. If you made a team-relevant decision, write to: .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} 3. SKILL EXTRACTION: If you found a reusable pattern, write/update .squad/skills/{skill-name}/SKILL.md (read templates/skill.md for format). @@ -878,18 +970,47 @@ prompt: | You are the Scribe. Read .squad/agents/scribe/charter.md. TEAM ROOT: {team_root} CURRENT_DATETIME: {current_datetime} + STATE_BACKEND: {state_backend} SPAWN MANIFEST: {spawn_manifest} Tasks (in order): - 0. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "git-notes" or STATE_BACKEND == "two-layer" %} + 0. STATE LEAK GUARD: Check if any agent accidentally committed or staged state files + (.squad/decisions.md, agents/*/history.md, log/*, orchestration-log/*, decisions/inbox/*) + to the working branch. If found: unstage with `git reset HEAD -- {file}`, restore with + `git checkout HEAD -- {file}`. If leaked in last commit, amend to remove. Log count. + {% endif %} + 0b. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. 1. DECISIONS ARCHIVE [HARD GATE]: If decisions.md >= 20480 bytes, archive entries older than 30 days NOW. If >= 51200 bytes, archive entries older than 7 days. Do not skip this step. + {% if STATE_BACKEND == "git-notes" %} + 2. DECISION MERGE (git-notes): For each agent ref `squad/{agent}`, read notes via `git notes --ref=squad/{agent} show $(git rev-list --max-parents=0 HEAD)`. Extract any `decision` entries. Merge into decisions.md. Clear the decision field by overwriting the note without it. + {% elif STATE_BACKEND == "two-layer" %} + 2. DECISION MERGE (two-layer): Merge .squad/decisions/inbox/ → decisions.md AND read agent note refs for any decisions with `promote_to_permanent`. Deduplicate. Push note refs: `git push origin 'refs/notes/squad/*'` + {% else %} 2. DECISION INBOX: Merge .squad/decisions/inbox/ → decisions.md, delete inbox files. Deduplicate. + {% endif %} 3. ORCHESTRATION LOG: Write .squad/orchestration-log/{timestamp}-{agent}.md per agent. Use ISO 8601 UTC timestamp. 4. SESSION LOG: Write .squad/log/{timestamp}-{topic}.md. Brief. Use ISO 8601 UTC timestamp. + {% if STATE_BACKEND == "git-notes" %} + 5. CROSS-AGENT (git-notes): For team updates, write to affected agents' note refs via `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{agent}" -Content '{json}'`. + {% else %} 5. CROSS-AGENT: Append team updates to affected agents' history.md. + {% endif %} 6. HISTORY SUMMARIZATION [HARD GATE]: If any history.md >= 15360 bytes (15KB), summarize now. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + 7. GIT COMMIT (orphan): Stage `.squad/` state files and commit to the `squad-state` orphan branch: + a. Identify changed `.squad/` state files via `git status --porcelain` (decisions.md, agents/*/history.md, log/*, orchestration-log/*). + b. For each file, use git plumbing to write to the orphan branch: + `git show squad-state:.squad/{path}` to check if file exists on orphan. + Use `git checkout squad-state -- .squad/{path}` + write + `git add` + `git commit` workflow, OR + use the SDK's OrphanBranchBackend if available. + c. Reset working tree state files: `git checkout HEAD -- .squad/` to avoid polluting the working branch. + d. Push orphan branch: `git push origin squad-state` + ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + {% else %} 7. GIT COMMIT: Stage only the exact `.squad/` files Scribe wrote in this session. Use `git status --porcelain` filtered to allowed paths (decisions.md, decisions-archive.md, agents/{name}/history.md, agents/{name}/history-archive.md, log/*, orchestration-log/*). Stage each file individually with `git add -- `. Handle renames by extracting destination path (`-replace '^.* -> ',''`). Commit with -F (write msg to temp file). Skip if nothing staged. ⚠️ NEVER use `git add .squad/` or broad globs. + {% endif %} 8. HEALTH REPORT: Log decisions.md before/after size, inbox count processed, history files summarized. Never speak to user. ⚠️ End with plain text summary after all tool calls. @@ -947,6 +1068,8 @@ If the user wants to remove someone: ## Source of Truth Hierarchy +> **State backend note:** Files below marked as "Derived / append-only" are **mutable state** — their storage location depends on the configured `STATE_BACKEND`. On `worktree` (default), they live on disk. On `git-notes`, they live in git notes refs. On `orphan`, they live on the `squad-state` orphan branch. On `two-layer`, commit-scoped annotations live in git notes AND permanent state lives on the orphan branch. Files marked as "Authoritative" are **static config** and always live on disk regardless of backend. + | File | Status | Who May Write | Who May Read | |------|--------|---------------|--------------| | `.github/agents/squad.agent.md` | **Authoritative governance.** All roles, handoffs, gates, and enforcement rules. | Repo maintainer (human) | Squad (Coordinator) | diff --git a/templates/notes-protocol.md b/templates/notes-protocol.md new file mode 100644 index 000000000..5a1d03a5f --- /dev/null +++ b/templates/notes-protocol.md @@ -0,0 +1,202 @@ +# Squad Notes Protocol + +> Contract for agent state via git notes. Agents write commit-scoped context +> here instead of modifying `.squad/` files in PRs. +> +> **Version:** 1.0 +> **Backends:** `git-notes`, `orphan` + +--- + +## Overview + +Squad state has two layers: + +1. **Git notes layer** (this document) — thin, commit-scoped annotations that + attach agent context to commits without appearing in PRs or diffs. +2. **Permanent state layer** — long-lived decisions, routing rules, and archives + stored via the configured state backend (`git-notes` or `orphan` branch). + +Agents write notes during their work rounds. Ralph promotes flagged notes to +permanent state after a PR merges. + +--- + +## Namespaces + +Each agent writes to its own namespace to prevent conflicts: + +| Namespace | Owner | Purpose | +|-----------|-------|---------| +| `refs/notes/squad/data` | Data | Architecture decisions, implementation choices | +| `refs/notes/squad/worf` | Worf | Security reviews, vulnerability assessments | +| `refs/notes/squad/seven` | Seven | Documentation quality, API contract decisions | +| `refs/notes/squad/ralph` | Ralph | Work-round progress, task-state annotations | +| `refs/notes/squad/q` | Q | Devil's advocate findings, risk assessments | +| `refs/notes/squad/research` | Any agent | Research notes that should survive branch deletion | +| `refs/notes/squad/review` | Any agent | Code review context (mirrors Gerrit's pattern) | + +**Rule**: Only write to your own namespace. The shared namespaces +(`research`, `review`) use `append` — never `add`. + +--- + +## Note JSON Schema + +All notes MUST be valid JSON. Minimum required fields: + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision | research | review | progress | security", + "content": "..." +} +``` + +### Decision notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "decision", + "decision": "Use JWT RS256 for auth middleware", + "reasoning": "Existing pattern in codebase — auth.go:47-89.", + "alternatives_considered": ["HS256", "session tokens"], + "confidence": "high", + "promote_to_permanent": true +} +``` + +Set `"promote_to_permanent": true` to signal Ralph to copy this to +`decisions.md` after the PR merges. + +### Research notes + +```json +{ + "agent": "Data", + "timestamp": "2026-03-23T14:00:00Z", + "type": "research", + "topic": "JWT vs session tokens", + "findings": {}, + "effort_hours": 2.5, + "archive_on_close": true +} +``` + +Set `"archive_on_close": true` to signal Ralph to archive this to +`state/research/` even if the PR is rejected. + +--- + +## Write Commands + +```bash +# Write a decision note on the current commit +git notes --ref=squad/{your-agent} add \ + -m '{"agent":"{Agent}","timestamp":"...","type":"decision","decision":"..."}' \ + HEAD + +# Append to an existing note (multiple items on same commit) +git notes --ref=squad/{your-agent} append \ + -m '{"agent":"{Agent}","timestamp":"...","type":"progress","content":"..."}' \ + HEAD + +# Read your note +git notes --ref=squad/{your-agent} show HEAD + +# List all commits with notes in your namespace +git notes --ref=squad/{your-agent} list +``` + +Or use the helper script: + +```powershell +./scripts/notes/write-note.ps1 -Agent data -Type decision \ + -Content '{"decision":"Use JWT","reasoning":"..."}' \ + [-Commit HEAD] [-Promote] [-Archive] +``` + +--- + +## Fetch / Push + +**Notes are NOT fetched or pushed by default.** Every clone needs setup. + +### One-time setup + +```bash +git config --add remote.origin.fetch 'refs/notes/*:refs/notes/*' +git fetch origin 'refs/notes/*:refs/notes/*' +``` + +Or use the helper: + +```powershell +./scripts/notes/fetch.ps1 -Setup +``` + +### Every work round + +1. **Start**: `git fetch origin 'refs/notes/*:refs/notes/*'` +2. **End**: `git push origin 'refs/notes/*:refs/notes/*'` + +--- + +## Conflict Handling + +1. **Per-agent namespaces prevent 99% of conflicts.** Only one agent writes to + `refs/notes/squad/data`, so there are no write conflicts in normal use. + +2. **Same agent, two machines:** First push wins. Losing machine should fetch + and append: + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes --ref=squad/{agent} append -m '{...}' HEAD + git push origin 'refs/notes/*:refs/notes/*' + ``` + +3. **Shared namespaces** (`research`, `review`): Always use `git notes append`, + never `git notes add`. + +4. **Push conflict recovery:** + ```bash + git fetch origin 'refs/notes/*:refs/notes/*' + git notes merge refs/notes/remotes/origin/squad/{namespace} + git push origin 'refs/notes/*:refs/notes/*' + ``` + +--- + +## When to Use Notes vs State Backend + +| Use git notes | Use state backend | +|---------------|-------------------| +| Why THIS choice on THIS commit | Universal routing rules, conventions | +| Decisions scoped to a feature | Long-lived decisions for all future work | +| Research for a specific investigation | Research archives (promoted from notes) | +| Security sign-offs per commit | Agent history persisting across features | +| Agent-to-agent context for current feature | Team agreements and policies | + +When in doubt: **notes first, promote to permanent state later.** Ralph handles +the promotion automatically when `promote_to_permanent` is set. + +--- + +## Ralph Promotion Rules + +**After PR merge:** + +1. Fetch all notes from remote +2. Traverse commits reachable from the default branch that have notes +3. For each note with `"promote_to_permanent": true` → append to `decisions.md` +4. Push state + +**After PR close/rejection:** + +1. List notes in `squad/research` on the closed branch's commits +2. For each note with `"archive_on_close": true` → archive to `research/` +3. Push state +4. Notes on rejected commits are NOT promoted — this is the desired behavior diff --git a/templates/scribe-charter.md b/templates/scribe-charter.md index 308b780e2..a2031d072 100644 --- a/templates/scribe-charter.md +++ b/templates/scribe-charter.md @@ -24,6 +24,59 @@ **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory). +**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. If it's `"orphan"` or `"git-notes"`, run the **State Leak Guard** before any other work. + +### State Leak Guard (orphan/git-notes backends only) + +Before logging or merging, check if any agent accidentally committed state files to the working branch: + +```powershell +# Check if state files are staged or committed but shouldn't be +$stateFiles = @( + '.squad/decisions.md', + '.squad/decisions-archive.md' +) +$statePatterns = @( + '.squad/agents/*/history.md', + '.squad/agents/*/history-archive.md', + '.squad/log/*', + '.squad/orchestration-log/*', + '.squad/decisions/inbox/*' +) + +# 1. Check git status for accidentally staged state files +$dirty = git status --porcelain | Where-Object { $_.Length -gt 3 } | ForEach-Object { + $_.Substring(3) -replace '^.* -> ','' +} | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($dirty) { + # Unstage any accidentally added state files + $dirty | ForEach-Object { git reset HEAD -- $_ 2>$null } + # Restore from HEAD (discard working tree changes for state files) + $dirty | ForEach-Object { git checkout HEAD -- $_ 2>$null } +} + +# 2. Check if the most recent commit on this branch has state files +$lastCommitFiles = git diff-tree --no-commit-id --name-only -r HEAD 2>$null +$leakedInCommit = $lastCommitFiles | Where-Object { + $f = $_ + ($f -in $stateFiles) -or ($statePatterns | Where-Object { $f -like $_ }) +} + +if ($leakedInCommit) { + # State files leaked into the last commit — amend to remove them + $leakedInCommit | ForEach-Object { git rm --cached -- $_ 2>$null } + git commit --amend --no-edit 2>$null +} +``` + +If any files were cleaned, log: `⚠️ State leak guard: removed {N} state file(s) from working branch.` + +After the guard, proceed with normal Scribe work (but persist state via the configured backend, not the working branch). + After every substantial work session: 1. **Log the session** to `.squad/log/{timestamp}-{topic}.md`: @@ -57,9 +110,50 @@ After every substantial work session: ``` 5. **Commit `.squad/` changes:** + **Check `STATE_BACKEND` from spawn prompt.** This determines WHERE state gets committed. + **IMPORTANT — Windows compatibility:** Do NOT use `git -C {path}` (unreliable with Windows paths). Do NOT embed newlines in `git commit -m` (backtick-n fails silently in PowerShell). - Instead: + + **If STATE_BACKEND is "orphan":** + State files must be committed to the `squad-state` orphan branch, NOT the working branch. + - Identify changed `.squad/` state files via `git status --porcelain` filtered to allowed paths. + - For each file, use git plumbing to write to the orphan branch: + ```powershell + # Create a temporary worktree for the orphan branch + $orphanWt = Join-Path ([System.IO.Path]::GetTempPath()) "squad-state-$(Get-Random)" + git worktree add $orphanWt squad-state 2>$null + if ($LASTEXITCODE -ne 0) { git worktree add --orphan $orphanWt squad-state } + # Copy state files to orphan worktree + $filesToSync | ForEach-Object { + $dest = Join-Path $orphanWt $_ + New-Item -ItemType Directory -Path (Split-Path $dest) -Force | Out-Null + Copy-Item $_ $dest -Force + } + # Commit in orphan worktree + Push-Location $orphanWt + git add .squad/ + git diff --cached --quiet + if ($LASTEXITCODE -ne 0) { + $msgFile = [System.IO.Path]::GetTempFileName() + Set-Content -Path $msgFile -Value "docs(ai-team): $summary" -Encoding utf8 + git commit -F $msgFile + Remove-Item $msgFile + git push origin squad-state + } + Pop-Location + git worktree remove $orphanWt --force + ``` + - After committing to orphan, reset working tree state files: `git checkout HEAD -- .squad/` + - ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + + **If STATE_BACKEND is "git-notes":** + State is already persisted in git notes refs by agents. Scribe only needs to: + - Push any locally created note refs: `git push origin 'refs/notes/squad/*'` + - Commit decisions.md (the merged canonical file) to the working branch as normal. + + **If STATE_BACKEND is "worktree" (default):** + Commit to the working branch as normal: - `cd` into the team root first. - Stage only files Scribe actually modified in this session. Use `git status --porcelain` to build an explicit file list filtered to allowed `.squad/` paths: diff --git a/templates/scripts/notes/fetch.ps1 b/templates/scripts/notes/fetch.ps1 new file mode 100644 index 000000000..5adc19480 --- /dev/null +++ b/templates/scripts/notes/fetch.ps1 @@ -0,0 +1,88 @@ +#!/usr/bin/env pwsh +# scripts/notes/fetch.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Fetch git notes from remote. Run on every Ralph-watch startup and before +# any agent reads or writes notes. +# +# Usage: +# ./scripts/notes/fetch.ps1 # fetch only +# ./scripts/notes/fetch.ps1 -Setup # first-time: add refspec + fetch +# ./scripts/notes/fetch.ps1 -Merge # fetch + merge (use after push conflict) +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [string]$Remote = "origin", + [string]$RepoPath = ".", + [switch]$Setup, + [switch]$Merge, + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/fetch] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath + +# ── One-time setup: add fetch refspec ────────────────────────────────────── +if ($Setup) { + $existing = git -C $repo config --get-all "remote.$Remote.fetch" 2>&1 | + Where-Object { $_ -match "refs/notes" } + if ($existing) { + Log "Notes refspec already configured." DarkGray + } else { + git -C $repo config --add "remote.$Remote.fetch" "refs/notes/*:refs/notes/*" + Log "Added notes refspec to remote.$Remote.fetch" Green + } +} + +# ── Fetch notes ───────────────────────────────────────────────────────────── +Log "Fetching notes from $Remote..." +$output = git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 +if ($LASTEXITCODE -ne 0) { + Log "Fetch warning: $output" DarkYellow +} else { + Log "Notes fetched." Green +} + +# ── Merge notes if requested (after push conflict) ────────────────────────── +if ($Merge) { + # Abort any stale merge-in-progress state + $mergeLock = Join-Path $repo ".git/NOTES_MERGE_PARTIAL" + if (Test-Path $mergeLock) { + Log "Stale notes merge in progress — aborting before retry" DarkYellow + git -C $repo notes merge --abort 2>&1 | Out-Null + } + + $namespaces = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + foreach ($ref in $namespaces) { + $ns = $ref -replace "refs/notes/", "" + $remoteRef = "refs/notes/remotes/$Remote/$ns" + $remoteExists = git -C $repo for-each-ref $remoteRef --format="%(refname)" 2>&1 + if ($remoteExists) { + Log "Merging notes: $ns (cat_sort_uniq)" + git -C $repo notes --ref=$ns merge -s cat_sort_uniq $remoteRef 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Log " Merge failed on $ns — aborting and continuing" Red + git -C $repo notes merge --abort 2>&1 | Out-Null + } + } + } + Log "Notes merge complete." Green +} + +# ── Show available namespaces ──────────────────────────────────────────────── +if (-not $Quiet) { + $refs = git -C $repo for-each-ref "refs/notes/squad/" --format="%(refname)" 2>&1 + if ($refs) { + Log "Available namespaces:" + foreach ($r in $refs) { + $count = (git -C $repo notes --ref=($r -replace "refs/notes/","") list 2>&1 | + Where-Object { $_ -ne "" } | Measure-Object -Line).Lines + Log " $r ($count notes)" DarkGray + } + } else { + Log "No squad notes yet." DarkGray + } +} diff --git a/templates/scripts/notes/write-note.ps1 b/templates/scripts/notes/write-note.ps1 new file mode 100644 index 000000000..dbc719d35 --- /dev/null +++ b/templates/scripts/notes/write-note.ps1 @@ -0,0 +1,126 @@ +#!/usr/bin/env pwsh +# scripts/notes/write-note.ps1 +# ───────────────────────────────────────────────────────────────────────────── +# Helper for agents to write notes without wrestling with JSON escaping. +# Validates namespace ownership, handles conflicts, pushes automatically. +# +# Usage: +# ./scripts/notes/write-note.ps1 -Agent data -Type decision \ +# -Content '{"decision":"Use JWT","reasoning":"..."}' \ +# [-Commit HEAD] [-Promote] [-Archive] +# ───────────────────────────────────────────────────────────────────────────── + +[CmdletBinding()] +param( + [Parameter(Mandatory)][string]$Agent, + + [Parameter(Mandatory)] + [ValidateSet("decision","research","review","security-review","progress", + "api-contract","risk-assessment","routing-discovery","counter-argument")] + [string]$Type, + + [Parameter(Mandatory)] + [string]$Content, # JSON object with type-specific fields + + [string]$Commit = "HEAD", + [string]$RepoPath = ".", + [string]$Remote = "origin", + [switch]$Promote, # set promote_to_permanent: true + [switch]$Archive, # set archive_on_close: true + [switch]$NoPush, # skip auto-push + [switch]$Quiet +) + +function Log ([string]$msg, [string]$color = "White") { + if (-not $Quiet) { Write-Host "[notes/write] $msg" -ForegroundColor $color } +} + +$repo = Resolve-Path $RepoPath +$namespace = "squad/$($Agent.ToLower())" + +# ── Validate JSON content ──────────────────────────────────────────────────── +try { + $parsed = $Content | ConvertFrom-Json -ErrorAction Stop +} catch { + Write-Error "Content must be valid JSON. Got: $Content" + exit 1 +} + +# ── Build full note object ──────────────────────────────────────────────────── +$note = [ordered]@{ + agent = (Get-Culture).TextInfo.ToTitleCase($Agent.ToLower()) + timestamp = [System.DateTime]::UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") + type = $Type +} + +# Merge content fields into note +$parsed.PSObject.Properties | ForEach-Object { $note[$_.Name] = $_.Value } + +# Add flag fields +if ($Promote) { $note["promote_to_permanent"] = $true } +if ($Archive) { $note["archive_on_close"] = $true } + +$noteJson = $note | ConvertTo-Json -Compress -Depth 10 + +# ── Fetch first to avoid conflicts ─────────────────────────────────────────── +Log "Fetching notes before write..." +git -C $repo fetch $Remote "refs/notes/*:refs/notes/*" 2>&1 | Out-Null + +# ── Check if note already exists on this commit ───────────────────────────── +$existing = git -C $repo notes --ref=$namespace show $Commit 2>&1 +$useAppend = ($LASTEXITCODE -eq 0) + +if ($useAppend) { + Log "Note exists on $Commit — appending" DarkYellow + git -C $repo notes --ref=$namespace append -m $noteJson $Commit +} else { + git -C $repo notes --ref=$namespace add -m $noteJson $Commit +} + +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to write note to refs/notes/$namespace on $Commit" + exit 1 +} + +Log "Note written to refs/notes/$namespace on $($Commit.Substring(0,[Math]::Min(8,$Commit.Length)))" Green + +# ── Push with retry ────────────────────────────────────────────────────────── +if (-not $NoPush) { + $maxRetries = 5 + $nsRef = "refs/notes/$namespace" + + for ($i = 0; $i -lt $maxRetries; $i++) { + Log "Pushing notes (attempt $($i+1))..." + $pushOut = git -C $repo push $Remote "${nsRef}:${nsRef}" 2>&1 + if ($LASTEXITCODE -eq 0) { + Log "Notes pushed successfully." Green + break + } + + if ($pushOut -match "non-fast-forward|fetch first|rejected") { + Log "Push conflict — fetch-first retry..." DarkYellow + + # Force-fetch: overwrite local ref with current remote state + git -C $repo fetch $Remote "${nsRef}:${nsRef}" 2>&1 | Out-Null + + # Re-append our note on top of the now-current remote state + git -C $repo notes --ref=$namespace append -m $noteJson $Commit 2>&1 | Out-Null + + $jitter = Get-Random -Minimum 0 -Maximum 1000 + $sleep = [Math]::Pow(2, $i) + $jitter / 1000 + Start-Sleep -Seconds $sleep + + } else { + Log "Push error: $pushOut" Red + if ($i -eq $maxRetries - 1) { + Write-Warning "Failed after $maxRetries retries. Push manually: git push origin '${nsRef}:${nsRef}'" + } + } + } +} + +# ── Show result ─────────────────────────────────────────────────────────────── +if (-not $Quiet) { + Log "Note content:" + $note | ConvertTo-Json -Depth 5 | Write-Host -ForegroundColor DarkGray +} diff --git a/templates/squad.agent.md.template b/templates/squad.agent.md.template index 01e18dfad..4ef1a3045 100644 --- a/templates/squad.agent.md.template +++ b/templates/squad.agent.md.template @@ -107,6 +107,8 @@ The `union` merge driver keeps all lines from both sides, which is correct for a **On every session start:** Run `git config user.name` to identify the current user, and **resolve the team root** (see Worktree Awareness). Store the team root — all `.squad/` paths must be resolved relative to it. Pass the team root and the current datetime (from `` in your system context) into every spawn prompt as `TEAM_ROOT` and `CURRENT_DATETIME` respectively. Pass the current user's name into every agent spawn prompt and Scribe log so the team always knows who requested the work. Check `.squad/identity/now.md` if it exists — it tells you what the team was last focused on. Update it if the focus has shifted. +**Resolve state backend:** Read `.squad/config.json` and check the `stateBackend` field. Valid values: `"worktree"` (default), `"git-notes"`, `"orphan"`, `"two-layer"`. Store as `STATE_BACKEND` and pass it into every spawn prompt. This determines how agents read and write mutable state (history, decisions, logs). Static config (charters, team.md, routing.md) always lives on disk regardless of backend. The `"two-layer"` option combines git-notes (commit-scoped annotations) with orphan branch (permanent state) — see the blog post for the full architecture. + **⚡ Context caching:** After the first message in a session, `team.md`, `routing.md`, and `registry.json` are already in your context. Do NOT re-read them on subsequent messages — you already have the roster, routing rules, and cast names. Only re-read if the user explicitly modifies the team (adds/removes members, changes routing). **Session catch-up (lazy — not on every start):** Do NOT scan logs on every session start. Only provide a catch-up summary when: @@ -223,13 +225,16 @@ The `name` parameter generates the human-readable agent ID shown in the tasks pa **When you detect a directive:** -1. Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: - ``` - ### {timestamp}: User directive - **By:** {user name} (via Copilot) - **What:** {the directive, verbatim or lightly paraphrased} - **Why:** User request — captured for team memory - ``` +1. Capture the directive: + - **worktree/orphan backend:** Write it immediately to `.squad/decisions/inbox/copilot-directive-{timestamp}.md` using this format: + ``` + ### {timestamp}: User directive + **By:** {user name} (via Copilot) + **What:** {the directive, verbatim or lightly paraphrased} + **Why:** User request — captured for team memory + ``` + - **git-notes backend:** Persist via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/directives" -Content '{"timestamp": "{timestamp}", "by": "{user name}", "what": "...", "why": "User request"}'` 2. Acknowledge briefly: `"📌 Captured. {one-line summary of the directive}."` 3. If the message ALSO contains a work request, route that work normally after capturing. If it's directive-only, you're done — no agent spawn needed. @@ -344,7 +349,12 @@ prompt: | TARGET FILE(S): {exact file path(s)} Do the work. Keep it focused. + {% if STATE_BACKEND == "git-notes" %} + If you made a meaningful decision, persist it via: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"decision": {"title": "...", "what": "...", "why": "..."}}'` + {% else %} If you made a meaningful decision, write to .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} ⚠️ OUTPUT: Report outcomes in human terms. Never expose tool internals or SQL. ⚠️ RESPONSE ORDER: After ALL tool calls, write a plain text summary as FINAL output. @@ -598,8 +608,9 @@ When the user gives any task, the Coordinator MUST: To enable full parallelism, shared writes use a drop-box pattern that eliminates file conflicts: **decisions.md** — Agents do NOT write directly to `decisions.md`. Instead: -- Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` -- Scribe merges inbox entries into the canonical `.squad/decisions.md` and clears the inbox +- **worktree/orphan backend:** Agents write decisions to individual drop files: `.squad/decisions/inbox/{agent-name}-{brief-slug}.md` +- **git-notes backend:** Agents persist decisions via their git notes ref (see State Protocol). Scribe reads all agent notes and merges decisions. +- Scribe merges into the canonical `.squad/decisions.md` and clears the inbox (or note entries) - All agents READ from `.squad/decisions.md` at spawn time (last-merged snapshot) **orchestration-log/** — Scribe writes one entry per agent after each batch: @@ -799,7 +810,71 @@ prompt: | - Commit and push from the worktree {% endif %} + STATE_BACKEND: {state_backend} + + {% if STATE_BACKEND == "git-notes" %} + ## State Protocol — Git Notes + This project uses git-notes for mutable state. **DO NOT write to `.squad/` files for state.** + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading your state:** + Run: `powershell .squad/scripts/notes/fetch.ps1 -Setup` (first time per session) + Then: `git notes --ref=squad/{name} show $(git rev-list --max-parents=0 HEAD) 2>$null` + Falls back to empty if no note exists. + + **Writing state (history, decisions, learnings):** + Run: `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{json}'` + The helper handles JSON validation, conflict retry, and push. + + **Decisions:** Write decisions as JSON via your note ref. Scribe will merge them. + **Skills:** Skills are static config — write to `.squad/skills/` on disk as normal. + {% endif %} + + {% if STATE_BACKEND == "orphan" %} + ## State Protocol — Orphan Branch + This project uses an orphan branch (`squad-state`) for mutable state. + Static config (charters, team.md, routing.md) is on disk as normal — read those with `view`. + + **Reading state:** Read `.squad/` files on disk — they are synced from the orphan branch. + **Writing state:** Write to `.squad/` files on disk as normal during your session. + Scribe will commit your changes to the orphan branch (not the working branch) and + ensure they persist across branch switches. + + **Important:** Do NOT commit `.squad/` state files to the working branch yourself. + Scribe handles the orphan branch commit workflow. + {% endif %} + + {% if STATE_BACKEND == "two-layer" %} + ## State Protocol — Two-Layer (Git Notes + Orphan Branch) + This project uses the two-layer architecture from Tamir's blog: + - **Layer 1 (git notes):** Commit-scoped "why" annotations — invisible in PRs + - **Layer 2 (orphan branch):** Permanent state store — decisions, histories, logs + + Static config (charters, team.md, routing.md) is on disk as normal. + + **During your session:** + 1. Write commit-scoped annotations as git notes on HEAD: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + 2. Write bulk state (history, logs) to `.squad/` files on disk — Scribe moves them to the orphan branch. + + **Note flags:** + - `"promote_to_permanent": true` — Ralph promotes this to decisions.md after PR merge + - `"archive_on_close": true` — Worth keeping even if PR is rejected (valuable research) + - Neither flag — silently ignored if PR is rejected (correct for branch-specific decisions) + + **Important:** Do NOT commit `.squad/` state files to the working branch. + Scribe handles orphan commits. Ralph handles note promotion. + {% endif %} + + {% if STATE_BACKEND == "worktree" or STATE_BACKEND is not defined %} Read .squad/agents/{name}/history.md (your project knowledge). + {% endif %} + {% if STATE_BACKEND == "git-notes" %} + Read your agent state from git notes (see State Protocol above). + {% endif %} + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + Read .squad/agents/{name}/history.md (your project knowledge — synced from orphan branch). + {% endif %} Read .squad/decisions.md (team decisions to respect). If .squad/identity/wisdom.md exists, read it before starting work. If .squad/identity/now.md exists, read it at spawn time. @@ -823,10 +898,27 @@ prompt: | ⚠️ DATES: When writing dates in any file (decisions, history, logs), use ONLY the CURRENT_DATETIME value above. Never infer or guess the date. AFTER work: + {% if STATE_BACKEND == "git-notes" %} + 1. Persist your learnings as JSON via the State Protocol: + `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{name}" -Content '{"learnings": ["..."], "timestamp": "{current_datetime}"}'` + 2. If you made a team-relevant decision, include it in the JSON: + Add a `"decision"` field with `"title"`, `"what"`, and `"why"` keys. + Scribe will merge decisions into the canonical decisions.md. + {% elif STATE_BACKEND == "two-layer" %} + 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": + architecture decisions, patterns, user preferences, key file paths. + (Scribe commits this to the orphan branch.) + 2. If you made a team-relevant decision, write BOTH: + a. A git note on HEAD with promote flag: + `git notes --ref=squad/{name} add -f -m '{"agent":"{Name}","type":"decision","decision":"...","promote_to_permanent":true}' HEAD` + b. A drop file: .squad/decisions/inbox/{name}-{brief-slug}.md + (Scribe merges to orphan branch; Ralph promotes note after PR merge.) + {% else %} 1. APPEND to .squad/agents/{name}/history.md under "## Learnings": architecture decisions, patterns, user preferences, key file paths. 2. If you made a team-relevant decision, write to: .squad/decisions/inbox/{name}-{brief-slug}.md + {% endif %} 3. SKILL EXTRACTION: If you found a reusable pattern, write/update .squad/skills/{skill-name}/SKILL.md (read templates/skill.md for format). @@ -878,18 +970,47 @@ prompt: | You are the Scribe. Read .squad/agents/scribe/charter.md. TEAM ROOT: {team_root} CURRENT_DATETIME: {current_datetime} + STATE_BACKEND: {state_backend} SPAWN MANIFEST: {spawn_manifest} Tasks (in order): - 0. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "git-notes" or STATE_BACKEND == "two-layer" %} + 0. STATE LEAK GUARD: Check if any agent accidentally committed or staged state files + (.squad/decisions.md, agents/*/history.md, log/*, orchestration-log/*, decisions/inbox/*) + to the working branch. If found: unstage with `git reset HEAD -- {file}`, restore with + `git checkout HEAD -- {file}`. If leaked in last commit, amend to remove. Log count. + {% endif %} + 0b. PRE-CHECK: Stat decisions.md size and count inbox/ files. Record measurements. 1. DECISIONS ARCHIVE [HARD GATE]: If decisions.md >= 20480 bytes, archive entries older than 30 days NOW. If >= 51200 bytes, archive entries older than 7 days. Do not skip this step. + {% if STATE_BACKEND == "git-notes" %} + 2. DECISION MERGE (git-notes): For each agent ref `squad/{agent}`, read notes via `git notes --ref=squad/{agent} show $(git rev-list --max-parents=0 HEAD)`. Extract any `decision` entries. Merge into decisions.md. Clear the decision field by overwriting the note without it. + {% elif STATE_BACKEND == "two-layer" %} + 2. DECISION MERGE (two-layer): Merge .squad/decisions/inbox/ → decisions.md AND read agent note refs for any decisions with `promote_to_permanent`. Deduplicate. Push note refs: `git push origin 'refs/notes/squad/*'` + {% else %} 2. DECISION INBOX: Merge .squad/decisions/inbox/ → decisions.md, delete inbox files. Deduplicate. + {% endif %} 3. ORCHESTRATION LOG: Write .squad/orchestration-log/{timestamp}-{agent}.md per agent. Use ISO 8601 UTC timestamp. 4. SESSION LOG: Write .squad/log/{timestamp}-{topic}.md. Brief. Use ISO 8601 UTC timestamp. + {% if STATE_BACKEND == "git-notes" %} + 5. CROSS-AGENT (git-notes): For team updates, write to affected agents' note refs via `powershell .squad/scripts/notes/write-note.ps1 -Ref "squad/{agent}" -Content '{json}'`. + {% else %} 5. CROSS-AGENT: Append team updates to affected agents' history.md. + {% endif %} 6. HISTORY SUMMARIZATION [HARD GATE]: If any history.md >= 15360 bytes (15KB), summarize now. + {% if STATE_BACKEND == "orphan" or STATE_BACKEND == "two-layer" %} + 7. GIT COMMIT (orphan): Stage `.squad/` state files and commit to the `squad-state` orphan branch: + a. Identify changed `.squad/` state files via `git status --porcelain` (decisions.md, agents/*/history.md, log/*, orchestration-log/*). + b. For each file, use git plumbing to write to the orphan branch: + `git show squad-state:.squad/{path}` to check if file exists on orphan. + Use `git checkout squad-state -- .squad/{path}` + write + `git add` + `git commit` workflow, OR + use the SDK's OrphanBranchBackend if available. + c. Reset working tree state files: `git checkout HEAD -- .squad/` to avoid polluting the working branch. + d. Push orphan branch: `git push origin squad-state` + ⚠️ NEVER commit `.squad/` state files to the working branch when using orphan backend. + {% else %} 7. GIT COMMIT: Stage only the exact `.squad/` files Scribe wrote in this session. Use `git status --porcelain` filtered to allowed paths (decisions.md, decisions-archive.md, agents/{name}/history.md, agents/{name}/history-archive.md, log/*, orchestration-log/*). Stage each file individually with `git add -- `. Handle renames by extracting destination path (`-replace '^.* -> ',''`). Commit with -F (write msg to temp file). Skip if nothing staged. ⚠️ NEVER use `git add .squad/` or broad globs. + {% endif %} 8. HEALTH REPORT: Log decisions.md before/after size, inbox count processed, history files summarized. Never speak to user. ⚠️ End with plain text summary after all tool calls. @@ -947,6 +1068,8 @@ If the user wants to remove someone: ## Source of Truth Hierarchy +> **State backend note:** Files below marked as "Derived / append-only" are **mutable state** — their storage location depends on the configured `STATE_BACKEND`. On `worktree` (default), they live on disk. On `git-notes`, they live in git notes refs. On `orphan`, they live on the `squad-state` orphan branch. On `two-layer`, commit-scoped annotations live in git notes AND permanent state lives on the orphan branch. Files marked as "Authoritative" are **static config** and always live on disk regardless of backend. + | File | Status | Who May Write | Who May Read | |------|--------|---------------|--------------| | `.github/agents/squad.agent.md` | **Authoritative governance.** All roles, handoffs, gates, and enforcement rules. | Repo maintainer (human) | Squad (Coordinator) | diff --git a/test/cli-command-wiring.test.ts b/test/cli-command-wiring.test.ts index 5a72adf81..7139988ca 100644 --- a/test/cli-command-wiring.test.ts +++ b/test/cli-command-wiring.test.ts @@ -15,8 +15,8 @@ import { join, basename } from 'node:path'; const COMMANDS_DIR = join(process.cwd(), 'packages', 'squad-cli', 'src', 'cli', 'commands'); const CLI_ENTRY = join(process.cwd(), 'packages', 'squad-cli', 'src', 'cli-entry.ts'); -// All commands should now be wired — no known-unwired commands remaining. -const KNOWN_UNWIRED = new Set([]); +// sync.ts is internal — not exposed as a user-facing command (absorbed into upgrade flow). +const KNOWN_UNWIRED = new Set(['sync']); describe('CLI command wiring regression (issues #224, #236, #237)', () => { const commandFiles = readdirSync(COMMANDS_DIR) diff --git a/test/speed-gates.test.ts b/test/speed-gates.test.ts index 03c2b930c..42c15cdd7 100644 --- a/test/speed-gates.test.ts +++ b/test/speed-gates.test.ts @@ -41,7 +41,7 @@ describe('Speed: --help is scannable', { timeout: 30_000 }, () => { await harness.waitForExit(15000); const output = harness.captureFrame(); const lines = output.split('\n').filter(l => l.trim()); - expect(lines.length).toBeLessThanOrEqual(125); + expect(lines.length).toBeLessThanOrEqual(130); }); it('first 5 lines tell user what to do next', async () => { diff --git a/test/state-backend.test.ts b/test/state-backend.test.ts index f7d1d9d82..e718efcf1 100644 --- a/test/state-backend.test.ts +++ b/test/state-backend.test.ts @@ -3,8 +3,9 @@ import { mkdirSync, rmSync, writeFileSync, readFileSync, existsSync } from 'node import { join } from 'node:path'; import { execSync } from 'node:child_process'; import { randomBytes } from 'node:crypto'; -import { WorktreeBackend, GitNotesBackend, OrphanBranchBackend, resolveStateBackend, validateStateKey } from '../packages/squad-sdk/src/state-backend.js'; +import { WorktreeBackend, GitNotesBackend, OrphanBranchBackend, resolveStateBackend, validateStateKey, StateBackendStorageAdapter } from '../packages/squad-sdk/src/state-backend.js'; import type { StateBackendType } from '../packages/squad-sdk/src/state-backend.js'; +import { resolveSquadState } from '../packages/squad-sdk/src/resolution.js'; const TMP = join(process.cwd(), `.test-state-backend-${randomBytes(4).toString('hex')}`); function git(args: string, cwd = TMP): string { @@ -31,7 +32,7 @@ describe('WorktreeBackend', () => { expect(b.list('agents')).toContain('data.md'); expect(b.list('agents')).toContain('picard.md'); }); it('list returns empty for non-existent directory', () => { expect(new WorktreeBackend(squadDir()).list('nonexistent')).toEqual([]); }); - it('name is worktree', () => { expect(new WorktreeBackend(squadDir()).name).toBe('worktree'); }); + it('name is local', () => { expect(new WorktreeBackend(squadDir()).name).toBe('local'); }); }); describe('GitNotesBackend', () => { @@ -47,6 +48,33 @@ describe('GitNotesBackend', () => { it('multiple writes update the same key', () => { const b = new GitNotesBackend(TMP); b.write('c.json', '1'); expect(b.read('c.json')).toBe('1'); b.write('c.json', '2'); expect(b.read('c.json')).toBe('2'); }); it('normalizes Windows paths', () => { const b = new GitNotesBackend(TMP); b.write('agents\\data.md', 'D'); expect(b.read('agents/data.md')).toBe('D'); }); it('name is git-notes', () => { expect(new GitNotesBackend(TMP).name).toBe('git-notes'); }); + it('state persists across branch switches (root-commit anchor)', { timeout: 15_000 }, () => { + // 1. Write state on main + const b = new GitNotesBackend(TMP); + b.write('decisions.md', '# Team Decisions'); + b.write('agents/data.md', 'Data config'); + expect(b.read('decisions.md')).toBe('# Team Decisions'); + + // 2. Create and switch to a feature branch + git('checkout -b feature-xyz'); + + // 3. Make a new commit on the feature branch (HEAD now differs from main) + writeFileSync(join(TMP, 'feature.txt'), 'new feature\n'); + git('add feature.txt'); + git('commit -m "add feature"'); + + // 4. Read state — should still be there (anchor is root commit, not HEAD) + const b2 = new GitNotesBackend(TMP); + expect(b2.read('decisions.md')).toBe('# Team Decisions'); + expect(b2.read('agents/data.md')).toBe('Data config'); + expect(b2.list('agents')).toContain('data.md'); + + // 5. Switch back to previous branch — state still there + git('checkout -'); + const b3 = new GitNotesBackend(TMP); + expect(b3.read('decisions.md')).toBe('# Team Decisions'); + expect(b3.read('agents/data.md')).toBe('Data config'); + }); }); describe('OrphanBranchBackend', () => { @@ -75,10 +103,10 @@ describe('resolveStateBackend()', () => { const squadDir = () => join(TMP, '.squad'); beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); initRepo(); mkdirSync(squadDir(), { recursive: true }); }); afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); - it('defaults to worktree', () => { expect(resolveStateBackend(squadDir(), TMP).name).toBe('worktree'); }); - it('reads stateBackend from config.json', () => { + it('defaults to local', () => { expect(resolveStateBackend(squadDir(), TMP).name).toBe('local'); }); + it('reads stateBackend from config.json (git-notes migrates to two-layer)', () => { writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'git-notes' })); - expect(resolveStateBackend(squadDir(), TMP).name).toBe('git-notes'); + expect(resolveStateBackend(squadDir(), TMP).name).toBe('two-layer'); }); it('CLI override wins over config', () => { writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'git-notes' })); @@ -86,12 +114,16 @@ describe('resolveStateBackend()', () => { }); it('falls back on invalid type', () => { writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'bad' })); - expect(resolveStateBackend(squadDir(), TMP).name).toBe('worktree'); + expect(resolveStateBackend(squadDir(), TMP).name).toBe('local'); }); - it('falls back on malformed JSON', () => { writeFileSync(join(squadDir(), 'config.json'), 'bad'); expect(resolveStateBackend(squadDir(), TMP).name).toBe('worktree'); }); - it('external returns worktree stub', () => { expect(resolveStateBackend(squadDir(), TMP, 'external').name).toBe('worktree'); }); + it('falls back on malformed JSON', () => { writeFileSync(join(squadDir(), 'config.json'), 'bad'); expect(resolveStateBackend(squadDir(), TMP).name).toBe('local'); }); + it('external returns local stub', () => { expect(resolveStateBackend(squadDir(), TMP, 'external').name).toBe('local'); }); + it('legacy worktree alias accepted', () => { expect(resolveStateBackend(squadDir(), TMP, 'worktree' as any).name).toBe('local'); }); it('all valid types accepted', () => { - for (const t of ['worktree', 'external', 'git-notes', 'orphan'] as const) expect(resolveStateBackend(squadDir(), TMP, t)).toBeDefined(); + for (const t of ['local', 'external', 'orphan', 'two-layer'] as const) expect(resolveStateBackend(squadDir(), TMP, t)).toBeDefined(); + }); + it('legacy git-notes migrates to two-layer', () => { + expect(resolveStateBackend(squadDir(), TMP, 'git-notes' as any).name).toBe('two-layer'); }); }); @@ -180,4 +212,276 @@ describe('State Backend: Key injection blocked at backend level', () => { // WorktreeBackend uses path.join which handles traversal, but normalizeKey now validates expect(() => b.write('../../../etc/passwd', 'pwned')).toThrow('. or ..'); }); +}); + +// ============================================================================ +// delete() and append() tests (Issue #1003) +// ============================================================================ + +describe('WorktreeBackend delete/append', () => { + const squadDir = () => join(TMP, '.squad'); + beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); mkdirSync(squadDir(), { recursive: true }); }); + afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); + + it('delete removes an existing file and returns true', () => { + const b = new WorktreeBackend(squadDir()); + b.write('decisions.md', '# Decisions'); + expect(b.delete('decisions.md')).toBe(true); + expect(b.exists('decisions.md')).toBe(false); + }); + + it('delete returns false for non-existent file', () => { + const b = new WorktreeBackend(squadDir()); + expect(b.delete('nonexistent.md')).toBe(false); + }); + + it('append creates file if it does not exist', () => { + const b = new WorktreeBackend(squadDir()); + b.append('log.md', 'line 1\n'); + expect(b.read('log.md')).toBe('line 1\n'); + }); + + it('append adds to existing content', () => { + const b = new WorktreeBackend(squadDir()); + b.write('log.md', 'line 1\n'); + b.append('log.md', 'line 2\n'); + expect(b.read('log.md')).toBe('line 1\nline 2\n'); + }); +}); + +describe('GitNotesBackend delete/append', () => { + beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); initRepo(); }); + afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); + + it('delete removes a key from the blob', { timeout: 15_000 }, () => { + const b = new GitNotesBackend(TMP); + b.write('team.md', '# Team'); + b.write('routing.md', '# Routing'); + expect(b.delete('team.md')).toBe(true); + expect(b.exists('team.md')).toBe(false); + expect(b.read('routing.md')).toBe('# Routing'); + }); + + it('delete returns false for non-existent key', { timeout: 10_000 }, () => { + const b = new GitNotesBackend(TMP); + expect(b.delete('nonexistent.md')).toBe(false); + }); + + it('append creates entry if it does not exist', { timeout: 10_000 }, () => { + const b = new GitNotesBackend(TMP); + b.append('log.md', 'entry 1\n'); + expect(b.read('log.md')).toBe('entry 1\n'); + }); + + it('append concatenates to existing entry', { timeout: 15_000 }, () => { + const b = new GitNotesBackend(TMP); + b.write('log.md', 'entry 1\n'); + b.append('log.md', 'entry 2\n'); + expect(b.read('log.md')).toBe('entry 1\nentry 2\n'); + }); +}); + +describe('OrphanBranchBackend delete/append', () => { + beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); initRepo(); }); + afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); + + it('delete removes a file from the orphan branch', { timeout: 15_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.write('team.md', '# Team'); + b.write('routing.md', '# Routing'); + expect(b.delete('team.md')).toBe(true); + expect(b.exists('team.md')).toBe(false); + expect(b.read('routing.md')).toBe('# Routing'); + }); + + it('delete returns false for non-existent file', () => { + const b = new OrphanBranchBackend(TMP); + expect(b.delete('nonexistent.md')).toBe(false); + }); + + it('append creates file if it does not exist', { timeout: 15_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.append('log.md', 'entry 1\n'); + expect(b.read('log.md')).toBe('entry 1\n'); + }); + + it('append concatenates to existing file', { timeout: 15_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.write('log.md', 'entry 1\n'); + b.append('log.md', 'entry 2\n'); + expect(b.read('log.md')).toBe('entry 1\nentry 2\n'); + }); + + it('delete does not disturb working tree', { timeout: 15_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.write('s.json', '{}'); + const before = readFileSync(join(TMP, 'README.md'), 'utf-8'); + b.delete('s.json'); + expect(readFileSync(join(TMP, 'README.md'), 'utf-8')).toBe(before); + expect(git('status --porcelain')).toBe(''); + }); + + it('delete last file in directory prunes the empty directory', { timeout: 15_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.write('agents/data.md', '# Data'); + b.write('team.md', '# Team'); + expect(b.list('')).toContain('agents'); + b.delete('agents/data.md'); + expect(b.exists('agents/data.md')).toBe(false); + // The agents directory should be pruned since it's now empty + expect(b.list('')).not.toContain('agents'); + // Other files should still be intact + expect(b.read('team.md')).toBe('# Team'); + }); + + it('delete in nested directory prunes empty parents', { timeout: 30_000 }, () => { + const b = new OrphanBranchBackend(TMP); + b.write('a/b/c.md', 'deep'); + b.write('top.md', 'root'); + expect(b.list('')).toContain('a'); + b.delete('a/b/c.md'); + expect(b.exists('a/b/c.md')).toBe(false); + // Both 'a' and 'a/b' should be pruned + expect(b.list('')).not.toContain('a'); + expect(b.read('top.md')).toBe('root'); + }); +}); + +describe('resolveSquadState()', () => { + const squadDir = () => join(TMP, '.squad'); + beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); initRepo(); mkdirSync(squadDir(), { recursive: true }); }); + afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); + + it('returns null when no squad dir exists', () => { + rmSync(squadDir(), { recursive: true, force: true }); + expect(resolveSquadState(TMP)).toBeNull(); + }); + + it('returns context with local backend by default', () => { + writeFileSync(join(squadDir(), 'team.md'), '# Team'); + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.' })); + const ctx = resolveSquadState(TMP); + expect(ctx).not.toBeNull(); + expect(ctx!.backend.name).toBe('local'); + expect(ctx!.paths.projectDir).toBe(squadDir()); + }); + + it('respects stateBackend in config.json (git-notes migrates to two-layer)', () => { + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'git-notes' })); + const ctx = resolveSquadState(TMP); + expect(ctx).not.toBeNull(); + expect(ctx!.backend.name).toBe('two-layer'); + }); + + it('CLI override wins over config', () => { + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'git-notes' })); + const ctx = resolveSquadState(TMP, 'orphan'); + expect(ctx).not.toBeNull(); + expect(ctx!.backend.name).toBe('orphan'); + }); + + it('repoRoot uses git rev-parse --show-toplevel, not path.resolve parent', () => { + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.' })); + const ctx = resolveSquadState(TMP); + expect(ctx).not.toBeNull(); + // repoRoot should match the actual git toplevel, which is TMP + const expected = execSync('git rev-parse --show-toplevel', { cwd: TMP, encoding: 'utf-8' }).trim(); + // Normalize path separators for cross-platform comparison + expect(ctx!.repoRoot.replace(/\\/g, '/')).toBe(expected.replace(/\\/g, '/')); + }); + + it('returns FSStorageProvider for local backend', () => { + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.' })); + const ctx = resolveSquadState(TMP); + expect(ctx).not.toBeNull(); + expect(ctx!.storage).toBeDefined(); + // Local backend should use FSStorageProvider, not the adapter + expect(ctx!.storage.constructor.name).toBe('FSStorageProvider'); + }); + + it('returns StateBackendStorageAdapter for two-layer backend (via git-notes migration)', () => { + writeFileSync(join(squadDir(), 'config.json'), JSON.stringify({ version: 1, teamRoot: '.', stateBackend: 'git-notes' })); + const ctx = resolveSquadState(TMP); + expect(ctx).not.toBeNull(); + expect(ctx!.storage.constructor.name).toBe('StateBackendStorageAdapter'); + }); +}); + +// ============================================================================ +// StateBackendStorageAdapter tests +// ============================================================================ + +describe('StateBackendStorageAdapter', () => { + const squadDir = () => join(TMP, '.squad'); + beforeEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); initRepo(); mkdirSync(squadDir(), { recursive: true }); }); + afterEach(() => { if (existsSync(TMP)) rmSync(TMP, { recursive: true, force: true }); }); + + it('readSync/writeSync/existsSync round-trip via git-notes', () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + expect(adapter.existsSync('team.md')).toBe(false); + adapter.writeSync('team.md', '# Team'); + expect(adapter.existsSync('team.md')).toBe(true); + expect(adapter.readSync('team.md')).toBe('# Team'); + }); + + it('listSync returns backend entries', () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + adapter.writeSync('agents/data.md', '# Data'); + adapter.writeSync('agents/picard.md', '# Picard'); + const entries = adapter.listSync('agents'); + expect(entries).toContain('data.md'); + expect(entries).toContain('picard.md'); + }); + + it('appendSync via adapter', () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + adapter.writeSync('log.md', 'line 1\n'); + adapter.appendSync('log.md', 'line 2\n'); + expect(adapter.readSync('log.md')).toBe('line 1\nline 2\n'); + }); + + it('toRelative strips absolute squad dir prefix', () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + // Write via absolute path, read via relative — should work + const absPath = join(squadDir(), 'decisions.md'); + adapter.writeSync(absPath, '# Decisions'); + expect(adapter.readSync('decisions.md')).toBe('# Decisions'); + }); + + it('deleteSync removes entries', () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + adapter.writeSync('temp.md', 'data'); + expect(adapter.existsSync('temp.md')).toBe(true); + adapter.deleteSync('temp.md'); + expect(adapter.existsSync('temp.md')).toBe(false); + }); + + it('async read/write round-trip', async () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + await adapter.write('async.md', '# Async'); + expect(await adapter.read('async.md')).toBe('# Async'); + expect(await adapter.exists('async.md')).toBe(true); + }); + + it('stat returns size and isDirectory false for files', async () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + adapter.writeSync('s.md', 'hello'); + const st = await adapter.stat('s.md'); + expect(st).toBeDefined(); + expect(st!.size).toBe(5); + expect(st!.isDirectory).toBe(false); + }); + + it('stat returns undefined for non-existent path', async () => { + const backend = new GitNotesBackend(TMP); + const adapter = new StateBackendStorageAdapter(backend, squadDir()); + expect(await adapter.stat('nope.md')).toBeUndefined(); + }); }); \ No newline at end of file