docs: production operations — observability, state backend selection, troubleshooting runbook

docs/src/content/docs/reference/state-backend-selection.md

@@ -120,20 +120,11 @@ git fetch origin squad-state:squad-state
 
 ## Known bug: FSStorageProvider `rootDir` (#1555)
 
-> ⚠️ **Open bug [#1555](https://github.com/bradygaster/squad/issues/1555):** `FSStorageProvider` resolves state paths relative to the Node.js process working directory rather than the volume mount path when `rootDir` is not set in `config.json`. This causes state writes to go to the wrong path when the process `cwd` does not match `/app/.squad/`.
+> ⚠️ **Open bug [#1555](https://github.com/bradygaster/squad/issues/1555):** `FSStorageProvider` is constructed without a `rootDir` argument in `resolveSquadState()`. All relative state keys resolve against the Node.js process working directory (the repo root) rather than `.squad/`, silently creating a shadow state tree that diverges from the authoritative `.squad/` copy agents actually read.
 >
-> **Workaround:** Explicitly set `rootDir` in `.squad/config.json`:
+> **There is no configuration-level workaround.** Because `FSStorageProvider` is constructed with zero arguments regardless of any user configuration, setting `rootDir` in `config.json` has no effect — the constructor call ignores it. This must be fixed in the SDK itself ([#1555](https://github.com/bradygaster/squad/issues/1555)).
 >
-> ```json
-> {
->   "version": 1,
->   "teamRoot": ".",
->   "stateBackend": "local",
->   "rootDir": "/app/.squad"
-> }
-> ```
->
-> This ensures Squad resolves state paths against the correct directory regardless of the process working directory. Track [#1555](https://github.com/bradygaster/squad/issues/1555) for a fix.
+> **Until #1555 is resolved:** Run only a single Squad replica. Avoid deployment paths where the MCP server process `cwd` differs from the volume mount path. Track [#1555](https://github.com/bradygaster/squad/issues/1555) for the fix.
 
 ---
 

docs/src/content/docs/scenarios/production-observability.md

@@ -8,7 +8,7 @@ order: 26
 
 > ⚠️ **Experimental** — Squad is alpha software. APIs, commands, and behavior may change between releases.
 
-> 🔗 **Related:** [Aspire Dashboard](./aspire-dashboard) (local dev telemetry) · [Container Image — Env Var Contract](/squad/docs/reference/container-image/) · [#1144](https://github.com/bradygaster/squad/issues/1144) (embedded-host telemetry — tracked separately)
+> 🔗 **Related:** [Aspire Dashboard](./aspire-dashboard) (local dev telemetry) · [Production Troubleshooting Runbook](./production-troubleshooting) · [Container Image — Env Var Contract](/squad/docs/reference/container-image/) · [#1144](https://github.com/bradygaster/squad/issues/1144) (embedded-host telemetry — tracked separately)
 
 ---
 
@@ -205,8 +205,9 @@ Recommended alerts once traces are flowing to Azure Monitor or Grafana:
 
 | Signal | Recommended threshold | Alert type |
 |---|---|---|
-| `squad.agent.duration` p95 > 5 minutes | Warning | Metric alert |
-| `squad.issue.dispatch.errors` rate > 5/min | Critical | Log alert |
+| `squad.agent.duration` p95 > 5 minutes | Warning | Metric alert (histogram) |
+| `squad.agent.errors` rate > 5/min | Critical | Metric alert (counter) |
+| `squad.sessions.errors` rate > 5/min | Warning | Metric alert (counter) |
 | Pod OOM restarts > 2 in 10 min | Critical | Container Insights alert |
 | GitHub API 401 errors in structured logs | Critical | Log alert |
 | No spans received from service in 10 min | Warning | Availability alert |
@@ -217,7 +218,7 @@ Recommended alerts once traces are flowing to Azure Monitor or Grafana:
 
 When Kubernetes sends `SIGTERM`, Squad begins graceful shutdown. The default pod `terminationGracePeriodSeconds` is 30 seconds. If an agent is mid-task, in-flight work may be interrupted. Squad does not currently checkpoint in-progress tasks before shutdown.
 
-Mitigation: Set `terminationGracePeriodSeconds: 120` in your Deployment spec to give long-running agents time to finish. There is no guarantee all work completes — monitor `squad.agent.interrupted` spans in your telemetry backend to measure the impact.
+Mitigation: Set `terminationGracePeriodSeconds: 120` in your Deployment spec to give long-running agents time to finish. There is no guarantee all work completes — monitor `squad.coordinator.shutdown` spans and the `squad.agent.errors` counter in your telemetry backend to assess shutdown impact. Squad does not emit an interrupted-specific span; shutdown scope is visible only through these existing signals.
 
 ---
 

docs/src/content/docs/scenarios/production-troubleshooting.md

@@ -47,7 +47,7 @@ Is the state backend accessible?
 Root causes in order of frequency:
 1. **KEDA not scaling** — queue depth below threshold or scaler unreachable
 2. **Wrong or missing env vars** — `GITHUB_TOKEN` absent or wrong scope
-3. **State backend inaccessible** — PVC mount failure or `rootDir` mismatch (#1555)
+3. **State backend inaccessible** — PVC mount failure or `FSStorageProvider` path bug (#1555)
 4. **GitHub API 401** — expired token, missing scope, or org SSO not enabled
 5. **No qualifying labels** — issues exist but labels don't match routing rules
 6. **Image pull failure** — ACR credentials expired or image tag not found
@@ -154,21 +154,13 @@ If the PVC mount is missing, the pod is writing state to the container's ephemer
 
 ### FSStorageProvider rootDir bug (#1555)
 
-If `kubectl exec -n squad <pod> -- ls /app/.squad` shows the correct files but Squad cannot find them, the process `cwd` does not match the volume mount path.
+If Squad logs show state files at the repo root (e.g., `agents/`, `log/`, `decisions.md` appearing at `/app/` rather than `/app/.squad/`), this is [#1555](https://github.com/bradygaster/squad/issues/1555): `FSStorageProvider` is constructed without a `rootDir` argument, so all state keys resolve against the process `cwd` (the repo root) instead of `.squad/`.
 
-**Workaround:**
-
-```json
-// /app/.squad/config.json
-{
-  "version": 1,
-  "teamRoot": ".",
-  "stateBackend": "local",
-  "rootDir": "/app/.squad"
-}
-```
+> ⚠️ **There is no configuration-level workaround for #1555.** Setting `rootDir` in `config.json` has no effect — `FSStorageProvider` ignores constructor arguments in the affected code path. This must be fixed in the SDK itself.
+>
+> **Mitigation until #1555 is resolved:** Run only a single Squad replica. Avoid deployment shapes where the MCP server process `cwd` differs from the intended state path. Track [#1555](https://github.com/bradygaster/squad/issues/1555) for status.
 
-See [State Backend Selection — Known bug #1555](/squad/docs/reference/state-backend-selection/#known-bug-fsstorageprovider-rootdir-1555) for details.
+See [State Backend Selection — Known bug #1555](/squad/docs/reference/state-backend-selection/#known-bug-fsstorageprovider-rootdir-1555) for background.
 
 ### `squad-state` branch push rejected (two-layer / orphan)
 
@@ -319,5 +311,5 @@ Open an issue at [bradygaster/squad](https://github.com/bradygaster/squad/issues
 |---|---|
 | [#1144](https://github.com/bradygaster/squad/issues/1144) | Embedded host (copilot-sdk) telemetry not available in production — agents must use Squad CLI container mode |
 | [#1402](https://github.com/bradygaster/squad/issues/1402) | External state backend for multi-pod deployments — not yet GA |
-| [#1555](https://github.com/bradygaster/squad/issues/1555) | FSStorageProvider `rootDir` bug — workaround documented above |
+| [#1555](https://github.com/bradygaster/squad/issues/1555) | FSStorageProvider `rootDir` bug — no configuration-level workaround; run single replica and track issue for fix |
 | [#1577](https://github.com/bradygaster/squad/issues/1577) | HTTP health/readiness endpoints — not yet implemented |