From 3286378905da7d27afdd0ae892a6508df2ca3142 Mon Sep 17 00:00:00 2001 From: barelyworkingcode Date: Sat, 7 Nov 2026 07:56:25 -0800 Subject: [PATCH] Retire the devbox from doc text Trivial doc text change. --- ROADMAP.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index bd42f82..176bd16 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -29,7 +29,7 @@ For *why* a non-obvious architectural choice was made, check the comment at the | Priority | Item | Rough size | Notes | |----------|------|-----------|-------| -| Medium | Live box needs re-registration with `model_host` | small, manual | `build.sh` now registers with `--capability manifest --capability projects --capability model_host`, but the running "Relay LLM" service record on the devbox was registered before this unit and only holds `manifest,projects`. Until it is re-registered (unregister, then `./build.sh` again — a presence-gated action, see relay's `CLAUDE.md`), relay refuses `RegisterModelHost` and relayLLM exits 78 partway through startup. Do this deliberately, not as a side effect of an unrelated build. | +| Medium | Live box needs re-registration with `model_host` | small, manual | `build.sh` now registers with `--capability manifest --capability projects --capability model_host`, but the running "Relay LLM" service record on the devbox (retired; history) was registered before this unit and only holds `manifest,projects`. Until it is re-registered (unregister, then `./build.sh` again — a presence-gated action, see relay's `CLAUDE.md`), relay refuses `RegisterModelHost` and relayLLM exits 78 partway through startup. Do this deliberately, not as a side effect of an unrelated build. | ## Cross-repo (tracked here for visibility, owned elsewhere) @@ -65,7 +65,7 @@ For *why* a non-obvious architectural choice was made, check the comment at the **Follow-up (same day): `--http-port` lost its bearer-token requirement entirely.** Wiring `build.sh`'s service registration back up surfaced that `--http-port` (added for the status dashboard, see its own Closed entry below) required a bearer token while the sibling `--router-port` never has — an inconsistency nobody had asked for and the operator caught in review, not something flagged at build time. The token requirement, the `?token=` → `HttpOnly`/`SameSite=Strict` cookie bootstrap, and the non-loopback-without-TLS fail-closed startup guard were all removed for this listener; it's now served anonymously (`recoverMiddleware(mux)`, no `bearerAuth`), protected purely by which `--http-bind` addresses are actually bound — exactly `--router-port`'s existing posture, which is the whole point of this session's `--http-bind`/`--router-bind` multi-address work in the first place. `auth.go::bearerAuth` itself shrank to a single-carrier (`Authorization: Bearer`) check, since the cookie/query-param convenience it dropped existed solely to make a bearer-token-gated browser page usable, and the only listener left behind it (the Unix socket) is dialed exclusively by machine clients — relay's manifest-bridged dispatcher, the permission hook, direct socket clients — that were never using the cookie/query-param path to begin with. `validateHTTPListener` shrank to the ordinary cert/key pairing check; `--http-tls-cert`/`--http-tls-key` remain available as an opt-in transport-encryption knob with no forced-use guard, matching the router's TLS story. Net effect: `--http-bind 127.0.0.1,` with no TLS cert now starts cleanly, same as `--router-bind` always could. -- Diagnostic status dashboard: `GET /status` (embedded HTML/CSS/JS, no build step) polling a new `GET /api/status/detailed` — open connections, throughput, idle/quiet/stalled state, models being served, virtual-model candidate reachability, and OpenAI-endpoint upstream availability, alongside the existing budgets/terminals. The relay-router's proxy path had zero connection instrumentation before this; a new `ProxyMetrics` registry (`status_metrics.go`) wraps the client `http.ResponseWriter` outermost — so the existing virtual-failover `virtualResponseRecorder` nests inside it — meaning a failed-over attempt contributes zero bytes and one connection row survives however many candidates were tried, with `defer`-based deregistration surviving the `http.ErrAbortHandler` panic a mid-stream backend failure raises. A two-bucket sliding-window rate estimator avoids the "long-frozen stream still reports a healthy average" failure mode a naive total/elapsed rate would have. Stall detection is deliberately asymmetric: a proxy connection mid-SSE-stream with no bytes for 60s (or a streaming request with no response at all for 180s, above the managed-server admission timeout so a queued cold-model load doesn't misreport) is a real bug signal, but a WebSocket viewer or chat session sitting idle between turns is normal and never flagged — conflating the two would make the dashboard cry wolf on every quiet devbox. `GET /api/status` (relay's Service Inspector) is untouched. Designed by Opus (backend instrumentation architecture) and Fable (visual/UX — the stalled-connection alarm treatment: inset border, solid ticking badge, glyph-only pulse disabled under `prefers-reduced-motion`, sparkline color-split at the stall boundary, document-title/favicon change for background tabs) in parallel, reconciled into one schema, then implemented by two Sonnet passes (backend, then frontend against the live endpoint) with independent verification at each stage. One real bug found in review after both implementer agents reported done and their own live-browser checks passed: every section's empty-state text (`setEmpty()` in `status.js`) toggled visibility but never replaced the placeholder "Loading…" string, so the common quiet-devbox case — the state the design explicitly called out as most important to get right — rendered as permanently stuck loading spinners instead of "No open connections" / "No managed models configured" / etc. across all six sections. Anthropic passthrough (`api.anthropic.com`, both the `/v1/messages` byte-for-byte branch and the raw `/api/*` catch-all Claude Code's OAuth bootstrap uses) is instrumented too — initially deferred as severable at design time, but explicitly requested as in-scope: every proxied route, Anthropic or otherwise, now shows up as a connection. **Follow-up landed:** the page turned out to be unreachable from an actual browser — it lives on the main mux, which bound only a Unix socket, while the one TCP port relayLLM listens on (`--router-port`) is a different mux whose catch-all answered `/status` with `{"error":"missing or invalid model field"}`; relay's front door is a socket only Eve dials, and Eve proxies only the `/api/*` + `/ws` routes it knows about. Fixed with `--http-port`/`--http-bind` (+ TLS pair), an opt-in second listener serving the *same handler value* as the socket — same routes, same `bearerAuth` — fail-closed on a non-loopback bind without TLS, and a true no-op when unset. Browser auth came with it: `?token=` is exchanged for an `HttpOnly`/`SameSite=Strict` cookie and redirected away, since a navigation can't send a header and the page's own `fetch` only inherits cookies. Deployed on the devbox as `--http-port 8181`, with the relay service record's `-url` pointing at `http://localhost:8181/status?token=…` so the tray links to it. +- Diagnostic status dashboard: `GET /status` (embedded HTML/CSS/JS, no build step) polling a new `GET /api/status/detailed` — open connections, throughput, idle/quiet/stalled state, models being served, virtual-model candidate reachability, and OpenAI-endpoint upstream availability, alongside the existing budgets/terminals. The relay-router's proxy path had zero connection instrumentation before this; a new `ProxyMetrics` registry (`status_metrics.go`) wraps the client `http.ResponseWriter` outermost — so the existing virtual-failover `virtualResponseRecorder` nests inside it — meaning a failed-over attempt contributes zero bytes and one connection row survives however many candidates were tried, with `defer`-based deregistration surviving the `http.ErrAbortHandler` panic a mid-stream backend failure raises. A two-bucket sliding-window rate estimator avoids the "long-frozen stream still reports a healthy average" failure mode a naive total/elapsed rate would have. Stall detection is deliberately asymmetric: a proxy connection mid-SSE-stream with no bytes for 60s (or a streaming request with no response at all for 180s, above the managed-server admission timeout so a queued cold-model load doesn't misreport) is a real bug signal, but a WebSocket viewer or chat session sitting idle between turns is normal and never flagged — conflating the two would make the dashboard cry wolf on every quiet box. `GET /api/status` (relay's Service Inspector) is untouched. Designed by Opus (backend instrumentation architecture) and Fable (visual/UX — the stalled-connection alarm treatment: inset border, solid ticking badge, glyph-only pulse disabled under `prefers-reduced-motion`, sparkline color-split at the stall boundary, document-title/favicon change for background tabs) in parallel, reconciled into one schema, then implemented by two Sonnet passes (backend, then frontend against the live endpoint) with independent verification at each stage. One real bug found in review after both implementer agents reported done and their own live-browser checks passed: every section's empty-state text (`setEmpty()` in `status.js`) toggled visibility but never replaced the placeholder "Loading…" string, so the common quiet-box case — the state the design explicitly called out as most important to get right — rendered as permanently stuck loading spinners instead of "No open connections" / "No managed models configured" / etc. across all six sections. Anthropic passthrough (`api.anthropic.com`, both the `/v1/messages` byte-for-byte branch and the raw `/api/*` catch-all Claude Code's OAuth bootstrap uses) is instrumented too — initially deferred as severable at design time, but explicitly requested as in-scope: every proxied route, Anthropic or otherwise, now shows up as a connection. **Follow-up landed:** the page turned out to be unreachable from an actual browser — it lives on the main mux, which bound only a Unix socket, while the one TCP port relayLLM listens on (`--router-port`) is a different mux whose catch-all answered `/status` with `{"error":"missing or invalid model field"}`; relay's front door is a socket only Eve dials, and Eve proxies only the `/api/*` + `/ws` routes it knows about. Fixed with `--http-port`/`--http-bind` (+ TLS pair), an opt-in second listener serving the *same handler value* as the socket — same routes, same `bearerAuth` — fail-closed on a non-loopback bind without TLS, and a true no-op when unset. Browser auth came with it: `?token=` is exchanged for an `HttpOnly`/`SameSite=Strict` cookie and redirected away, since a navigation can't send a header and the page's own `fetch` only inherits cookies. Deployed on the devbox (retired; history) as `--http-port 8181`, with the relay service record's `-url` pointing at `http://localhost:8181/status?token=…` so the tray links to it. - Anthropic Messages API compatibility on the relay-router (`router.anthropic` in settings.json). Lets Claude Code point at relayLLM via `ANTHROPIC_BASE_URL`: real Claude models proxy through to `api.anthropic.com` byte-for-byte (headers, body, SSE — including the `/api/claude_cli/bootstrap` + `/api/hello` traffic Claude Code's own OAuth-mode UI depends on), and a config-driven `modelMap` redirects specific model ids to a local managed/virtual/endpoint model via a full Anthropic↔OpenAI translation layer (typed content blocks, tool_use/tool_result, streaming state machine with buffered tool-call blocks and a ping keepalive for Claude Code's idle-byte watchdog). Design input from reading the actual `claude` CLI binary found that Claude Code's model-list visibility for a subscription-OAuth login doesn't come from a discovery endpoint at all (it's explicitly skipped for that auth mode) — so no Anthropic-shaped `/v1/models` was built; visibility is a client-side `ANTHROPIC_CUSTOM_MODEL_OPTION` setting instead. One real bug caught by the test suite: `httputil.ReverseProxy`'s unconditional per-write `Flush()` call was reaching the real `http.ResponseWriter` even in non-streaming mode, implicitly sending a premature 200 before the aggregated response's real status was known. - Virtual LLM failover (`virtual-llms` in settings.json): a stable model name with ordered fallback targets, resolved by the relay-router. Shipped, then hardened after `vCode` returned `400 unknown model` in the field. Three defects: endpoint reachability was treated as a hard gate on a cache that is 15s stale by design, so a healthy endpoint stayed unroutable after recovering; `ProxyRegistry.probe` inherited the inbound request's context, so a client disconnect recorded a false "offline" and poisoned that cache; and failover happened only at selection time, leaving remaining targets untried when one died mid-request. Reachability is now a preference ordering — believed-usable targets first, the rest as last-resort attempts — with retry on any failure that lands before the first response byte, a 3s dial cap so a black-holing host cannot eat 30s per candidate, and 503-naming-what-was-tried instead of `unknown model` for a name that is static config. Virtual rows now always appear in `/v1/models` and inherit `input_modalities` + `meta.n_ctx` from their first candidate.