diff --git a/docs/rfcs/0043-status-pages-and-maintenance.md b/docs/rfcs/0043-status-pages-and-maintenance.md new file mode 100644 index 00000000..570c7423 --- /dev/null +++ b/docs/rfcs/0043-status-pages-and-maintenance.md @@ -0,0 +1,1056 @@ +# RFC-0043: Site status pages and maintenance mode + +- Status: Accepted (2026-11-08, by Halil; the maintenance contract's ADR lands with rollout PR 3). + Open questions decided as recommended; see "Decisions (2026-11-08)". +- Author: Halil (Madraka), drafted with Claude · Date: 2026-10-23, finalized 2026-11-08 +- Related: + - #117 (status pages: the agreed direction this RFC turns into a design; it adds maintenance + mode at the owner's request). #143, #148 and #150 (the Docker self-host image: a static site + from `cloudhal build` behind Caddy, the admin's `/api/health`), RFC-0044 (remote MCP + authorization: the scope mapping of §2.9). + - RFC-0008 / ADR-0023 (§7.1 one clock per request, §7.4 the edge cache capped at the next + gate, §7.7 the `features` handshake and writer guard, §9 time model, §10.3 call classes), + RFC-0042 / ADR-0044 (`gates: 2`), RFC-0034 (multi-site serving; the `Vary: Host` correction + from #62), RFC-0036 / ADR-0038 (playground route guard), RFC-0033 / ADR-0035 (locale from + host and path prefix), ADR-0033 (RTL), ADR-0032 and ADR-0034 (ICU catalogs; only English + hand-edited), RFC-0012 / ADR-0026 (CSP), RFC-0007 §4.5 and §8.3 (settings outside the graph; + the shared purge queue and budget), RFC-0006 §3.3 and §5.4 (render events), ADR-0014 + (single-use approvals), ADR-0015 (agent modes), ADR-0018 (approver must be able to make the + call), RFC-0039 (site archives). + - Code read: `packages/astro-renderer/src/cloudflare-entry.ts` (purge, `features`, media + before Astro, preview link rewriting, `cacheOnlySuccess`, `varyByHost`), `runtime-workerd.ts` + (`getSource`, `edgeMaxAge`), `resolver.ts` (preview tokens; an invalid token falls back to + the published page today), `site-dispatch.ts` (`unknownHost`), `media-middleware.ts`, + `collection-feed-middleware.ts`, `integration.ts` (the dev middleware list, the virtual + modules), `packages/cloudflare/src/site-routes.ts` and `features.ts`, + `packages/agent-runtime/src/policy.ts`, `starters/blog/src/pages/[...path].astro:117` and + the feed routes, `packages/cli/src/commands/astro.ts` (`build` resolves published → latest → + fresh snapshot), `deploy/docker/bin/site-build` and `deploy/docker/Caddyfile`, + `docs/guides/self-host.md`, `scripts/i18n-exchange.ts`, + `cloudhal.com/scripts/post-deploy-check.mjs`. + - Prior art and specifications (checked 2026-10-23; "unverified" where no primary page could + be read): + - RFC 9110: [§15.6.4 503](https://www.rfc-editor.org/rfc/rfc9110.html#section-15.6.4) + ("temporary overload or scheduled maintenance", `Retry-After` MAY be sent); + [§10.2.3 Retry-After](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3) + (HTTP-date or delay-seconds; with a 503 it says how long the service is expected to be + unavailable); [§15.5.11 410](https://www.rfc-editor.org/rfc/rfc9110.html#section-15.5.11) + (likely permanent; **heuristically cacheable**); §15.1 (404 and 410 are heuristically + cacheable, 500 and 503 are not). + - Google Search Central: [Pausing your online business](https://developers.google.com/search/docs/crawling-indexing/pause-online-business) + (updated 2025-12-10): a full 503 shutdown only "for a very short period of time (a few days + at most)"; send `Retry-After`; **do not return 503 for robots.txt**; do not use 403/404/410 + or `noindex` for a pause; for longer closures serve an indexable 200 placeholder. + [HTTP and network errors](https://developers.google.com/search/docs/crawling-indexing/http-network-errors) + (updated 2026-02-04): 5xx slows crawling, content of 5xx responses is ignored, indexed URLs + that persistently return server errors are eventually dropped (no number of days given). + [robots.txt specification](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec) + (updated 2026-10-23): a 5xx robots.txt stops crawling of the whole site for the first 12 + hours. + - Cloudflare: [Workers Cache](https://developers.cloudflare.com/workers/cache/) (updated + 2026-10-12): keyed by path, entrypoint, `ctx.props` and Worker version, **not hostname**; + checked **before** the Worker runs; honours `Vary` (separate variants; `Vary: *` disables + caching), `Cache-Control` per RFC 9111, `private` bypasses; responses with `Set-Cookie` and + requests with `Authorization` bypass automatically; nothing on cookies in the key. Whether + the query string is in the key is not stated (assumed yes: preview links depend on it + today; unverified). The path-only key is not a theory: cloudhal.com's www redirect looped + in 2026-11 because a cached redirect for `/` was served on both hosts (#62's `Vary: Host` + is the fix; "vary by host" is designed, never assumed). + [Default cache behaviour](https://developers.cloudflare.com/cache/concepts/default-cache-behavior/): + without `Cache-Control`, 404/410 are cached 3 minutes and 5xx are not; `no-store`/`private` + prevent caching. [Custom Errors](https://developers.cloudflare.com/rules/custom-errors/) + (updated 2026-09-08): Custom Error Rules can replace Worker 4xx/5xx responses, but not on + the Free plan, per zone, and Error Pages never apply to 500/503. + [Static assets `_headers`](https://developers.cloudflare.com/workers/static-assets/headers/) + sets headers only, never a status code; `_redirects` allows 200, 301, 302, 303, 307 and + 308. + - Caddy: [`file_server`](https://caddyserver.com/docs/caddyfile/directives/file_server) takes + a `status` option (the code written when a file is served) and the + [`file` matcher](https://caddyserver.com/docs/caddyfile/matchers#file) matches on a file's + existence, so a static host can answer 503 for a marker the build writes. + - WordPress: [`wp_maintenance()`](https://developer.wordpress.org/reference/functions/wp_maintenance/) + answers 503 with `Retry-After: 600`; a `wp-content/maintenance.php` drop-in replaces the + page; [`wp_is_maintenance_mode()`](https://developer.wordpress.org/reference/functions/wp_is_maintenance_mode/) + reads a `.maintenance` file whose `$upgrading` timestamp expires after 10 minutes + (auto-off). Coming-soon plugins add role bypass and page allowlists (common practice; + unverified per plugin). + - Shopify [password page](https://help.shopify.com/en/manual/online-store/themes/password-page): + new stores start private; a shared password lets visitors in; a "message to your + visitors"; the theme can restyle the page; pages are not indexed while private. How staff + bypass is not on that page (unverified). + - Webflow [site password](https://help.webflow.com/hc/en-us/articles/33961298873235): a + site-wide password page the designer can style; uploaded **assets stay public** by URL + (from a secondary summary of the help page; unverified). + - Ghost private mode: a shared password, indexing off, the page is the theme's + `private.hbs` (third-party and forum sources only; unverified). + - Vercel's [maintenance-page template](https://vercel.com/templates/next.js/maintenance-page): + middleware reads an Edge Config flag on every request (secondary summary; unverified). + Netlify: no primary source found (unverified). + +## Summary + +A Cloudhal site answers every state in which it cannot show its pages — not published yet, in +maintenance, unavailable, failed, or behind an expired preview link — with one small, translated, +self-contained HTML page served by the renderer before Astro, with the right status code (503, +500 or 410), `Retry-After` and `no-store`. Old theme copies get it with no change; a theme may +restyle it through an optional `statusPage` export; a static build of an unpublished site writes +it as its whole output, so a self-hosted site never serves an empty directory. + +Maintenance mode is a per-site record outside the graph (it never rolls back with a snapshot), +switched on now or for a scheduled window by people with `site:settings` and by agents only with +a person's approval; it ends on its own at its end time. Signed-in people keep seeing the real +site through signed links that are never cached, `robots.txt` and media stay up, and a public +health endpoint with the admin's `{ "ok": true }` shape tells monitors, Docker and post-deploy +checks "maintenance" from "down". + +## Motivation + +- **Today:** + - A new site answers every page with a plain-text `503 This site has not been published yet.` + from the theme (`[...path].astro:117`, and the feeds, sitemap and `llms.txt` routes). It is + the first thing the owner sees on a fresh domain, from the Deploy button, + `npm create cloudhal` or try.cloudhal.com. + - A render failure shows the runtime's default error. An expired or forged preview link + silently shows the **published** page (`resolver.ts` falls back), so a reviewer may approve + what they think is the draft. + - There is no way to take a site down for a migration, a domain move, an incident or a launch. + The only lever is "unpublish everything", which loses the pointer and is visible in the + content history. + - A self-hosted site (#148) is `cloudhal build` output behind Caddy; before the first publish + the build renders the latest or a fresh snapshot rather than saying "not published". +- **Who:** every new install (first impression), every owner running a migration or a launch, + agents (which today cannot tell "the site is down" from "the site is in maintenance"), and the + cloudhal.com deploy pipeline, whose `post-deploy-check.mjs` rolls both Workers back unless + `https://cloudhal.com/` answers 200 — a maintenance 503 would trigger a rollback. +- **Why now:** the 0.6 line opened Cloudhal to outside installs (try.cloudhal.com, npm, the + Docker image). RFC-0008's render-time gates and RFC-0034's per-site dispatch give us the + pieces: an instant compared per request, a site chosen before Astro runs, and a purge path per + site. + +## Proposal + +### 1. Status pages (from #117) + +#### 1.1 The module + +```ts +// packages/astro-renderer/src/status-page.ts — pure, Worker-safe; imports only the catalog +export type SiteState = + | "not-published" | "maintenance" | "unavailable" | "error" | "preview-expired"; + +export interface StatusPageInput { + readonly site?: { readonly name: string; readonly logo?: string }; // §1.6 decides what is passed + readonly locale: string; // already chosen (§1.4) + readonly page?: "maintenance" | "coming-soon"; + // maintenance only, plain text + readonly message?: { readonly text: string; readonly locale: string }; + // maintenance end, for "back by …" + readonly until?: { readonly at: string; readonly timeZone: string }; + readonly theme?: StatusTheme; // §1.3: tokens of the published site, when allowed + readonly rayId?: string; // error only + readonly now: number; // injected clock (RFC-0008 §9.4) +} + +/** The published site's look, read from `Site.theme.config` (§1.3); every field optional. */ +export interface StatusTheme { + readonly brand?: string; // `#rrggbb` + readonly surface?: string; // `#rrggbb` + readonly text?: string; // `#rrggbb` + readonly font?: "sans" | "serif" | "mono"; +} + +/** The built-in page: status, headers and body. Never throws (bad input degrades to English). */ +export function statusPage(state: SiteState, input: StatusPageInput): Response; +/** The same document as a string, for theme hooks that only restyle it (§1.7). */ +export function statusPageHtml( + state: SiteState, + input: StatusPageInput, + options?: { css?: string }, +): string; +/** Machine endpoints in the same states (§2.3): short text or JSON, same status and headers. */ +export function statusPlain( + state: SiteState, + input: StatusPageInput, + as: "text" | "json", +): Response; +``` + +`maintenance` joins the closed set now (#117 deferred it). The set stays closed: a new state is +an RFC change, so themes can switch over it exhaustively. + +#### 1.2 States, codes and headers + +| State | When | Status | `Retry-After` | `x-robots-tag` | +|---|---|---|---|---| +| `not-published` | the site has no `publishedSnapshotId` and the request carries no valid preview | 503 | `3600` | `noindex` | +| `maintenance` | §2 is in effect and the request has no valid preview or bypass link | 503 | seconds to the end (≥ 60); `3600` when open-ended | **none** | +| `unavailable` | the Worker serves no site yet (`siteId` undefined), the site row is missing (deleted, RFC-0034), or the store cannot be read before rendering | 503 | `300` | none | +| `error` | an exception reached the entry's top-level catch, or Astro answered 5xx with no themed error page | 500 | — | none | +| `preview-expired` | the URL carries `cloudhal-preview` and the token is expired, forged, for another site, malformed, or previews are off | 410 | — | `noindex` | + +- **Every state:** `cache-control: no-store`, `cloudflare-cdn-cache-control: no-store`, + `content-type: text/html; charset=utf-8`, `x-cloudhal-state: `, + `x-content-type-options: nosniff`, `referrer-policy: no-referrer`, and `Vary: Host` (added by + `varyByHost` as for every response). Status responses do not pass through + `withSecurityHeaders`, which would overwrite the referrer policy. `HEAD` gets the same headers + without a body. +- **`no-store` is not optional, and it is what keeps hosts apart.** RFC 9110 makes 410 + heuristically cacheable and Cloudflare caches 404/410 for 3 minutes without explicit headers; a + cached 410 or 503 would outlive the publish or the end of maintenance that fixes it + (`cacheOnlySuccess` already enforces this for Astro responses; status pages set it themselves). + Workers Cache keys on the path, not the hostname (the www-redirect loop on cloudhal.com, + 2026-11), so a cached status page for `/` would be served on every host the Worker answers on: + `no-store` means there is nothing to key, `Vary: Host` is the second line, and the workerd + multi-site test asserts that two hosts of one Worker, one published and one not, never share a + status page. +- **`noindex` only where nothing should be kept** (an amendment of #117, which put `noindex` on + every state). Google's pause guidance says not to use `noindex` during a temporary closure, + and it ignores the content of 5xx responses anyway. A site that was never published has + nothing to keep; a preview URL must never be indexed. Maintenance, `unavailable` and `error` + are temporary and stay without it. +- **`Retry-After` uses delay-seconds**, never an HTTP-date: the response is not cached, so the + value is fresh, and clock skew between the edge and a crawler does not matter. A maintenance + end in the past or within a minute gives `60`. +- **`preview-expired` changes behaviour on purpose:** an invalid preview token no longer falls + back to the published page. The reviewer learns that the link is dead instead of reviewing the + wrong version. `?cloudhal-preview=exit` and the dev preview cookie keep their behaviour; media + URLs keep their short plain answers (§1.8), since nobody views an image as a page. + +#### 1.3 Design of the page + +- **One self-contained document, ≤ 8 KB:** inline `