feat(renderer): status pages before Astro (RFC-0043 PR 1)

.changeset/status-pages.md

@@ -0,0 +1,6 @@
+---
+"@cloudhal/astro-renderer": patch
+"@cloudhal/cli": patch
+---
+
+Status pages (RFC-0043 §1, #117). A site that cannot show its pages answers with one small, translated, self-contained page instead of plain text: `not-published` (503, `Retry-After: 3600`, `noindex`) before the first publish, `preview-expired` (410) for a preview link that does not verify, `unavailable` (503) for a Worker that serves no site, and `error` (500, with the `cf-ray` as a reference) when a render fails. The Site Worker answers them before Astro runs, every one `no-store` with its own CSP; `GET /__cloudhal/health` answers `{ "ok": true, "state": "published" | "not-published" }` in the admin's `/api/health` shape. The page is in the request's locale (host or path prefix, then the site default, then English) for every language Studio speaks, right to left where needed, through the same `pnpm i18n:export/import` flow as Studio's catalogs. A theme may restyle it with an optional `statusPage` export in `src/theme.ts` (`@cloudhal/astro-renderer/status-page`), which falls back to the built-in page on a throw, a slow answer or an oversized document and never runs for `error`; `cloudhal dev` draws each state at `/__cloudhal/status/<state>`. The blog starter's routes use `statusPage`/`statusPlain` as their own fallback. **Behaviour change:** an invalid or expired preview link now answers 410 instead of silently showing the published page.

AGENTS.md

@@ -30,7 +30,7 @@ pnpm i18n:export / pnpm i18n:import [--locale xx]  # translations as nested ICU
 - `packages/storage` — `SqlDriver`/`BlobStore` + adapters (`sqlite`, `d1`, `do-sqlite`, `fs`, `r2`; `SqlBlobStore`, `LayeredBlobStore`, `FetcherBlobStore` at the root) + migrations (`migrations/*.ts`, plain SQL). Conformance suites in `src/testing.ts`.
 - `packages/core-ts` — `SiteGraphStore` (CAS on `rev`, snapshots, diff, publish, rollback). Behavioural suite in `src/testing.ts` runs on sqlite and Miniflare D1. `./local` opens the offline store; `seedStarterSite` seeds a demo. `./surfaces` is RFC-0020's generative UI: the catalog (primitives + block types), eval-free surface checks, bindings, repairs, JSON Patch (`patchPage`), `catalogPrompt`; `./interop` converts A2UI v0.9 and json-render to and from surfaces; `./design` is RFC-0021's DESIGN.md lint and guidance. `./fields` is RFC-0041's typed custom fields (definition and value checks, `resolveFields`, `formatFieldValue`, the agents' JSON Schema, the conversion table); `src/fields-operations/` runs rename/retype/localize/remove. `search.ts` is RFC-0019's FTS5 site search (index written with the publish batch; articles, pages, terms and authors, RFC-0029); `search-provider.ts` is RFC-0029's `SearchProvider` (`fts5`, `memory`, `withFallback`), set with `SiteGraphStoreOptions.search`; `search-ai.ts` is the experimental Cloudflare AI Search provider (`hybridProvider`, `searchProviderFromEnv`, `cloudhal.json` `search`, docs/guides/search.md).
 - `packages/schedules` — RFC-0008: time model (only `temporal-polyfill/implementation`, `now` injected, never `Temporal.Now`), RRULE subset, release model (`rl1:`), automation/run state, taint and budgets, `runDue`; `./sql` is `SqlScheduleStore`, `./engine` the Temporal-free part core-ts and the site Worker use. Worker-safe; conformance suites in `src/testing.ts` on sqlite and Miniflare D1.
-- `packages/astro-renderer` — `GraphSource` (live/snapshot), `cloudhal()` integration (virtual config + dev route-cache watcher), `.astro` section components (`Surface.astro` for RFC-0020 surfaces; `./surfaces` renders them, `./kit` has the table of contents and reading time; `./fields` and `EntryFields.astro` render typed field values, RFC-0041). Routes live in starters, not here.
+- `packages/astro-renderer` — `GraphSource` (live/snapshot), `cloudhal()` integration (virtual config + dev route-cache watcher), `.astro` section components (`Surface.astro` for RFC-0020 surfaces; `./surfaces` renders them, `./kit` has the table of contents and reading time; `./fields` and `EntryFields.astro` render typed field values, RFC-0041). `./status-page` is RFC-0043's status pages (`statusPage(state, input) → Response`, served by the Site Worker entry before Astro and by `cloudhal dev` at `/__cloudhal/status/<state>`; the optional `statusPage` theme hook; the catalog in `src/status-strings/` where only `en.ts` is hand-edited and the other languages come from `pnpm i18n:import`). Routes live in starters, not here.
 - `packages/cli` — the `cloudhal` CLI (`init`, `dev`, `studio`, `mcp`, `deploy`, `push`, `pull`, `export`, `import`, …; every command is in `src/cli/help.ts`, which `cloudhal --help` prints and the docs site's CLI reference is generated from); TS bin run by Node 24 type stripping. `src/cli.ts` parses flags and dispatches through `COMMANDS`, built from `src/cli/commands-{local,remote,content,ops,site}.ts` (a new command is one handler plus one line in `src/cli/help.ts`). `src/archive/` is RFC-0039's `.cloudhal` container (tar, gzip, manifest, limits), read and written by `src/commands/export.ts` and `import*.ts` (`archive-bundle.ts`, `archive-reassemble.ts`); the starter demo archive `test/fixtures/site-archives/starter-blog.cloudhal` is written by `pnpm demo:archive`, and `pnpm check` fails when it is stale. E2E test spawns the bin (Astro disables dev middleware under `VITEST`). Cloudflare tests run both Workers on workerd via wrangler's `createTestHarness`; test files run serially (shared starter caches).
 - `packages/cloudflare` — the admin Worker (`createAdminWorker`); `./playground` is RFC-0036's playground Worker (`createPlaygroundWorker`, the `PlaygroundSite` Durable Object, signed session cookies). Generated entries must import `@cloudhal/cloudflare/workerd-prelude` first (ADR-0012).
 - `packages/agent-runtime` / `packages/mcp-server` — policy, skills, audit; MCP tools and transports (ADR-0011). `@cloudhal/mcp-server/render` is RFC-0021's `render_page` core: a small DevTools client, the in-page checks and limits; the CLI drives local Chrome with it (`render-chrome.ts`), the admin Worker Browser Rendering (`browser-render.ts`).

packages/astro-renderer/package.json

@@ -62,7 +62,11 @@
     "./locales": "./src/locales.ts",
     "./locale-middleware": "./src/locale-middleware.ts",
     "./locale-feeds": "./src/locale-feeds.ts",
-    "./not-found": "./src/not-found.ts"
+    "./not-found": "./src/not-found.ts",
+    "./status-page": "./src/status-page.ts",
+    "./status-hook": "./src/status-hook.ts",
+    "./status-serve": "./src/status-serve.ts",
+    "./status-middleware": "./src/status-middleware.ts"
   },
   "files": [
     "dist",
@@ -296,6 +300,22 @@
       "./not-found": {
         "types": "./dist/src/not-found.d.ts",
         "default": "./dist/src/not-found.js"
+      },
+      "./status-page": {
+        "types": "./dist/src/status-page.d.ts",
+        "default": "./dist/src/status-page.js"
+      },
+      "./status-hook": {
+        "types": "./dist/src/status-hook.d.ts",
+        "default": "./dist/src/status-hook.js"
+      },
+      "./status-serve": {
+        "types": "./dist/src/status-serve.d.ts",
+        "default": "./dist/src/status-serve.js"
+      },
+      "./status-middleware": {
+        "types": "./dist/src/status-middleware.d.ts",
+        "default": "./dist/src/status-middleware.js"
       }
     }
   }

packages/astro-renderer/src/astro-env.d.ts

@@ -19,3 +19,12 @@ declare module "virtual:cloudhal/comments-client" {
   const client: import("./comments-client.ts").CommentsClient | null;
   export default client;
 }
+
+declare module "virtual:cloudhal/theme-hooks" {
+  /**
+   * RFC-0043 §1.7: loads the project's `src/theme.ts` lazily (an empty module when the project
+   * has none), so a status page can ask for the theme's `statusPage` hook without evaluating
+   * the theme on the normal path.
+   */
+  export const loadTheme: () => Promise<Record<string, unknown>>;
+}

packages/astro-renderer/src/cloudflare-entry.ts

@@ -32,17 +32,26 @@
  *   render the section themselves (`loadComments`, `<Comments />`) and set their own CSP when an
  *   open form uses Turnstile; `withSecurityHeaders` keeps it. All of it is off without a usable
  *   `CLOUDHAL_COMMENTS_KEY` (or when the build has no comments module).
+ * - Status pages (RFC-0043 §1): before Astro, the entry decides the site's state — `not-published`
+ *   before the first publish, `preview-expired` for a preview token that does not verify,
+ *   `unavailable` for a Worker that serves no site — and answers page paths with the renderer's
+ *   own page (`status-page.ts`: 503/410, `Retry-After`, `no-store`, its own CSP), restyled by the
+ *   theme's optional `statusPage` hook. A failure anywhere becomes the `error` page (500) with the
+ *   `cf-ray` as its reference. `GET /__cloudhal/health` answers `{ ok, state }` for monitors.
  */
 
 import { cache } from "cloudflare:workers";
 import config from "virtual:cloudhal/config";
+import { loadTheme } from "virtual:cloudhal/theme-hooks";
 import { handle } from "@astrojs/cloudflare/handler";
 import { type WaeDataset, WaeWriter } from "@cloudhal/analytics/wae";
 import {
   analyticsBackend,
   CACHE_TAG_PREFIX,
   resetSource,
+  type SiteStateResult,
   serveMedia,
+  siteState,
   withRequestSite,
 } from "@cloudhal/astro-renderer/runtime";
 import { SqlCommentStore } from "@cloudhal/comments/sql";
@@ -63,6 +72,9 @@ import { hostRedirect, type RedirectEnv } from "./host-redirect.ts";
 import { PREVIEW_PARAM } from "./resolver.ts";
 import { SearchLimiter, searchCost } from "./search-limit.ts";
 import { SiteDispatch, unknownHost, withNoindex } from "./site-dispatch.ts";
+import { renderStatus, themeStatusHook } from "./status-hook.ts";
+import { type SiteState, statusPage } from "./status-page.ts";
+import { isStatusPagePath, statusInput } from "./status-serve.ts";
 
 interface Env extends RedirectEnv {
   CLOUDHAL_SITE_ID?: string;
@@ -142,6 +154,67 @@ const pendingWrites = new PendingWrites();
 const analyticsBySite = new Map<string, SiteAnalytics>();
 const commentsBySite = new Map<string, SiteComments>();
 let dispatch: SiteDispatch | undefined;
+/** RFC-0043 §1.7: the theme's `statusPage` hook, loaded the first time a status page is drawn. */
+const statusHook = themeStatusHook(loadTheme, (m, e) => console.error(m, e));
+
+/** RFC-0043 §1.3: the `error` page's reference — Cloudflare's ray id, else a random one. */
+function referenceId(request: Request): string {
+  return (
+    request.headers.get("cf-ray") ??
+    Array.from(crypto.getRandomValues(new Uint8Array(8)), (b) =>
+      b.toString(16).padStart(2, "0"),
+    ).join("")
+  );
+}
+
+/** The `error` page (RFC-0043 §1.2): never the theme's hook, nothing read from the store. */
+function errorPage(request: Request, error: unknown): Response {
+  const rayId = referenceId(request);
+  console.error(`cloudhal: request failed (reference ${rayId})`, error);
+  return statusPage("error", { locale: "en", now: Date.now(), rayId }, { method: request.method });
+}
+
+/**
+ * A 5xx HTML answer from Astro becomes the `error` page unless the theme marked it as its own
+ * (`x-cloudhal-error-page: theme`, a theme's `500.astro`).
+ */
+function withErrorPage(response: Response, request: Request): Response {
+  if (
+    response.status < 500 ||
+    !(response.headers.get("content-type") ?? "").includes("text/html") ||
+    response.headers.get("x-cloudhal-error-page") === "theme"
+  )
+    return response;
+  return errorPage(request, new Error(`Astro answered ${response.status}`));
+}
+
+/**
+ * RFC-0043 §1.9: `{ ok, state }`, the admin's `/api/health` shape plus the site's state, so one
+ * monitor rule reads both. Public, never cached, never indexed; reads only the memoised pointer.
+ */
+async function health(request: Request): Promise<Response> {
+  const headers = {
+    "cache-control": "no-store",
+    "cloudflare-cdn-cache-control": "no-store",
+    "x-robots-tag": "noindex",
+  };
+  if (request.method !== "GET" && request.method !== "HEAD")
+    return Response.json(
+      { error: "method_not_allowed" },
+      { status: 405, headers: { ...headers, allow: "GET" } },
+    );
+  try {
+    const state = await siteState(request);
+    if (state.kind === "unavailable") return Response.json({ ok: false }, { status: 503, headers });
+    return Response.json(
+      { ok: true, state: state.kind === "not-published" ? "not-published" : "published" },
+      { headers },
+    );
+  } catch (error) {
+    console.error("cloudhal: health check could not read the store", error);
+    return Response.json({ ok: false }, { status: 503, headers });
+  }
+}
 
 /** One per isolate: which site a request is for (RFC-0034 §4). */
 function siteDispatch(env: Env, siteIdFromBuild: string | undefined): SiteDispatch {
@@ -504,8 +577,11 @@ async function serveSite(
   const media = siteId ? await serveMedia(request, { siteId }) : undefined;
   const rendered =
     media ??
-    withSecurityHeaders(
-      cacheOnlySuccess((await handle(request, env as never, ctx)) as unknown as Response),
+    withErrorPage(
+      withSecurityHeaders(
+        cacheOnlySuccess((await handle(request, env as never, ctx)) as unknown as Response),
+      ),
+      request,
     );
   // Server-side render events (§5.4), off the response path: published HTML 200s (cache
   // misses), redirects, 404, 410, 451 and 5xx; never previews or HEAD.
@@ -529,6 +605,44 @@ async function serveSite(
     : stripped;
 }
 
+/**
+ * RFC-0043 §2.5: the status page for a request the site cannot serve, drawn with the site data
+ * the state allows (§1.6) and the theme's hook; counted as a render event like any 5xx (§1.8).
+ */
+async function statusResponse(
+  call: { request: Request; env: Env; ctx: ExecutionContext },
+  state: Exclude<SiteStateResult, { kind: "serve" }>,
+  siteId: string,
+): Promise<Response> {
+  const { request, env, ctx } = call;
+  const url = new URL(request.url);
+  const kind: SiteState = state.kind;
+  const sites =
+    state.kind === "unavailable"
+      ? {}
+      : {
+          live: state.site,
+          published: state.kind === "preview-expired" ? state.published : undefined,
+        };
+  const started = Date.now();
+  const response = await renderStatus(
+    kind,
+    statusInput(kind, sites, { host: url.host, pathname: url.pathname }, started),
+    {
+      hook: await statusHook(),
+      method: request.method,
+      csp: SITE_CSP,
+      onFailure: (m, e) => console.error(m, e),
+    },
+  );
+  const a = state.kind !== "unavailable" ? await siteAnalytics(env, siteId) : undefined;
+  if (a?.available)
+    ctx.waitUntil(
+      a.render(request, response, Date.now() - started).then(() => pendingWrites.settled()),
+    );
+  return response;
+}
+
 export function createSiteWorker(options: { siteId?: string } = {}) {
   const searchLimiter = new SearchLimiter();
   async function route(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
@@ -541,15 +655,30 @@ export function createSiteWorker(options: { siteId?: string } = {}) {
     // runs (Astro's render included) then sees that site.
     const choice = await siteDispatch(env, options.siteId).choose(request);
     if (choice.kind === "unknown") return unknownHost();
-    const siteId = choice.siteId;
-    const run = () => serveSite({ request, env, ctx }, siteId, searchLimiter);
-    const response = siteId ? await withRequestSite(siteId, run) : await run();
+    // Without `CLOUDHAL_SITE_ID` a single-site Worker serves the site it was built for, as
+    // Astro's `getSource` always did; `unavailable` is for a site row that is gone (§2.5).
+    const siteId = choice.siteId ?? config.siteId;
+    const call = { request, env, ctx };
+    const run = async () => {
+      if (url.pathname === "/__cloudhal/health") return health(request);
+      const page =
+        (request.method === "GET" || request.method === "HEAD") && isStatusPagePath(url.pathname);
+      const state = page ? await siteState(request) : undefined;
+      if (state && state.kind !== "serve") return statusResponse(call, state, siteId);
+      return serveSite(call, siteId, searchLimiter);
+    };
+    const response = await withRequestSite(siteId, run);
     return choice.noindex ? withNoindex(response) : response;
   }
   return {
     async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
       // #62: the edge cache is shared by every host this Worker answers on, unless told to vary.
-      return varyByHost(await route(request, env, ctx));
+      try {
+        return varyByHost(await route(request, env, ctx));
+      } catch (error) {
+        // RFC-0043 §1.8: whatever failed, the visitor gets a page and a reference to quote.
+        return varyByHost(errorPage(request, error));
+      }
     },
   };
 }

packages/astro-renderer/src/integration.ts

@@ -1,3 +1,5 @@
+import { existsSync } from "node:fs";
+import path from "node:path";
 import type { AstroIntegration } from "astro";
 import type { AnalyticsClient } from "./analytics-client.ts";
 import type { CommentsClient } from "./comments-client.ts";
@@ -61,34 +63,61 @@ const RESOLVED_CLIENT_ID = `\0${CLIENT_ID}`;
 /** Dev only: the built comments module and worker, served from memory by the comments middleware. */
 const COMMENTS_CLIENT_ID = "virtual:cloudhal/comments-client";
 const RESOLVED_COMMENTS_CLIENT_ID = `\0${COMMENTS_CLIENT_ID}`;
+/**
+ * RFC-0043 §1.7: a lazy loader of the project's `src/theme.ts` for the status pages' theme hook
+ * (root-relative, so Vite resolves it in dev and in the Site Worker build), or of an empty module
+ * when the project has no theme module. Evaluated only when a status page is drawn.
+ */
+const THEME_HOOKS_ID = "virtual:cloudhal/theme-hooks";
+const RESOLVED_THEME_HOOKS_ID = `\0${THEME_HOOKS_ID}`;
+const THEME_MODULE = "src/theme.ts";
 
 /** The virtual modules: the project config, and in dev the built analytics and comments clients. */
 function configPlugin(modules: { config: unknown; client: unknown; comments: unknown }) {
   const ids: Record<string, string> = {
     [VIRTUAL_ID]: RESOLVED_ID,
     [CLIENT_ID]: RESOLVED_CLIENT_ID,
     [COMMENTS_CLIENT_ID]: RESOLVED_COMMENTS_CLIENT_ID,
+    [THEME_HOOKS_ID]: RESOLVED_THEME_HOOKS_ID,
   };
   const bodies: Record<string, unknown> = {
     [RESOLVED_ID]: modules.config,
     [RESOLVED_CLIENT_ID]: modules.client,
     [RESOLVED_COMMENTS_CLIENT_ID]: modules.comments,
   };
+  let root = process.cwd();
   return {
     name: "cloudhal-config",
+    configResolved: (config: { root: string }) => {
+      root = config.root;
+    },
     resolveId: (id: string) => (Object.hasOwn(ids, id) ? ids[id] : undefined),
-    load: (id: string) =>
-      Object.hasOwn(bodies, id) ? `export default ${JSON.stringify(bodies[id])};` : undefined,
+    load: (id: string) => {
+      if (id === RESOLVED_THEME_HOOKS_ID)
+        return existsSync(path.join(root, THEME_MODULE))
+          ? `export const loadTheme = () => import("/${THEME_MODULE}");`
+          : "export const loadTheme = () => Promise.resolve({});";
+      return Object.hasOwn(bodies, id)
+        ? `export default ${JSON.stringify(bodies[id])};`
+        : undefined;
+    },
   };
 }
 
 /**
  * What only `cloudhal dev` runs as middleware (the Site Worker's entry and builds do these jobs):
  * the analytics beacon (RFC-0006 §5.1); the click-to-load video player (RFC-0015 §4) and the
- * interaction runtime (RFC-0020 §4), which builds write as files; and the Site Worker's CSP sent
- * report-only, so the console shows what a deploy would block.
+ * interaction runtime (RFC-0020 §4), which builds write as files; the Site Worker's CSP sent
+ * report-only, so the console shows what a deploy would block; and the status pages' preview
+ * route (RFC-0043 §1.8).
  */
-const DEV_MIDDLEWARE = ["analytics-middleware", "embed-middleware", "csp-middleware"].map((m) => ({
+const DEV_MIDDLEWARE = [
+  "analytics-middleware",
+  "embed-middleware",
+  "csp-middleware",
+  // RFC-0043 §1.8: the status pages drawn on demand at `/__cloudhal/status/<state>`.
+  "status-middleware",
+].map((m) => ({
   entrypoint: `@cloudhal/astro-renderer/${m}`,
   order: "pre" as const,
 }));