feat(renderer): collections stage 3 in the renderer (RFC-0042 step 3)

.changeset/collections-stage3-renderer.md

@@ -0,0 +1,10 @@
+---
+"@cloudhal/astro-renderer": minor
+"@cloudhal/core-ts": minor
+"@cloudhal/cloudflare": minor
+"@cloudhal/agent-runtime": minor
+"@cloudhal/mcp-server": minor
+"@cloudhal/cli": minor
+---
+
+Collections stage 3 in the renderer (RFC-0042 step 3, ADR-0044). A collection's `expiry` gives every entry a derived gate (`min(availability.until, field date + grace)`; a date ends at the next day's start in the site's zone), applied by both published sources at render time: `unlist` keeps the entry's page but leaves `entries()`, `allEntries()`, term filters and archives, related posts, feeds, the sitemap, `llms.txt` and search (search rows' `gateUntil`); `hide` answers 404 like an expired `availability`. The gate index adds every derived instant and every value instant of date/datetime list fields, so the edge cache expires there without a purge; previews ignore derived gates and say "Ended on …". New theme APIs: `GraphSource.queryEntries` and `queryEntries(source, q)` (`./queries`) list a collection by its list fields (`where`, `orderBy`, `sort`, `filter`, `limit`, `offset`, `includeExpired`) with one evaluator on live and snapshot sources; `contentListEntries`; `GraphSource.routableEntries` (entries with a public URL, unlisted included); `withStructuredData` and `entryStructuredData` (`./seo`) replace the generic `BlogPosting` with the collection's JSON-LD preset when the entry has every required value. `ContentList.astro` honours `where`/`orderBy`, also for theme copies that pass a source's list. `entryHead` is unchanged, so old themes keep today's output; the blog starter calls `withStructuredData` and `contentListEntries`. The site Worker's `GET /__cloudhal/features` answers `{ "gates": 2 }`, and writers (store, API, MCP, bundle import) refuse a new or changed `expiry` with `409 gates_unsupported` until it does (`GateSupport.enforced(siteId, version)`, `EXPIRY_GATES_VERSION`). A live entry whose published change adds a derived gate or moves it earlier is never low risk in `auto` mode (`expiringEntries`, auto-check code `expires`, for publishes and releases). Sites without the new settings render as before.

packages/agent-runtime/src/modes.ts

@@ -68,7 +68,12 @@ export type AutoCheckCode =
   /** RFC-0008 §10.7 (D9): the release would run after the key's window (elevation end or 24 h). */
   | "outside-window"
   /** RFC-0019 §2.4: an article gains a correction, an editorial statement a person signs off. */
-  | "correction";
+  | "correction"
+  /**
+   * RFC-0042 §3.5: a live article gains a derived gate (its collection's expiry) or an earlier one,
+   * which removes content at a set time, like `schedule_unpublish`.
+   */
+  | "expires";
 
 /** The `auto`-mode check behind a publish (stored on audit rows and approvals, ADR-0015). */
 export interface AutoCheck {
@@ -92,6 +97,7 @@ const AUTO_CHECK_TEXT: Record<AutoCheckCode, (c: AutoCheck) => string> = {
   "outside-window": () =>
     "it would run after your autonomy ends (the elevation's end, or 24 hours from now)",
   correction: () => "it adds a correction to a published article",
+  expires: () => "it makes a published article expire sooner (its collection's expiry rule)",
 };
 
 /** English wording of a check, for agents and audit reasons. */
@@ -109,18 +115,24 @@ export function describeAutoCheck(check: AutoCheck): string {
  * not count toward the limit. Any other asset change is `media` and waits for a person.
  *
  * Corrections (RFC-0019 §2.4): an entry in `corrections` (it adds a correction to its live
- * version; the store decides) makes the publish wait for a person.
+ * version; the store decides) makes the publish wait for a person. So does an entry in `expiries`
+ * (RFC-0042 §3.5: it adds or advances a live entry's derived gate; core-ts `expiringEntries`).
  */
 export function checkAutoPublish(
   diff: {
     added: PendingChange[];
     changed: { after: PendingChange }[];
     removed: PendingChange[];
   },
-  options: { ridingAssets?: ReadonlySet<string>; corrections?: ReadonlySet<string> } = {},
+  options: {
+    ridingAssets?: ReadonlySet<string>;
+    corrections?: ReadonlySet<string>;
+    expiries?: ReadonlySet<string>;
+  } = {},
 ): AutoCheck {
   const riding = options.ridingAssets ?? new Set<string>();
   const corrects = options.corrections ?? new Set<string>();
+  const expires = options.expiries ?? new Set<string>();
   const rides = (n: PendingChange) => n.kind === "asset" && riding.has(n.id);
   const added = diff.added.filter((n) => !rides(n));
   const touched = [...added, ...diff.changed.map((c) => c.after)];
@@ -129,21 +141,18 @@ export function checkAutoPublish(
     changes: touched.length + diff.removed.length,
     limit: AUTO_PUBLISH_LIMIT,
   };
-  const media = touched.some((n) => n.kind === "asset");
-  const code: AutoCheckCode =
-    diff.removed.length > 0
-      ? "removes-content"
-      : touched.length === 0
-        ? "nothing"
-        : touched.some((n) => n.kind !== "contentEntry" && n.kind !== "asset")
-          ? "structure"
-          : media
-            ? "media"
-            : touched.some((n) => corrects.has(n.id))
-              ? "correction"
-              : touched.length > AUTO_PUBLISH_LIMIT
-                ? "too-many"
-                : "low-risk";
+  const any = (test: (n: PendingChange) => boolean) => touched.some(test);
+  // The first rule that holds decides, in this order; none: low risk.
+  const rules: [AutoCheckCode, () => boolean][] = [
+    ["removes-content", () => diff.removed.length > 0],
+    ["nothing", () => touched.length === 0],
+    ["structure", () => any((n) => n.kind !== "contentEntry" && n.kind !== "asset")],
+    ["media", () => any((n) => n.kind === "asset")],
+    ["correction", () => any((n) => corrects.has(n.id))],
+    ["expires", () => any((n) => expires.has(n.id))],
+    ["too-many", () => touched.length > AUTO_PUBLISH_LIMIT],
+  ];
+  const code: AutoCheckCode = rules.find(([, holds]) => holds())?.[0] ?? "low-risk";
   return { ...base, code, lowRisk: code === "low-risk" };
 }
 
@@ -235,6 +244,8 @@ export interface ReleaseItemRisk {
   deps?: readonly string[];
   /** RFC-0019 §2.4: the target adds a correction to the live version. */
   corrects?: boolean;
+  /** RFC-0042 §3.5: the target adds or advances the live version's derived gate. */
+  expires?: boolean;
 }
 
 /**
@@ -262,6 +273,7 @@ export function checkAutoRelease(items: readonly ReleaseItemRisk[]): AutoCheck {
           .map((i) => i.nodeId),
       ),
       corrections: new Set(set.filter((i) => i.corrects).map((i) => i.nodeId)),
+      expiries: new Set(set.filter((i) => i.expires).map((i) => i.nodeId)),
     },
   );
   return { ...check, rule: "low-risk-release" };

packages/agent-runtime/test/modes.test.ts

@@ -182,6 +182,13 @@ describe("agent modes", () => {
     );
     expect(corrected).toMatchObject({ code: "correction", lowRisk: false, changes: 2 });
     expect(describeAutoCheck(corrected)).toBe("it adds a correction to a published article");
+    // RFC-0042 §3.5: a live article that would expire sooner is classed like schedule_unpublish
+    const expiring = checkAutoPublish(
+      { added: [], changed: [{ after: entry("e2") }], removed: [] },
+      { expiries: new Set(["e2"]) },
+    );
+    expect(expiring).toMatchObject({ code: "expires", lowRisk: false, changes: 1 });
+    expect(describeAutoCheck(expiring)).toContain("expire sooner");
   });
 
   it("media rides along with its article only when the store says so (RFC-0004 §6.3)", () => {

packages/agent-runtime/test/schedules.test.ts

@@ -234,6 +234,11 @@ describe("auto-mode releases (RFC-0008 §10.7, D9)", () => {
         { ...entry("entry_b00001", true), corrects: true },
       ]),
     ).toMatchObject({ lowRisk: false, code: "correction" });
+    // RFC-0042 §3.5: an article whose derived gate the release adds or moves earlier waits too
+    expect(checkAutoRelease([{ ...entry("entry_b00001", true), expires: true }])).toMatchObject({
+      lowRisk: false,
+      code: "expires",
+    });
   });
 
   it("the window ends at the elevation's end or after 24 hours, whichever is first", () => {

packages/api/test/listings-api.test.ts

@@ -1,15 +1,19 @@
-import { SiteGraphStore } from "@cloudhal/core-ts";
+import { GATES_ENFORCED, SiteGraphStore, seedStarterSite } from "@cloudhal/core-ts";
 import { MemoryBlobStore } from "@cloudhal/storage";
 import { D1Driver } from "@cloudhal/storage/d1";
 import { BetterSqlite3Driver } from "@cloudhal/storage/sqlite";
 import { createMiniflareD1, createTableReset } from "@cloudhal/storage/testing";
+import { describe, expect, it } from "vitest";
+import { createApi } from "../src/app.ts";
 import { describeListingsApi } from "../src/testing-listings.ts";
 
 describeListingsApi("better-sqlite3", {
   async create() {
     const store = new SiteGraphStore({
       sql: await BetterSqlite3Driver.open(":memory:"),
       blobs: new MemoryBlobStore(),
+      // RFC-0042 §3.6: `expiry` needs a renderer with derived gates (one process on Node).
+      gates: GATES_ENFORCED,
     });
     await store.init();
     return store;
@@ -22,11 +26,41 @@ describeListingsApi("Cloudflare D1 (Miniflare)", {
   async create() {
     d1 ??= await createMiniflareD1();
     const sql = new D1Driver(d1.db);
-    const store = new SiteGraphStore({ sql, blobs: new MemoryBlobStore() });
+    const store = new SiteGraphStore({ sql, blobs: new MemoryBlobStore(), gates: GATES_ENFORCED });
     await reset(sql, () => store.init());
     return store;
   },
   async dispose() {
     await d1?.dispose();
   },
 });
+
+describe("API: collection expiry and the site Worker's gates (RFC-0042 §3.6)", () => {
+  it("answers 409 gates_unsupported until the renderer reports gates 2", async () => {
+    const store = new SiteGraphStore({
+      sql: await BetterSqlite3Driver.open(":memory:"),
+      blobs: new MemoryBlobStore(),
+      // an RFC-0008 site Worker: availability yes, derived gates no
+      gates: { enforced: (_siteId, version) => (version ?? 1) <= 1 },
+    });
+    await store.init();
+    const site = await seedStarterSite(store, { name: "Old Worker", slug: "old-worker" });
+    const app = createApi({
+      store,
+      resolveActor: async () => ({ kind: "human", id: "user_01HZX8K3Q9CD", name: "Halil" }),
+    });
+    const put = (settings: unknown) =>
+      app.request(`/api/sites/${site.id}/collections/events`, {
+        method: "PUT",
+        headers: { "content-type": "application/json" },
+        body: JSON.stringify({ settings, rev: site.rev }),
+      });
+    const fields = [
+      { key: "endsAt", type: "datetime", labels: { en: { label: "Ends" } }, localized: false },
+    ];
+    const refused = await put({ fields, expiry: { field: "endsAt", action: "unlist" } });
+    expect(refused.status).toBe(409);
+    expect(((await refused.json()) as { error?: string }).error).toBe("gates_unsupported");
+    expect((await put({ fields })).status).toBe(200);
+  });
+});

packages/astro-renderer/README.md

@@ -45,6 +45,21 @@ Additive; `entry.fields` still means exactly what is stored, and themes that nev
 
 `EntryFieldsOptions`: `locale` (default: the entry's), `media` (the page's `MediaContext`), `timeZone`, `words` (`{ yes, no }`), `sizes`. Each `RenderedField` has `key`, `label`, `help?`, `empty`, `text` (escaped plain text), and per type `html` (rich text through the body's markdown pipeline), `href` (`safeHref`, dropped when unsafe), `image` (`imageAttributes` plus `assetId`, alt in the page's language) or `datetime`, plus `def`, `value` and `fromOriginal` from `resolveFields`. Theme guide: [`docs/guides/fields.md`](../../docs/guides/fields.md).
 
+### Collections stage 3 (RFC-0042, added in 0.6.0)
+Additive. `entryHead`, `entries()`, `allEntries()` and every existing export keep their meaning; the only differences an old theme copy renders come from settings a site opts into (`theme-v0.5.0` in the compat test): entries past a collection's `expiry` leave lists, feeds, the sitemap and search, and `contentList` sections with `where`/`orderBy` list by field (`ContentList.astro` ships here and queries the source a theme's list came from).
+
+| Export | Signature |
+|---|---|
+| `GraphSource.queryEntries?` | `(q: FieldEntryQuery) => Promise<FieldEntryResult>` on `LiveSource` and `SnapshotSource`: `{ collection, locale?, where?, orderBy?, sort?, filter?, limit? (1–100, default 20), offset?, includeExpired? }` → `{ entries, total }`; `now` in conditions is the source's instant |
+| `queryEntries` (`./queries`, `.`) | `(source: GraphSource, q: FieldEntryQuery) => Promise<FieldEntryResult>`: any source (falls back to `entries()` on sources without `queryEntries`); an invalid query matches nothing and is logged |
+| `contentListEntries` (`./queries`, `.`) | `(source, sections, locale?) => Promise<Record<string, ContentEntry[]>>`: the `entries` map `Sections` reads (by collection; by section id for `filter` and for `where`/`orderBy`) |
+| `sectionQuery`, `listsByField`, `evaluateEntryQuery` (`./queries`) | the query a section runs; whether it lists by field; the pure evaluator both sources share |
+| `GraphSource.routableEntries?` | `(locale?) => Promise<ContentEntry[]>`: every entry with a public URL, unlisted ones included (routes, static builds, alternates); `allEntries` stays the listed ones |
+| `withStructuredData` (`./seo`, `.`) | `(head: HeadMeta, source, entry, { origin, media? }) => Promise<HeadMeta>`: call after `entryHead`; replaces the `BlogPosting` with the collection's preset (`Event`, `JobPosting`, `Product`, `Recipe`) when the entry has every required value, keeps the `BreadcrumbList`, returns `head` unchanged otherwise |
+| `entryStructuredData` (`./seo`, `.`) | `(source, entry, { origin, media? }) => Promise<{ doc?, issues }>`: the preset document and why there is none |
+
+Derived gates: `unlist` keeps an entry's page (200), comments and menu links but leaves `entries()`, `allEntries()`, `entriesByFilter`/`entriesByTerm` (archives, related), feeds, the sitemap and search; `hide` is a 404 like an expired `availability`. Both sources apply them, the gate index caps the edge cache at every derived instant and at every value instant of date/datetime list fields (for `now` conditions), and previews ignore them with an "Ended on …" notice. The site Worker's `GET /__cloudhal/features` answers `{ "gates": 2 }`; writers refuse `expiry` until it does.
+
 ## Site analytics (RFC-0006)
 
 The renderer carries the site side of Cloudhal's cookie-less analytics (`@cloudhal/analytics`):