feat: collections stage 3 store and API (RFC-0042 step 2)

.changeset/collections-stage3-store.md

@@ -0,0 +1,6 @@
+---
+"@cloudhal/core-ts": minor
+"@cloudhal/api": minor
+---
+
+Collections stage 3 in the store and the API (RFC-0042 step 2, ADR-0044). `updateSite` checks the new collection settings when they change: `listFields` and `expiry.field` name fields of the collection (`unknown_field`), the expiry field is a date or datetime field (`invalid_expiry`), and a structured-data preset passes `checkStructuredData` (`structured_data_invalid`; warnings are not refused). Unlisting, dropping or retyping a list field, or dropping a select option, that `contentList` sections still filter or sort by is refused as `list_field_in_use` naming the pages, kept true by a guard on the site CAS. Page writes check changed `contentList` `where`/`orderBy` against the collection's list fields (`invalid_query`, with `sections/<i>/…` paths). Field operations follow: `rename_field` renames the field in `listFields`, `expiry`, structured-data sources and sections' queries in phase A's batch (and back on revert); `remove_field` refuses while any of them names the field; `change_field_type` refuses when the new type would break them. New `queryContentEntries` (and `applyFieldQuery`) filters and orders a collection's entries by list fields with the shared evaluator over resolved values. `GET /api/sites/:id/entries` takes `where` (JSON), `orderBy` (JSON), `limit` (1–100) and `cursor`; with `limit` it answers `{ entries, total, nextCursor? }`, without it the plain array as before. New error codes `invalid_query`, `structured_data_invalid`, `invalid_expiry` (422) and `list_field_in_use` (409). Sites without the new settings behave as before.

packages/api/README.md

@@ -12,7 +12,7 @@ The core routes are below; the feature routes (taxonomy, menus, media, comments,
 | `GET/PUT /api/sites[/:id]` | sites (PUT = full document, CAS on `rev`) |
 | `GET /api/sites/:id/pending` | what a publish would carry, as counts (Studio's badge beside Publish) |
 | `/api/sites/:id/pages[/:pageId]` | pages (POST creates and appends to `Site.pageIds` atomically; DELETE needs `?rev=`) |
-| `/api/sites/:id/entries[/:entryId]` | content entries (`?collection=` filter); `fields` is checked against the collection's typed fields (RFC-0041), and `null` removes a key there |
+| `/api/sites/:id/entries[/:entryId]` | content entries (`?collection=`, `locale`, `status`, `filter`); `fields` is checked against the collection's typed fields (RFC-0041), and `null` removes a key there. RFC-0042 §4: with `collection`, `where` (a JSON list of up to 4 `{field, op, value?/values?}`; ops by type, `value: "now"` for date/datetime fields) and `orderBy` (JSON `{field, direction: "asc"\|"desc"}`) filter and sort by the collection's `listFields`, e.g. `?collection=events&where=[{"field":"startsAt","op":"gte","value":"now"}]&orderBy={"field":"startsAt","direction":"asc"}` (URL-encoded). `limit` (1–100) pages the answer as `{entries, total, nextCursor?}`, continued with `cursor=<nextCursor>`; without `limit` the answer stays the plain array |
 | `/api/sites/:id/collections[/:key]` | the collection registry; `PUT /:key {settings, rev}` (`?dryRun=1`) declares a collection or changes its settings, typed `fields` included; `DELETE /:key?rev=` |
 | `/api/sites/:id/collections/:key/fields/operations` | RFC-0041 §6 field operations (`site:settings` and `content:write`): `POST {op, …, dryRun?}` with `op` one of `rename_field {from, to}`, `change_field_type {key, type, options?, multiple?, onInvalid?: "clear"}`, `set_field_localized {key, localized}`, `remove_field {key}` → `{operationId, done, report, rewritten, skipped}` (a dry run writes nothing; `done: false` when the 2,000-entry / 20 s budget ran out, then `POST ?continue=<operationId>`); `GET /:operationId` → state and pending entries; `POST /:operationId/revert` (again while `done: false`). Only the draft graph changes |
 | `/api/sites/:id/snapshots` | create (`{label, publish}`), list, `:id/publish`, `:id/rollback`, `:a/diff/:b` |
@@ -37,6 +37,16 @@ Typed fields (RFC-0041) answer with every problem and its path in `details.issue
 | `field_values_unconvertible` | 422 | a type change would drop values the conversion table cannot convert, without `onInvalid: "clear"` (`details.report` is the dry run) |
 | `field_operation_incomplete` | 409 | reverting an operation that has not finished, or was already reverted |
 
+Collections stage 3 (RFC-0042) answers the same way (`details.issues`, slash paths):
+
+| Code | Status | When |
+|---|---|---|
+| `invalid_query` | 422 | an entry list's `where`/`orderBy`/`limit`/`cursor`, or a `contentList` section's (`sections/2/where/0/value`), names an unlisted field, an operator or value its type does not take, or breaks a bound |
+| `unknown_field` | 422 | `listFields` or `expiry.field` names no field of the collection |
+| `invalid_expiry` | 422 | `expiry.field` is not a date or datetime field |
+| `structured_data_invalid` | 422 | a `structuredData` preset that does not fit the type's table or the collection's fields (warnings are not refused) |
+| `list_field_in_use` | 409 | unlisting, dropping or retyping a list field (or dropping a select option) that `contentList` sections still use (`details.pages`); `remove_field` of a field named by `listFields`, `expiry` or `structuredData`; a type change that would break those sections. `rename_field` renames those references instead |
+
 ```ts
 import { createApi } from "@cloudhal/api";
 import { createAuth } from "@cloudhal/api/auth";

packages/api/src/entry-list.ts

@@ -0,0 +1,105 @@
+/**
+ * RFC-0042 §4: the entry list's field query in a query string. `where` is a JSON list of
+ * conditions and `orderBy` a JSON object (the contract's `FieldCondition` / `FieldOrder`), `limit`
+ * pages (1–100) and `cursor` continues. Without `limit` the answer stays today's unpaged array;
+ * with it, `{ entries, total, nextCursor? }`. Problems answer `422 invalid_query` with paths.
+ */
+import type { ContentEntry, FieldCondition, FieldOrder } from "@cloudhal/core-contracts";
+import {
+  type FieldEntryListQuery,
+  InvariantError,
+  queryContentEntries,
+  type SiteGraphStore,
+} from "@cloudhal/core-ts";
+
+export interface FieldListParams {
+  where?: string | undefined;
+  orderBy?: string | undefined;
+  limit?: string | undefined;
+  cursor?: string | undefined;
+}
+
+export interface EntryPage {
+  entries: ContentEntry[];
+  total: number;
+  nextCursor?: string;
+}
+
+const invalid = (path: string, message: string): never => {
+  throw new InvariantError("invalid_query", `${path}: ${message}`, {
+    issues: [{ path, code: "invalid_query", message }],
+  });
+};
+
+function json(raw: string, path: string): unknown {
+  try {
+    return JSON.parse(raw);
+  } catch {
+    return invalid(path, "not valid JSON");
+  }
+}
+
+/** An opaque cursor: the offset of the next page. */
+const encodeCursor = (offset: number) => btoa(`o:${offset}`).replace(/=+$/, "");
+function decodeCursor(raw: string): number {
+  let text = "";
+  try {
+    text = atob(raw);
+  } catch {
+    // falls through to the refusal below
+  }
+  const m = /^o:(\d{1,9})$/.exec(text);
+  return m ? Number(m[1]) : invalid("cursor", "not a cursor this list returned");
+}
+
+/** True when the request uses any of the field-query parameters. */
+export const usesFieldQuery = (p: FieldListParams): boolean =>
+  p.where !== undefined ||
+  p.orderBy !== undefined ||
+  p.limit !== undefined ||
+  p.cursor !== undefined;
+
+function parseParams(p: FieldListParams) {
+  const where = p.where === undefined ? undefined : json(p.where, "where");
+  if (where !== undefined && !Array.isArray(where)) invalid("where", "a JSON list of conditions");
+  const orderBy = p.orderBy === undefined ? undefined : json(p.orderBy, "orderBy");
+  if (
+    orderBy !== undefined &&
+    (typeof orderBy !== "object" || orderBy === null || Array.isArray(orderBy))
+  )
+    invalid("orderBy", 'a JSON object such as {"field":"startsAt","direction":"asc"}');
+  const limit =
+    p.limit === undefined ? undefined : /^\d{1,4}$/.test(p.limit) ? Number(p.limit) : Number.NaN;
+  if (p.cursor !== undefined && limit === undefined) invalid("cursor", "a cursor needs limit");
+  const offset = p.cursor === undefined ? undefined : decodeCursor(p.cursor);
+  return {
+    where: where as FieldCondition[] | undefined,
+    orderBy: orderBy as FieldOrder | undefined,
+    limit: limit === undefined ? undefined : Number.isNaN(limit) ? -1 : limit,
+    offset,
+  };
+}
+
+/** The entry list with a field query: an array without `limit`, a page with it. */
+export async function listEntriesByField(
+  store: SiteGraphStore,
+  siteId: string,
+  base: Omit<FieldEntryListQuery, "where" | "orderBy" | "limit" | "offset">,
+  params: FieldListParams,
+): Promise<ContentEntry[] | EntryPage> {
+  const q = parseParams(params);
+  const result = await queryContentEntries(store, siteId, {
+    ...base,
+    ...(q.where ? { where: q.where } : {}),
+    ...(q.orderBy ? { orderBy: q.orderBy } : {}),
+    ...(q.limit !== undefined ? { limit: q.limit } : {}),
+    ...(q.offset !== undefined ? { offset: q.offset } : {}),
+  });
+  if (q.limit === undefined) return result.entries;
+  const next = (q.offset ?? 0) + result.entries.length;
+  return {
+    entries: result.entries,
+    total: result.total,
+    ...(next < result.total ? { nextCursor: encodeCursor(next) } : {}),
+  };
+}

packages/api/src/errors.ts

@@ -85,6 +85,12 @@ export const INVARIANT_STATUS: Record<InvariantCode, 400 | 409 | 410 | 415 | 422
   operation_in_progress: 409,
   field_values_unconvertible: 422,
   field_operation_incomplete: 409,
+  // RFC-0042 (collections stage 3): a query or setting that does not fit the collection's fields
+  // is 422; dropping or retyping a list field sections still use collides with them, 409.
+  invalid_query: 422,
+  structured_data_invalid: 422,
+  invalid_expiry: 422,
+  list_field_in_use: 409,
   reserved_key: 400,
   immutable_field: 400,
   primary_not_assigned: 422,

packages/api/src/routes/entries.ts

@@ -8,6 +8,7 @@ import { patchBody } from "../body.ts";
 import type { Env } from "../context.ts";
 import { entryDraft } from "../drafts.ts";
 import { withoutNullFields } from "../entry-fields.ts";
+import { listEntriesByField, usesFieldQuery } from "../entry-list.ts";
 import { badRequest, notFound } from "../errors.ts";
 import { API_ACTOR, writerOf } from "../governance.ts";
 import { editsLiveCorrections, entryLimit, personOf } from "../own.ts";
@@ -29,15 +30,30 @@ export const entries: Hono<Env> = new Hono<Env>()
     const filter = c.req.query("filter") || undefined;
     const locale = c.req.query("locale") || undefined;
     const status = statusList(c.req.query("status"));
+    const base = {
+      ...(collection ? { collection } : {}),
+      ...(filter ? { filter } : {}),
+      ...(locale ? { locale } : {}),
+      ...(status ? { status } : {}),
+    };
+    // RFC-0042 §4: `where`/`orderBy` by list fields, `limit`/`cursor` paging (`422 invalid_query`).
+    const fieldQuery = {
+      where: c.req.query("where"),
+      orderBy: c.req.query("orderBy"),
+      limit: c.req.query("limit"),
+      cursor: c.req.query("cursor"),
+    };
+    if (usesFieldQuery(fieldQuery))
+      return c.json(
+        await listEntriesByField(
+          c.get("store"),
+          param(c, "siteId"),
+          { ...base, now: c.get("now")().getTime() },
+          fieldQuery,
+        ),
+      );
     // `filter` is the RFC-0002 §5 language; a FilterError maps to 400 with suggestions.
-    return c.json(
-      await c.get("store").listContentEntries(param(c, "siteId"), {
-        ...(collection ? { collection } : {}),
-        ...(filter ? { filter } : {}),
-        ...(locale ? { locale } : {}),
-        ...(status ? { status } : {}),
-      }),
-    );
+    return c.json(await c.get("store").listContentEntries(param(c, "siteId"), base));
   })
   .post("/", async (c) => {
     const siteId = param(c, "siteId");