feat: field operations and translation freshness (RFC-0041 step 3)

.changeset/typed-fields-operations.md

@@ -0,0 +1,13 @@
+---
+"@cloudhal/core-ts": minor
+"@cloudhal/api": minor
+"@cloudhal/cli": minor
+---
+
+Typed custom fields, step 3 (RFC-0041 §6 / ADR-0043): field operations.
+
+- **`runFieldOperation` / `continueFieldOperation` / `revertFieldOperation` / `fieldOperationStatus`** in `@cloudhal/core-ts` run `rename_field`, `change_field_type`, `set_field_localized` and `remove_field`: a dry run (entries touched, values moved, converted, cleared, removed, unchanged and unconvertible, per locale, with samples), a `before …` snapshot whose id is the operation id, the new definition with its `migration` marker (phase A), entries rewritten in guarded batches of 200 that resume after an interruption (phase B), the marker dropped once nothing is pending (phase C) and an `after` snapshot. Only the draft graph changes. A type change follows a closed conversion table (`convertFieldValue`, exported from `@cloudhal/core-ts/fields`) and refuses values it cannot convert (`field_values_unconvertible`) unless `onInvalid: "clear"`. One operation per collection at a time (`operation_in_progress`). Reverting restores the earlier definition and the entries nobody edited since, and converts edited ones back.
+- **Readers stay consistent mid-run:** `resolveFields` converts values still in the old type, and a field being removed reads as gone; a person's entry write during an operation goes through the marker, so edits and the operation converge.
+- **Translations:** changing only non-localized field values of an original no longer marks its translations stale (their recorded source hash follows, in the same batch); operations keep fresh translations fresh.
+- **API:** `POST /api/sites/:id/collections/:key/fields/operations` (`dryRun`, `?continue=<id>`), `GET …/:operationId`, `POST …/:operationId/revert` (`site:settings` and `content:write`), and the same under `/api/admin` with the deployment token.
+- **CLI:** `cloudhal fields rename|retype|localize|remove|continue|revert [--dry-run] [--remote]`; the dry run also lists theme files that read the field directly.

packages/api/README.md

@@ -14,6 +14,7 @@ The core routes are below; the feature routes (taxonomy, menus, media, comments,
 | `/api/sites/:id/pages[/:pageId]` | pages (POST creates and appends to `Site.pageIds` atomically; DELETE needs `?rev=`) |
 | `/api/sites/:id/entries[/:entryId]` | content entries (`?collection=` filter); `fields` is checked against the collection's typed fields (RFC-0041), and `null` removes a key there |
 | `/api/sites/:id/collections[/:key]` | the collection registry; `PUT /:key {settings, rev}` (`?dryRun=1`) declares a collection or changes its settings, typed `fields` included; `DELETE /:key?rev=` |
+| `/api/sites/:id/collections/:key/fields/operations` | RFC-0041 §6 field operations (`site:settings` and `content:write`): `POST {op, …, dryRun?}` with `op` one of `rename_field {from, to}`, `change_field_type {key, type, options?, multiple?, onInvalid?: "clear"}`, `set_field_localized {key, localized}`, `remove_field {key}` → `{operationId, done, report, rewritten, skipped}` (a dry run writes nothing; `done: false` when the 2,000-entry / 20 s budget ran out, then `POST ?continue=<operationId>`); `GET /:operationId` → state and pending entries; `POST /:operationId/revert` (again while `done: false`). Only the draft graph changes |
 | `/api/sites/:id/snapshots` | create (`{label, publish}`), list, `:id/publish`, `:id/rollback`, `:a/diff/:b` |
 | `/api/agent-keys` | API keys for agents: `POST {name, scopes}` (key shown once), `GET`, `DELETE /:id` — signed-in humans only |
 | `/mcp` | the Cloudhal MCP server (ADR-0011); session cookie → human, `x-api-key` or `Authorization: Bearer <key>` → agent limited to the key's scopes (REST answers agent keys with a pointer to `/mcp`) |
@@ -32,6 +33,9 @@ Typed fields (RFC-0041) answer with every problem and its path in `details.issue
 | `field_key_reserved`, `field_key_duplicate`, `field_option_duplicate`, `field_bounds` | 400 | definitions that can never be saved as written |
 | `too_many_fields` | 422 | over 30 fields per collection, 200 per site or 50 options per select |
 | `field_change_needs_operation` | 409 | changing a field's type, localization or `multiple`, or dropping a key or option that entries still hold |
+| `operation_in_progress` | 409 | another field operation is still rewriting the collection (`details.operationId`) |
+| `field_values_unconvertible` | 422 | a type change would drop values the conversion table cannot convert, without `onInvalid: "clear"` (`details.report` is the dry run) |
+| `field_operation_incomplete` | 409 | reverting an operation that has not finished, or was already reverted |
 
 ```ts
 import { createApi } from "@cloudhal/api";

packages/api/src/errors.ts

@@ -81,6 +81,10 @@ export const INVARIANT_STATUS: Record<InvariantCode, 400 | 409 | 410 | 415 | 422
   field_option_duplicate: 400,
   field_bounds: 400,
   field_change_needs_operation: 409,
+  // RFC-0041 §6 field operations: one at a time per collection, values a type change would lose.
+  operation_in_progress: 409,
+  field_values_unconvertible: 422,
+  field_operation_incomplete: 409,
   reserved_key: 400,
   immutable_field: 400,
   primary_not_assigned: 422,

packages/api/src/field-operations.ts

@@ -0,0 +1,137 @@
+/**
+ * RFC-0041 §6, §8: field operations over REST. Studio's person holding `site:settings` (and
+ * `content:write`, which the operations also need because they rewrite entries) runs them
+ * directly; agents ask through MCP, where a person approves. The operator's CLI reaches the same
+ * handlers through `/api/admin` with the deployment token (`cloudhal fields … --remote`).
+ *
+ * - `POST …/collections/:key/fields/operations` ← `{ op, …, dryRun? }` → the result (`done: false`
+ *   when the budget ran out); `?continue=<operationId>` resumes one instead
+ * - `GET  …/collections/:key/fields/operations/:operationId` → its status
+ * - `POST …/collections/:key/fields/operations/:operationId/revert` → the revert result (call again
+ *   while `done: false`)
+ *
+ * Refusals are the store's codes: `operation_in_progress` (409), `field_values_unconvertible` (422,
+ * with the dry run's report in `details.report`), `field_operation_incomplete` (409), and the
+ * definition codes (`unknown_field`, `field_key_duplicate`, `field_key_reserved`, …).
+ */
+import { holds } from "@cloudhal/agent-runtime";
+import type { Actor, FieldOption } from "@cloudhal/core-contracts";
+import {
+  continueFieldOperation,
+  FIELD_OPERATIONS,
+  type FieldOperation,
+  fieldOperationStatus,
+  revertFieldOperation,
+  runFieldOperation,
+  type SiteGraphStore,
+} from "@cloudhal/core-ts";
+import { FIELD_TYPES, type FieldType } from "@cloudhal/core-ts/fields";
+import type { Context } from "hono";
+import { Hono } from "hono";
+import type { Env } from "./context.ts";
+import { badRequest, missingScope, notFound } from "./errors.ts";
+import { actorOf } from "./governance.ts";
+import { param } from "./params.ts";
+
+type Body = Record<string, unknown>;
+
+const str = (b: Body, name: string): string => {
+  const v = b[name];
+  if (typeof v !== "string" || !v) throw badRequest(`${name} is required`);
+  return v;
+};
+
+/** The request body as an operation on `collection`; 400 for a malformed one. */
+export function operationOf(collection: string, b: Body): FieldOperation {
+  const op = b.op;
+  if (typeof op !== "string" || !FIELD_OPERATIONS.includes(op as FieldOperation["op"]))
+    throw badRequest(`op must be one of ${FIELD_OPERATIONS.join(", ")}`);
+  if (op === "rename_field") return { op, collection, from: str(b, "from"), to: str(b, "to") };
+  if (op === "remove_field") return { op, collection, key: str(b, "key") };
+  if (op === "set_field_localized") {
+    if (typeof b.localized !== "boolean") throw badRequest("localized must be a boolean");
+    return { op, collection, key: str(b, "key"), localized: b.localized };
+  }
+  const type = str(b, "type");
+  if (!FIELD_TYPES.includes(type as FieldType))
+    throw badRequest(`type must be one of ${FIELD_TYPES.join(", ")}`);
+  if (b.onInvalid !== undefined && b.onInvalid !== "clear")
+    throw badRequest('onInvalid is "clear" or absent');
+  if (b.multiple !== undefined && typeof b.multiple !== "boolean")
+    throw badRequest("multiple must be a boolean");
+  if (b.options !== undefined && !Array.isArray(b.options))
+    throw badRequest("options must be a list of { value, labels }");
+  return {
+    op: "change_field_type",
+    collection,
+    key: str(b, "key"),
+    type: type as FieldType,
+    ...(b.options ? { options: b.options as FieldOption[] } : {}),
+    ...(b.multiple !== undefined ? { multiple: b.multiple as boolean } : {}),
+    ...(b.onInvalid === "clear" ? { onInvalid: "clear" as const } : {}),
+  };
+}
+
+/** The operation must belong to the collection in the path. */
+async function ownStatus(store: SiteGraphStore, siteId: string, key: string, id: string) {
+  const status = await fieldOperationStatus(store, siteId, id);
+  if (status.operation.collection !== key) throw notFound("not found: field operation");
+  return status;
+}
+
+/** The handlers, for the people's routes and the operator's alike. */
+export function fieldOperationHandlers(actor: (c: Context<Env>) => Actor) {
+  return new Hono<Env>()
+    .post("/:key/fields/operations", async (c) => {
+      const store = c.get("store");
+      const siteId = param(c, "siteId");
+      const key = param(c, "key");
+      const resume = c.req.query("continue");
+      if (resume) {
+        await ownStatus(store, siteId, key, resume);
+        return c.json(await continueFieldOperation(store, siteId, resume, { actor: actor(c) }));
+      }
+      const b = ((await c.req.json().catch(() => null)) ?? {}) as Body;
+      if (b.dryRun !== undefined && typeof b.dryRun !== "boolean")
+        throw badRequest("dryRun must be a boolean");
+      const op = operationOf(key, b);
+      return c.json(
+        await runFieldOperation(store, siteId, op, { actor: actor(c), dryRun: b.dryRun === true }),
+      );
+    })
+    .get("/:key/fields/operations/:operationId", async (c) =>
+      c.json(
+        await ownStatus(
+          c.get("store"),
+          param(c, "siteId"),
+          param(c, "key"),
+          param(c, "operationId"),
+        ),
+      ),
+    )
+    .post("/:key/fields/operations/:operationId/revert", async (c) => {
+      const store = c.get("store");
+      const siteId = param(c, "siteId");
+      const id = param(c, "operationId");
+      await ownStatus(store, siteId, param(c, "key"), id);
+      return c.json(await revertFieldOperation(store, siteId, id, { actor: actor(c) }));
+    });
+}
+
+/**
+ * The people's routes under `/api/sites/:siteId/collections`. `ROUTE_SCOPES` asks `site:settings`
+ * for the writes; the operations rewrite entries, so `content:write` is checked here too.
+ */
+export const fieldOperations: Hono<Env> = new Hono<Env>()
+  .use("/:key/fields/operations/*", async (c, next) => {
+    const principal = c.get("principal");
+    if (c.req.method === "POST" && principal && !holds(principal.scopes, "content:write"))
+      throw missingScope("content:write");
+    await next();
+  })
+  .use("/:key/fields/operations", async (c, next) => {
+    const principal = c.get("principal");
+    if (principal && !holds(principal.scopes, "content:write")) throw missingScope("content:write");
+    await next();
+  })
+  .route("/", fieldOperationHandlers(actorOf));

packages/api/src/playground.ts

@@ -114,6 +114,10 @@ export const PLAYGROUND_ALLOW: ReadonlySet<string> = new Set([
   `GET ${S}/collections`,
   `PUT ${S}/collections/:key`,
   `DELETE ${S}/collections/:key`,
+  // RFC-0041 §6: field operations rewrite the visitor's own draft entries.
+  `POST ${S}/collections/:key/fields/operations`,
+  `GET ${S}/collections/:key/fields/operations/:operationId`,
+  `POST ${S}/collections/:key/fields/operations/:operationId/revert`,
   `GET ${S}/pages`,
   `POST ${S}/pages`,
   `GET ${S}/pages/:pageId/preview`,

packages/api/src/routes/admin.ts

@@ -35,6 +35,9 @@
  *   run, then the summary (RFC-0020 open question 7, `cloudhal ai build-eval --remote`); in memory
  * - `POST /api/admin/sites/:siteId/search/reindex` → rebuilds the search index from the live
  *   snapshot (RFC-0019 §3, `cloudhal search reindex --remote`)
+ * - `POST /api/admin/sites/:siteId/collections/:key/fields/operations[?continue=<id>]`, `GET
+ *   …/operations/:id` and `POST …/operations/:id/revert`: field operations as the operator
+ *   (RFC-0041 §6, `cloudhal fields … --remote`; the same handlers as Studio's, `../field-operations.ts`)
  * - `GET  /api/admin/sites/:siteId/media-placeholders?all=1` → images without a placeholder, and
  *   `PUT …/media-placeholders/:assetId` ← {rev, lqip?, color?}: `cloudhal media placeholders
  *   --remote` draws them in a local Chrome (RFC-0028 §5)
@@ -52,12 +55,13 @@ import type { RenderDeps } from "@cloudhal/mcp-server/tools";
 import { Hono } from "hono";
 import type { AiTransport } from "../ai/client.ts";
 import type { Env } from "../context.ts";
+import { fieldOperationHandlers } from "../field-operations.ts";
 import { aiAdminRoutes } from "./admin/ai.ts";
 import { analyticsAdminRoutes } from "./admin/analytics.ts";
 import { commentsAdminRoutes } from "./admin/comments.ts";
 import { ownerAdminRoutes } from "./admin/owner.ts";
 import { schedulesAdminRoutes } from "./admin/schedules.ts";
-import { timingSafeEqual } from "./admin/shared.ts";
+import { OPERATOR_ACTOR, timingSafeEqual } from "./admin/shared.ts";
 import { siteOpsAdminRoutes } from "./admin/site-ops.ts";
 import { siteWorkersAdminRoutes } from "./admin/site-workers.ts";
 import { sitesAdminRoutes } from "./admin/sites.ts";
@@ -91,6 +95,10 @@ export function adminRoutes(
       .route("/", sitesAdminRoutes(deps))
       .route("/", siteWorkersAdminRoutes(deps))
       .route("/", siteOpsAdminRoutes())
+      .route(
+        "/sites/:siteId/collections",
+        fieldOperationHandlers(() => OPERATOR_ACTOR),
+      )
       .route("/", syncAdminRoutes(deps))
   );
 }

packages/api/src/routes/collections.ts

@@ -4,12 +4,14 @@
  * `PUT /:key { settings, rev }` (site:settings) declares a collection or replaces its settings
  * through the site update, so the store's overlap and permalink-redirect checks run (`rev` is the
  * site's: a stale one is 409); `?dryRun=1` runs them and writes nothing. `DELETE /:key?rev=`
- * (site:settings) undeclares a collection without entries.
+ * (site:settings) undeclares a collection without entries. Field operations (RFC-0041 §6) live
+ * under `/:key/fields/operations` (`../field-operations.ts`).
  */
 import type { CollectionSettings, Site } from "@cloudhal/core-contracts";
 import { Hono } from "hono";
 import type { Env } from "../context.ts";
 import { badRequest, notFound } from "../errors.ts";
+import { fieldOperations } from "../field-operations.ts";
 import { param } from "../params.ts";
 
 const KEY = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
@@ -70,4 +72,5 @@ export const collections: Hono<Env> = new Hono<Env>()
     if (!site.collections?.[key]) throw notFound("collection not declared");
     const next = withCollection({ ...site, rev: revOf(c.req.query("rev")) }, key, undefined);
     return c.json({ site: await store.updateSite(next) });
-  });
+  })
+  .route("/", fieldOperations);

packages/api/src/scopes.ts

@@ -66,6 +66,18 @@ export const ROUTE_SCOPES: readonly (readonly [Method, string, RouteRequirement]
   ["POST", "/api/admin/sites/:siteId/ai/judge-check", "deploy-token"],
   // RFC-0019 §3: `cloudhal search reindex --remote`
   ["POST", "/api/admin/sites/:siteId/search/reindex", "deploy-token"],
+  // RFC-0041 §6: `cloudhal fields rename|retype|localize|remove|revert --remote`
+  ["POST", "/api/admin/sites/:siteId/collections/:key/fields/operations", "deploy-token"],
+  [
+    "GET",
+    "/api/admin/sites/:siteId/collections/:key/fields/operations/:operationId",
+    "deploy-token",
+  ],
+  [
+    "POST",
+    "/api/admin/sites/:siteId/collections/:key/fields/operations/:operationId/revert",
+    "deploy-token",
+  ],
   // RFC-0028 §5: `cloudhal media placeholders --remote`
   ["GET", "/api/admin/sites/:siteId/media-placeholders", "deploy-token"],
   ["PUT", "/api/admin/sites/:siteId/media-placeholders/:assetId", "deploy-token"],
@@ -124,6 +136,10 @@ export const ROUTE_SCOPES: readonly (readonly [Method, string, RouteRequirement]
   ["GET", `${S}/collections`, "content:read"],
   ["PUT", `${S}/collections/:key`, "site:settings"],
   ["DELETE", `${S}/collections/:key`, "site:settings"],
+  // RFC-0041 §6: field operations (the handler also asks content:write; they rewrite entries)
+  ["POST", `${S}/collections/:key/fields/operations`, "site:settings"],
+  ["GET", `${S}/collections/:key/fields/operations/:operationId`, "content:read"],
+  ["POST", `${S}/collections/:key/fields/operations/:operationId/revert`, "site:settings"],
   ["GET", `${S}/pages`, "content:read"],
   ["POST", `${S}/pages`, "content:write"],
   ["GET", `${S}/pages/:pageId/preview`, "content:read"],

packages/api/src/testing-fields.ts

@@ -160,5 +160,46 @@ export function describeFieldsApi(label: string, factory: ApiStoreFactory): void
       expect(made.status, JSON.stringify(made.body)).toBe(201);
       expect(made.body.fields).toEqual(loose);
     });
+
+    it("field operations: dry run, run, status, continue and revert (RFC-0041 §6)", async () => {
+      expect((await put(EVENTS)).status).toBe(200);
+      const made = await create({ venue: "Hall", level: "expert" });
+      expect(made.status).toBe(201);
+      const ops = `/api/sites/${siteId}/collections/events/fields/operations`;
+      const post = (body: Json, query = "") =>
+        call(`${ops}${query}`, { method: "POST", body: JSON.stringify(body) });
+      expect((await post({ op: "explode" })).status).toBe(400);
+      const unknown = await post({ op: "remove_field", key: "colour" });
+      expect(unknown.status).toBe(422);
+      expect(unknown.body.error).toBe("unknown_field");
+      const dry = await post({ op: "rename_field", from: "venue", to: "place", dryRun: true });
+      expect(dry.status, JSON.stringify(dry.body)).toBe(200);
+      expect(dry.body).toMatchObject({ dryRun: true, report: { moved: 1, touched: 1 } });
+      const run = await post({ op: "rename_field", from: "venue", to: "place" });
+      expect(run.status, JSON.stringify(run.body)).toBe(200);
+      expect(run.body.done).toBe(true);
+      const id = run.body.operationId as string;
+      const entry = await call(`/api/sites/${siteId}/entries/${made.body.id}`);
+      expect(entry.body.fields).toEqual({ place: "Hall", level: "expert" });
+      const status = await call(`${ops}/${id}`);
+      expect(status.body).toMatchObject({ state: "done", pending: 0 });
+      expect((await post({}, `?continue=${id}`)).body).toMatchObject({ done: true });
+      // A type change that would lose values is refused with the report (422).
+      const lossy = await post({ op: "change_field_type", key: "level", type: "number" });
+      expect(lossy.status).toBe(422);
+      expect(lossy.body.error).toBe("field_values_unconvertible");
+      expect((lossy.body.details as Json).report).toMatchObject({ unconvertible: 1 });
+      const back = await call(`${ops}/${id}/revert`, { method: "POST" });
+      expect(back.status, JSON.stringify(back.body)).toBe(200);
+      expect(back.body).toMatchObject({ done: true, restored: 1 });
+      expect((await call(`/api/sites/${siteId}/entries/${made.body.id}`)).body.fields).toEqual({
+        venue: "Hall",
+        level: "expert",
+      });
+      const again = await call(`${ops}/${id}/revert`, { method: "POST" });
+      expect(again.status).toBe(409);
+      expect(again.body.error).toBe("field_operation_incomplete");
+      expect((await call(`${ops}/snap_doesnotexist`)).status).toBe(404);
+    });
   });
 }