feat(core-ts): entry_refs and reference integrity, gated off (RFC-0045 step 2a, #114)

.changeset/reference-store-integrity.md

@@ -0,0 +1,21 @@
+---
+"@cloudhal/storage": minor
+"@cloudhal/core-ts": minor
+"@cloudhal/api": patch
+"@cloudhal/mcp-server": patch
+---
+
+Reference fields get their index and integrity rules (RFC-0045 §2, rollout step 2a, #114). Both new field types stay refused on every write path in production, so no reference value can exist yet; the store and the shared suites run them through `SiteGraphStoreOptions.fieldTypes` and `CopySiteOptions.fieldTypes`.
+
+- **Storage.** Migration `0033_entry_refs` adds `entry_refs`, one row per stored reference value, removed with its site.
+- **Store.** Entry writes keep the rows in their own batch and check their targets at batch time (`unknown_entry`, `reference_target_moved`). Deletes, moves and group changes are refused while a published entry shows a reference to the group's last usable member (`entry_referenced`, `reference_target_moved`, `field_operation_running`). Imports rebuild the rows beside a written node set. Field operations naming a reference or repeater field answer `field_operation_unsupported`. Definition changes that would strand values are refused (`field_key_in_use`, removed target collections or sub-fields, `multiple`).
+- **API and MCP.** The new refusals answer 409 or 422, with MCP hints.
+
+**Behaviour changes on existing sites** (RFC-0045 decision 3):
+
+- relinking a node that other nodes use as their translation group is refused (`translation_group_in_use`, 409);
+- `resolveFields` and the store read an original's non-localized values only when the original's own group is its id: a node relinked into another group lends them to nobody (OQ 14). The list-values fingerprint format moves to 2, so every listed collection's `entry_list_values` rebuild once (`lists.backfill`); listings scan until they are ready;
+- a change of `translation.group` alone gets the same source-group check as a source change, and a write joining a group checks that its node is still that group's original;
+- imports store node bodies in canonical form (the hash is unchanged);
+- deletes, moves and group changes of entries carry the collections anchor, and entry writes that change `fields` or `collection` carry the definitions anchor: a concurrent change of the collections' settings makes them answer 409, and a retry succeeds. The anchors compare the stored text read when the write is planned, so site rows stored with any key order work;
+- field operations that would turn a field into a reference or repeater answer `field_operation_unsupported` (409) instead of `fields_unsupported`.

docs/adr/0046-references-and-repeaters.md

@@ -0,0 +1,21 @@
+# ADR-0046: Collections stage 4: reference fields by translation group, guarded by an `entry_refs` index; repeaters of flat items
+- Status: Accepted
+- Date: 2026-11-11 (decided by Halil, RFC-0045 accepted 2026-11-10; lands with rollout step 2a)
+- Related: RFC-0045, RFC-0041 / ADR-0043, RFC-0042 / ADR-0044, RFC-0033 / ADR-0035, RFC-0004 (`asset_refs`), RFC-0008 / ADR-0023 (releases), #114, #71
+- Sources: [Sanity reference type](https://www.sanity.io/docs/studio/reference-type) · [Payload relationship field](https://payloadcms.com/docs/fields/relationship) · [Cloudflare D1 limits](https://developers.cloudflare.com/d1/platform/limits/) · [SQLite JSON functions](https://www.sqlite.org/json1.html)
+
+## Context
+Collections have typed fields (ADR-0043), but a venue, a speaker or a brand is still plain text retyped on every entry, and a recipe's ingredients are lines of one text field. Two field types fix that, and each needs integrity rules that hold under D1, which has only atomic batches (AGENTS.md rule 4).
+
+## Decision
+1. **Two types.** `reference` (one to five target collections of the same site, one target or up to 50) and `repeater` (up to 50 items of up to ten flat scalar sub-fields; no nesting). Both are refused on every write path until rollout step 7, when the site Worker reports `fields: 2`: definitions with `fields_unsupported`, field operations that name them with `field_operation_unsupported` (until step 4).
+2. **A reference stores a translation group id**, `COALESCE(translation.group, id)`, the one identity shared by every language version in every site. It resolves only through members of the page's own site, in the page's locale, then `localization.fallback`. Membership is that expression everywhere (guards, renderer, purges).
+3. **`entry_refs`** holds one row per (holder, field path, stored group id), whatever the target's state, written in the holder's own batch, anchored on its new hash, with a plain `INSERT` (never `OR IGNORE`) that reads `shared` from the site row at batch time. Every guard and lookup filters `site_id`.
+4. **Strong where the live site depends.** Only what a published entry shows locks: a delete, a move out of every allowed collection, or a group change of a group's last usable member is refused while a published entry shows a reference to it (`entry_referenced`, `reference_target_moved`). Archiving and unpublishing are always allowed; an unresolved reference renders nothing. Publishing a *site* warns (`referenceIssues`) and never fails; publishing a *referrer* whose own value names a group with no usable member is refused (`unknown_entry`).
+5. **Races are closed at batch time**, not by reads: the target guard in the referrer's CAS; the definitions anchor (a collection's definitions as planned) on every entry write in a site with reference fields, and on writes that change values elsewhere; the collections anchor on every delete, move and group change. Both anchors compare the fragment's stored text, read when the write is planned and checked against the planned `Site`, so they never depend on the row's key order; planned pairs with an unseen-field check; the written-group and dependants guards on translation links (`translation_group_in_use`). The cases R1–R16 of RFC-0045 §2.3 are shared-suite tests, except R9 (field-operation rewrites, rollout step 2b; unreachable before step 4) and R14 (accepted, holds).
+6. **Rebuilds only beside a written node set** (imports, pulls, pushes, restores), chunked below 90 KB a statement. References are not release dependencies (rollout step 2b).
+
+## Consequences
+- Existing sites see the phase-2 behaviour changes listed in RFC-0045's "Decisions" (relink refusal, the `readsOriginal` tightening with a one-time rebuild of listed collections' `entry_list_values`, canonical import bodies, the anchors' 409s on concurrent settings changes, `field_operation_unsupported`).
+- Purges read the published and previous snapshots' bodies (rollout step 3); no new cache tags.
+- Per-sub-field localization, `datetime` and multiple values inside items, and sorting by a referenced entry's field stay out of stage 4 (RFC-0045 open questions 2 and 3, §4).

docs/adr/README.md

@@ -62,3 +62,4 @@ One file per decision, numbered, immutable once accepted (write a new ADR to sup
 | [0043](0043-typed-custom-fields.md) | Typed custom fields: nine closed types on `Site.collections.<key>.fields` (discriminator `type`), values in `ContentEntry.fields`, one eval-free checker (`@cloudhal/core-ts/fields`) for store, API and MCP that also emits the agents' JSON Schema, `requiredToPublish` at review and publish, per-field localization, definitions on the site node with caps, renames/type changes/removals only as snapshotted, approved operations (RFC-0041) |
 | [0044](0044-collections-stage-3.md) | Collections stage 3: JSON-LD presets (Event, JobPosting, Product, Recipe) mapped from fields through a closed table, expiry as a render-time gate (`unlist`/`hide`, no automation), listings filtered and sorted by field with one shared evaluator and no indexes yet, listing pages as ordinary pages and per-collection feeds through a middleware (RFC-0042) |
 | [0045](0045-mcp-authorization-server.md) | MCP authorization server: Better Auth `mcp()` + `jwt()` on the Studio database, issuer `<public origin>/api/auth`, Ed25519 JWT access tokens (1 h, `aud` = `/mcp`) verified locally plus one `agent_grants` read per call, refresh rotation with a 30 s reuse window, consent on Studio's screen narrowed by the installation role, a token resolves to the same `Principal` as a key; no AS without a stable public origin (RFC-0044 Phase 2) |
+| [0046](0046-references-and-repeaters.md) | Collections stage 4: `reference` fields store a translation group id resolved per site and locale, `repeater` fields hold up to 50 flat items; an `entry_refs` index written in each entry's batch guards deletes, moves and group changes only while a published entry shows the reference, with every race closed by batch-time anchors and guards (RFC-0045) |