[Madraka]

What

Closes #110 (RFC-0042 §4, the side table; open question 10's trigger fired in #105's benchmark).

queryContentEntries serves GET /entries?where=…, Studio's lists and MCP list_entries. It now answers where, orderBy, status, locale, limit and offset in SQL from a new entry_list_values table once a collection is ready. Until now it read and evaluated the whole collection on every call.

10,000 entries, store path	SQLite CPU / rows	D1 CPU / rows read
before (a95b7113)	128.7 ms / 10,001	221.4 ms / 10,003
after, first page (startsAt ≥ now, ordered, 20)	0.6 ms / 22	4.7 ms / 5,068
after, page at offset 4,980	0.5 / 22	7.7 / 5,088
after, unordered page matching 70%	3.6 / 7,021	10.6 / 7,147
public site, warm snapshot (unchanged)	22.8 / 0	22.6 / 0

The bench (pnpm --filter @cloudhal/astro-renderer bench:lists) asserts the RFC's threshold (≤ 50 ms CPU, ≤ 10,000 rows) for the API path and the broad unordered page.

How

• storage: migration 0029 adds entry_list_values and entry_list_state. entry_list_values holds field, presence, number, text, sort tie-break, created, node, locale and status, with a covering query index and a per-node index. entry_list_state holds each listed collection's fingerprint, readiness, backfill epoch and cursor. The migration is DDL only.
• core-ts (src/list-values/):
  • Which values are indexed: rows come from resolveFields, for listFields keys only (at most 4 per collection). Every entry gets exactly one row per listed field, plus one row per chosen option for multi-choice fields. Long text and doubles SQL can't compare exactly are left NULL, and those queries fall back to the scan.
  • Writes keep the table exact:
    • Entry writes write their rows in their own atomic batch, anchored on the entry's new hash (AGENTS rule 4).
    • An original's write rewrites its same-site translations' rows.
    • Site writes, field operations, reverts and imports that change list definitions reset readiness in the same batch. A race never refuses a write; it marks the collection not ready instead.
  • Backfill: rebuildListValues is resumable and budgeted, and safe to run concurrently because every chunk is guarded on fingerprint, epoch and cursor. updateSite, finished field operations, importSite and openLocalStore run it for whatever they reset.
  • Query path: only ready collections under the current fingerprint use SQL, and every statement re-checks readiness. Everything else, including term filters, takes today's scan unchanged. The evaluator remains the oracle in a seeded property suite on SQLite, Miniflare D1 and Durable Object SQLite.
• cloudflare: a lists.backfill system job runs every five minutes. Its declared cost (16 statements on free, 150 on paid) is enforced as a statement budget on every statement. Each run starts at another waiting collection. On paid it also tidies rows of collections that are no longer listed, never those of a collection listed while it ran.
• mcp-server: list_entries with orderBy and limit lets the store page. Arguments, cursors and output are unchanged.
• Behaviour change: revertTagsToTerms now leaves an entry edited after its before-snapshot as it is and reports it in edited. Before, it restored the entry's earlier fields, status or collection.

Known limits (written up in RFC-0042)

• An unpaged answer with a where and no orderBy (MCP list_entries) returns every matching body. For 5,000 matches that is about 15,000 rows read on D1 (71 ms CPU), still over the threshold. Per-collection counters or paging would fix it.
• A query whose total is the whole collection (orderBy alone, ne, unset) still counts about N index rows, though with far less CPU.

Verification (local, one process at a time)

• core-ts: the full suite, 60 files and 1,085 tests. That includes 51 write-path, race, backfill and repair tests and 18 property-suite tests on three backends. Each race and restore fix was confirmed by removing it and watching its test fail.
• test/list-values-sql.test.ts uses EXPLAIN QUERY PLAN to check that no write-path or backfill statement scans nodes, and that the query statements seek their indexes.
• storage: 82 tests, with 0029 on better-sqlite3 and D1, idempotent. mcp-server listings and e2e, api listings-api/fields-api/collections-api on sqlite and D1, and cloudflare list-values-job (D1: 4 sites and 20 collections never exceed the job's cost) all pass.
• packages/cli: archive, export, import, site and fields tests, 14 files and 210 tests. Run because openLocalStore now backfills.
• pnpm typecheck (20/20), pnpm lint, pnpm size:check (store.ts shrinks 1221 → 1212), pnpm drift and the MCP tools doc check all pass.

Checklist

• pnpm check and cargo test --workspace pass locally. Everything touched was run, including the full core-ts suite. There is no Rust or contract change; CI runs the rest.
• Contract change? No (a storage migration, not a contract schema).
• User-visible change in a published package? Changesets for storage, core-ts, mcp-server and cloudflare.
• Commits are signed off (git commit -s, DCO)
• Docs / AGENTS.md updated if behaviour or commands changed. The RFC-0042 §4 implementation note and ADR-0044 are updated.

Closes #110

🤖 Generated with Claude Code