Skip to content

fix(core-ts): scope asset_refs by site and skip rebuilds on unchanged imports (#196) - #200

Merged
Madraka merged 2 commits into
mainfrom
claude/practical-lichterman-fc3a5d
Nov 11, 2026
Merged

Madraka merged 2 commits into
mainfrom
claude/practical-lichterman-fc3a5d

Conversation

@Madraka

@Madraka Madraka commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

RFC-0045 rollout step 0 (§2.5, "A prerequisite, decided, not optional"). It does not touch the RFC's new field types.

Closes #196

What changed

  • refsRebuild (packages/core-ts/src/media/refs-rebuild.ts)
    • The INSERT … SELECT joins nodes (a.site_id = ? AND a.kind = 'asset'), so only references to assets of the same site get rows.
    • It is chunked so each statement (SQL plus parameters, UTF-8 bytes) stays below REFS_STATEMENT_BYTES = 90,000.
    • It is a plain INSERT now, without OR IGNORE. Rows are deduplicated in JS by (asset, node, field).
  • Import batch (sync-plan.ts nodeBatch): the rebuild is emitted only inside !changes.unchanged && !changes.conflict, after the node INSERTs. The join needs those rows, and the chunks count against the existing 1,000-statement cap. An unchanged push, pull or import no longer replaces a concurrent edit's fresh row with plan-time rows (Sync: an 'unchanged' import rebuilds asset_refs from plan-time rows and can drop a concurrent edit's row #196).
  • deleteAsset guard (assets.ts): NOT EXISTS (SELECT 1 FROM asset_refs WHERE site_id = ? AND asset_id = ?).
  • unused filter (assets.ts): it uses UNUSED_ASSET_SQL, which matches r.site_id = nodes.site_id. It lives in refs-rebuild.ts so that assets.ts adds no net lines (4+/4−).
  • Purge planner: the asset rule moved out of planCachePurge into the new cache-purge-assets.ts (planAssetPurge).
    • It also reads typed field assets with fieldAssetRefs, using the target snapshot's site definitions. A changed asset shown only through an image or richText field now purges that entry's node tag.
    • planCachePurge went from 233 to 207 lines, and the baseline is tightened.
  • store.ts is untouched. Changeset: @cloudhal/core-ts patch.

Tests

  • New shared suite src/testing-asset-refs.ts, registered in describeSiteGraphStore, so it runs on sqlite, D1 and DO SQLite:
    • Sync: an 'unchanged' import rebuilds asset_refs from plan-time rows and can drop a concurrent edit's row #196: an import planned unchanged, then an asset-adding edit, then the planned batch. The edit's row survives, the asset is not listed as unused, and its delete is refused with in_use. With the old unconditional rebuild restored, this case fails.
    • A foreign archive: site X's rows naming site Y's asset get no row, while X's own logo does. A legacy unscoped row does not hide Y's asset from Y's unused list or block its delete.
    • A large rebuild: 5,000 rows go into a handful of statements, each below 90 KB, and all 5,000 rows land.
  • test/cache-purge.test.ts: a changed asset purges the entry that shows it through a typed image field. It fails without the fix.
  • Adjusted: "sync carries asset nodes…" (testing-assets.ts) and "an import rebuilds the field references with the rest" (testing-fields.ts). Both used an unchanged import to repair a hand-deleted index. They now edit after the export, so the import writes the node set.

Run locally

The full D1 and DO runs are left to CI because of the machine constraint.

  • pnpm exec vitest run test/store.test.ts -t "better-sqlite3" → 160 passed
  • pnpm exec vitest run test/cache-purge.test.ts → 11 passed
  • pnpm exec vitest run test/sync.test.ts test/sync-plan.test.ts test/media.test.ts test/multi-site.test.ts → 61 passed
  • pnpm --filter @cloudhal/core-ts typecheck → clean
  • pnpm size:check → ok (baseline tightened)
  • biome check on the touched files → clean (the complexity warnings were there before)

Review notes (RFC watch list)

  • No OR IGNORE on the rebuild. The rebuild never runs beside an unchanged node set. Every new probe carries site_id.
  • AssetEngine.rebuildRefsStatements (repair, no callers) still returns a standalone rebuild. Anyone who calls it later must put it beside a node-set write.

🤖 Generated with Claude Code

Madraka and others added 2 commits November 11, 2026 02:55
… imports (#196)

RFC-0045 rollout step 0 (§2.5), independent of the RFC's new field types.

- refsRebuild joins nodes so only same-site assets get rows, chunks its
  INSERT below 90 KB per statement (SQL plus parameters), drops OR IGNORE
  (rows are deduplicated in JS), and is emitted by the import batch only
  when that batch writes the node set. An unchanged import can no longer
  replace a concurrent edit's fresh row with plan-time rows (#196).
- deleteAsset's guard and listAssets' unused filter read asset_refs by
  site_id, so legacy rows a foreign archive left in another site neither
  block the delete nor hide the asset.
- The purge planner's asset rule moves to cache-purge-assets.ts and also
  reads typed field assets (fieldAssetRefs, the target snapshot's site
  definitions). planCachePurge shrinks 233 -> 207 lines (baseline
  tightened); assets.ts adds no net lines.
- Tests: new shared suite testing-asset-refs.ts (#196 race, foreign
  archive, 5,000-row rebuild under the size limit); a field-image purge
  case; the two "import rebuilds refs" cases now edit after the export so
  the import writes the node set.

Closes #196

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Madraka <8898640+Madraka@users.noreply.github.com>
 review)

PR #200 review: the purge planner's asset rule read field values only by
their definition key, so mid-rename (phase A's marker written, phase B
stopped on its budget) entries still holding the old key were missed, and
an asset change left their cached pages pointing at the old URL.

- migratedFieldValues (media/field-refs.ts) reads a defined key the entry
  does not hold yet from migration.fromKey, as resolveFields' read() does.
- The purge asset rule (cache-purge-assets.ts) uses it.
- refsRebuild uses it too: otherwise an archive exported mid-rename would
  drop those entries' asset_refs rows, and an asset they still show could
  be deleted (RFC-0045 §2.5 point 4 asks the same of entryRefsRebuild).
  The writer path and fieldAssetIssues are unchanged.
- Tests: a cache-purge case with a budget-stopped rename_field and three
  entries showing the asset; a shared-suite rebuild case with a marker.
  Both fail without the fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Madraka <8898640+Madraka@users.noreply.github.com>
@Madraka
Madraka merged commit 5126113 into main Nov 11, 2026
8 checks passed
@Madraka
Madraka deleted the claude/practical-lichterman-fc3a5d branch November 11, 2026 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sync: an 'unchanged' import rebuilds asset_refs from plan-time rows and can drop a concurrent edit's row

1 participant