diff --git a/.changeset/collections-stage3-feeds.md b/.changeset/collections-stage3-feeds.md new file mode 100644 index 00000000..3b75811a --- /dev/null +++ b/.changeset/collections-stage3-feeds.md @@ -0,0 +1,6 @@ +--- +"@cloudhal/astro-renderer": minor +"@cloudhal/cli": minor +--- + +Per-collection feeds (RFC-0042 step 6, ADR-0044). Each collection with a permalink prefix has an RSS feed at `/rss.xml` (and `//rss.xml` or the locale's domain): its newest listed entries, so expiry and time gates apply as in the site feed, with the same cache tag and edge-cache cap; `feed: false` answers 404. The `cloudhal()` integration serves it from a middleware on every target, only where the theme's own routes answered 404, so a theme's own route wins and old theme copies gain the feeds; static builds write them because `staticPaths().feeds` now lists every collection feed path for the starter's `[...path]/rss.xml` route. New theme API `collectionFeedLink(site, collection, origin, locale?)` (`./seo`, `./collection-feeds`) gives the ``; the blog starter adds it on entry pages and on pages that list a collection. Previews now list `contentList` sections with `where`/`orderBy` through the same evaluator as the site (`renderSectionHtml`'s new optional `listing` argument, passed by `renderPagePreview`), so an entry without the field's value no longer shows under "upcoming". New guide: `docs/guides/structured-data.md`. diff --git a/apps/docs/src/lib/site-pages.ts b/apps/docs/src/lib/site-pages.ts index 36d139fa..84f2f4c5 100644 --- a/apps/docs/src/lib/site-pages.ts +++ b/apps/docs/src/lib/site-pages.ts @@ -87,6 +87,7 @@ const docsIndex: SitePage = { - [Search](/docs/guides/search), [schedules](/docs/guides/schedules), [comments](/docs/guides/comments) - [Typed custom fields](/docs/guides/fields): definitions, operations, languages and themes +- [Structured data, expiry and listings](/docs/guides/structured-data): JSON-LD presets, entries that end on a date, listings by field and collection feeds - [Several sites in one deployment](/docs/guides/multi-site) - [Analytics and privacy](/docs/guides/analytics-privacy) - [Translations with Weblate](/docs/guides/translations-weblate) diff --git a/docs/guides/structured-data.md b/docs/guides/structured-data.md new file mode 100644 index 00000000..f24d627c --- /dev/null +++ b/docs/guides/structured-data.md @@ -0,0 +1,115 @@ +# Structured data, expiry and listings: a guide for site owners and theme authors + +A collection of events, jobs, products or recipes can do more than hold typed fields. It can tell search engines what each entry **is** (a JSON-LD preset), let entries **end on a date** of their own (expiry), and **list entries by their fields** ("upcoming events, soonest first"), on listing pages and in the collection's own feed. + +All three are collection settings: they build on [typed custom fields](fields.md), are approved like any other site setting, and change nothing until a site opts in. The design and its reasons are in [RFC-0042](../rfcs/0042-collections-stage-3.md) and [ADR-0044](../adr/0044-collections-stage-3.md). + +## Structured data presets + +Without a preset, every entry page describes itself as a `BlogPosting`. A preset replaces that document with the entry's real type, built from its fields, and keeps the `BreadcrumbList`. + +| Type | Required (Google) | Typical mapping | +|---|---|---| +| `Event` | `name`, `startDate`, `location.name`, at least one `location.address.*` | `startDate` ← a datetime field, `location.name` ← a venue text field, `eventStatus` ← a select | +| `JobPosting` | `title`, `description`, `datePosted`, `hiringOrganization.name`, `jobLocation.address.*` (or `jobLocationType` with `applicantLocationRequirements.name`) | `validThrough` ← a date field, `employmentType` ← a select, `jobLocationType` ← a boolean (true means remote) | +| `Product` | `name`, `offers.price` | `offers.price` ← a number field, `offers.priceCurrency` ← a constant such as `EUR` | +| `Recipe` | `name`, `image` | `recipeIngredient` ← a multiline text field (one per line), `recipeInstructions` ← rich text (one step per paragraph), times ← whole minutes | + +Why these four: they are in Google's current search gallery, and each describes one item per page, which is exactly an entry page. + +- **`FAQPage` is not offered.** Google deprecated FAQ rich results and stopped showing them in 2026; it was also never a one-item-per-page type. +- **No ratings or reviews.** Google's policy forbids self-serving review markup and Cloudhal stores no reviews, so `aggregateRating` and `review` are not in the table (a constant could otherwise fake one). +- **`Organization` / `LocalBusiness`** describe the site, not an entry, and belong to site settings later. + +Every property takes its value from one of four sources: a **field** of the collection (its type must fit the property), an **entry** property (`title`, `summary`, `body`, `coverImage`, `url`, `publishedAt`, `updatedAt`), a **site** property (`name`, `url`, `logo`), or a short plain **constant** (only on enum paths such as `eventStatus` and on currencies; required values always come from the entry). Many paths have a default (an event's `name` is the entry title, a job's `hiringOrganization.name` is the site name), so a working Event needs only a start date, a venue and a city. + +Values are converted per field type: a datetime gives its instant with its own zone's offset (`2026-11-05T19:30:00+03:00`), a date gives `YYYY-MM-DD`, images become absolute asset URLs, rich text becomes plain text, URLs are made absolute (unsafe ones are dropped), and select options map through the property's vocabulary (`cancelled` → `https://schema.org/EventCancelled`). An option with no meaning there leaves the property out. + +### Mapping fields + +- **Studio:** a collection's settings, **Structured data**. Pick a type, then a source for each property (required first). The card flags mismatches, suggests marking fields mapped to required properties **Needed to publish**, and reminds you that search engines expect the values to be visible on the page (the starter's `EntryFields` shows every field with a value). +- **Agents (MCP):** `update_collection` takes `structuredData` (`{ "type": "Event", "properties": { "startDate": { "kind": "field", "field": "startsAt" } } }`). It waits for a person who may change site settings; try `dryRun: true` first. +- **API:** `PUT /api/sites/:siteId/collections/:key`. An invalid preset answers `422 structured_data_invalid` with one issue per path. + +### Checking the result + +An entry whose required values are missing emits **no** preset document (Google treats incomplete items as invalid): its page keeps the `BlogPosting`, and Studio says which value is missing. + +- **Studio:** the Structured data card's **Preview** shows, for a chosen entry, exactly what its page emits with the saved settings, and links to **Google's Rich Results Test** for the entry's public URL. +- **Agents:** `preview_structured_data { siteId, entryId }` (read-only, `content:read`) returns the document and its issues. Check it before asking for a publish. + +## Expiry: entries that end on a date + +`expiry` names a date or datetime field and an action. Once an entry's date has passed, the site treats it as ended. Nothing runs at that moment: no job, no write, no unpublish. Every page view compares the date with the clock, as [time gates](schedules.md#embargoes-and-expiries-time-gates) do, so removing the rule brings every entry back at the next publish. + +| Action | Entry page | Lists, `contentList`, archives, related | Feeds, sitemap, search, `llms.txt` | +|---|---|---|---| +| `unlist` (default) | stays (200), JSON-LD unchanged with its dates in the past | left out (themes can ask for them with `includeExpired`) | left out | +| `hide` | 404 `no-store`, like an expired embargo | left out | left out | + +`unlist` suits events (pages keep their links and history) and jobs (a `JobPosting` with `validThrough` in the past is one of Google's accepted ways to retire a posting). `hide` suits offers that must not stay reachable. Neither changes the entry's status; Studio shows it as ended, and previews show it with an "Ended on …" notice. + +- **When it ends:** a datetime ends at its instant. A **date ends at midnight after it in the site's time zone** (`Site.timeZone`, UTC when unset), so an event "on 5 November" is listed all of 5 November. `afterMinutes` adds a grace period of up to 10,080 minutes (one week). An entry without a value never ends. +- **Your own gates still apply:** the entry ends at the earlier of its own `availability.until` and the derived date. +- **Caching:** every ended instant joins the published snapshot's gate index, so the edge cache expires at each one without a purge. Static builds show the site as of the build and warn when a gate lies ahead; build again then. +- **Who decides:** the rule is a collection setting (`site:settings`). Publishing a change that ends a live entry earlier, or gives it an end, removes content at a set time, so an agent's publish carrying it waits for a person. +- **Deploy first on Cloudflare:** an older site Worker would ignore the rule and keep showing ended entries. The admin Worker refuses `expiry` (`409 gates_unsupported`, over REST, MCP and `cloudhal push`) until the site Worker answers `{ "gates": 2 }` on `GET /__cloudhal/features`. After upgrading, deploy both Workers (`cloudhal deploy` does), then save the rule. Locally and on Node there is nothing to wait for. + +## Listings by field + +`listFields` names up to four fields a collection may be filtered and sorted by: they are the columns of its list in Studio, the conditions `contentList` sections and agents may use, and what the gate index watches for `now`. Conditions are ANDed; values are strings read by the field's type. + +| Field type | `where` operators | `orderBy` | `value` | +|---|---|---|---| +| `number` | `eq ne lt lte gt gte set unset` | yes | a decimal | +| `date` | same | yes | `YYYY-MM-DD` or `now` (today in the site's zone) | +| `datetime` | same (compares the instant) | yes | an RFC 3339 instant or `now` | +| `boolean` | `eq` (no value reads as false) | no | `true` / `false` | +| `select` | `eq ne in set unset` (with `multiple`: contains, contains any) | no | option values (`values` for `in`, up to 20) | +| `text`, `url` | `eq set unset` | no | the exact string | +| `richText`, `image` | `set unset` | no | none | + +Entries without a value sort last in both directions; ties fall back to the section's or collection's `sort`, then the id. Not offered: text sorting (collation differs between SQL and browsers), `contains`/`LIKE` (search covers it), OR, and offsets on `now`. A translation filters by its original's non-localized values. + +```json +{ "where": [{ "field": "startsAt", "op": "gte", "value": "now" }], + "orderBy": { "field": "startsAt", "direction": "asc" } } +``` + +Everywhere: `contentList` sections (`where`, `orderBy` beside `sort` and `filter`), `list_entries` and `GET /api/sites/:siteId/entries` (`where` as JSON, `orderBy`, `limit`/`cursor`), and Studio's filter bar and sortable columns. Writers check conditions against the collection (`422 invalid_query`), and a field cannot leave `listFields` or change type while a section still uses it (`409 list_field_in_use`, naming the pages). On a published site an invalid query matches nothing and is logged, so a snapshot keeps rendering. + +### Listing pages + +A collection's index page is an ordinary page with a `contentList` section, at the collection's prefix (`/events` for `/events/{slug}`). **Create listing page** in the collection's settings makes a draft there, titled with the plural label; for a collection with a date listing field and an expiry, the section starts as "upcoming, soonest first". Agents do the same with `create_page`. Menus link to it through the renderer's `collectionIndex` default, and old themes render it, because pages and `ContentList.astro` are the renderer's. + +### Collection feeds + +Each collection with a prefix has its own RSS feed at `/rss.xml` (`/events/rss.xml`, and `//events/rss.xml` or the locale's domain for other languages): its newest listed entries, so ended and gated entries stay out, titled after the plural label, with the site feed's cache tag and edge-cache cap. A collection with `feed: false` has none (404), and so does one whose permalink has no prefix (`/{slug}`, whose feed would be the site's `/rss.xml`). + +The renderer serves it with a middleware that answers **only when the theme's own routes answered 404**: a theme with its own route for the path keeps it, and theme copies from before this feature gain the feeds on upgrade. Static builds write the files through the starter's `[...path]/rss.xml` route (it lists the paths, answers 404, and the middleware fills in the feed); the Cloudflare site Worker and `cloudhal dev` answer per request. The starter links the feed (``) on entry pages and on pages that list the collection. + +## Theme API + +All additive (`@cloudhal/astro-renderer`; see its README's "Theme-facing API stability"). `entryHead`, `entries()`, `allEntries()` and every older export keep their meaning. + +| Export | What | +|---|---| +| `withStructuredData(head, source, entry, { origin, media? })` (`./seo`) | Call after `entryHead`: replaces the `BlogPosting` with the collection's preset when the entry yields one, keeps the breadcrumbs, otherwise returns `head` unchanged. `entryStructuredData` returns the document and its issues. | +| `queryEntries(source, q)` (`./queries`) | `{ collection, locale?, where?, orderBy?, sort?, filter?, limit? (1–100, default 20), offset?, includeExpired? }` → `{ entries, total }` on any source; `now` is the source's clock. | +| `contentListEntries(source, sections, locale?)` (`./queries`) | The `entries` map `Sections` reads: by collection, and by section id for sections with a term `filter` or `where`/`orderBy`. | +| `collectionFeedLink(site, collection, origin, locale?)` (`./seo`, `./collection-feeds`) | `{ title, href }` for the collection's feed on the locale's URL, or `undefined` without one. Add it to `head.feeds` **after** `localizeHead`, which would otherwise localize the URL twice. | + +```astro +--- +const head = await withStructuredData(entryHead(site, entry, origin, terms, media), source, entry, { origin, media }); +const meta = localizeHead(head, site, { locale, origin, alternates }); +const feed = collectionFeedLink(site, entry.collection, origin, locale); +const shown = feed ? { ...meta, feeds: [...meta.feeds, feed] } : meta; +const upcoming = await queryEntries(source, { + collection: "events", + where: [{ field: "startsAt", op: "gte", value: "now" }], + orderBy: { field: "startsAt", direction: "asc" }, + limit: 5, +}); +--- +``` diff --git a/packages/astro-renderer/README.md b/packages/astro-renderer/README.md index 3499e38c..bc22f5c7 100644 --- a/packages/astro-renderer/README.md +++ b/packages/astro-renderer/README.md @@ -57,6 +57,15 @@ Additive. `entryHead`, `entries()`, `allEntries()` and every existing export kee | `GraphSource.routableEntries?` | `(locale?) => Promise`: every entry with a public URL, unlisted ones included (routes, static builds, alternates); `allEntries` stays the listed ones | | `withStructuredData` (`./seo`, `.`) | `(head: HeadMeta, source, entry, { origin, media? }) => Promise`: call after `entryHead`; replaces the `BlogPosting` with the collection's preset (`Event`, `JobPosting`, `Product`, `Recipe`) when the entry has every required value, keeps the `BreadcrumbList`, returns `head` unchanged otherwise | | `entryStructuredData` (`./seo`, `.`) | `(source, entry, { origin, media? }) => Promise<{ doc?, issues }>`: the preset document and why there is none | +| `collectionFeedLink` (`./seo`, `./collection-feeds`, `.`) | `(site, collection, origin, locale?) => FeedLink \| undefined`: `{ title, href }` of the collection's feed (`/rss.xml`) on the locale's URL, `undefined` with `feed: false` or a permalink without a prefix; add it to `head.feeds` after `localizeHead` | +| `collectionFeedPath`, `feedCollections`, `collectionFeedPaths`, `collectionForFeedPath` (`./collection-feeds`) | which collections have a feed and where; `staticPaths().feeds` now includes every collection feed path | +| `collectionFeedXml`, `serveCollectionFeed` (`./collection-feed-route`) | a collection's RSS in a locale; the middleware's decision (a theme's 404 for a feed path becomes the feed) | + +Collection feeds: the `cloudhal()` integration adds a middleware (`collection-feed-middleware`) on every target that serves `/rss.xml` (and `//rss.xml`) only where the theme's own routes answered 404, so a theme's own route wins and old copies gain the feeds; static builds write them through the starter's `[...path]/rss.xml` route, whose paths come from `staticPaths().feeds`. Same rules and cache tag as `rss.xml`. + +Previews: `renderSectionHtml(section, entries, media?, listing?)` takes `{ site, now? }` as a fourth argument, and `renderPagePreview` passes it, so `contentList` sections with `where`/`orderBy` preview through `evaluateEntryQuery` like the site (a term `filter` is still applied on the site only). + +Guide: [`docs/guides/structured-data.md`](../../docs/guides/structured-data.md). Derived gates: `unlist` keeps an entry's page (200), comments and menu links but leaves `entries()`, `allEntries()`, `entriesByFilter`/`entriesByTerm` (archives, related), feeds, the sitemap and search; `hide` is a 404 like an expired `availability`. Both sources apply them, the gate index caps the edge cache at every derived instant and at every value instant of date/datetime list fields (for `now` conditions), and previews ignore them with an "Ended on …" notice. The site Worker's `GET /__cloudhal/features` answers `{ "gates": 2 }`; writers refuse `expiry` until it does. diff --git a/packages/astro-renderer/package.json b/packages/astro-renderer/package.json index 268738ce..1b9e61c3 100644 --- a/packages/astro-renderer/package.json +++ b/packages/astro-renderer/package.json @@ -22,6 +22,9 @@ "./search": "./src/search.ts", "./seo": "./src/seo.ts", "./feeds": "./src/feeds.ts", + "./collection-feeds": "./src/collection-feeds.ts", + "./collection-feed-route": "./src/collection-feed-route.ts", + "./collection-feed-middleware": "./src/collection-feed-middleware.ts", "./taxonomy": "./src/taxonomy.ts", "./archive": "./src/archive.ts", "./cache-tags": "./src/cache-tags.ts", @@ -138,6 +141,18 @@ "types": "./dist/src/feeds.d.ts", "default": "./dist/src/feeds.js" }, + "./collection-feeds": { + "types": "./dist/src/collection-feeds.d.ts", + "default": "./dist/src/collection-feeds.js" + }, + "./collection-feed-route": { + "types": "./dist/src/collection-feed-route.d.ts", + "default": "./dist/src/collection-feed-route.js" + }, + "./collection-feed-middleware": { + "types": "./dist/src/collection-feed-middleware.d.ts", + "default": "./dist/src/collection-feed-middleware.js" + }, "./taxonomy": { "types": "./dist/src/taxonomy.d.ts", "default": "./dist/src/taxonomy.js" diff --git a/packages/astro-renderer/src/collection-feed-middleware.ts b/packages/astro-renderer/src/collection-feed-middleware.ts new file mode 100644 index 00000000..2237a55f --- /dev/null +++ b/packages/astro-renderer/src/collection-feed-middleware.ts @@ -0,0 +1,22 @@ +/** + * RFC-0042 §5 (added by the `cloudhal()` integration on every target): a collection's feed at + * `/rss.xml`, served only where the theme's own routes answered 404 + * (`collection-feed-route.ts`). + */ +import config from "virtual:cloudhal/config"; +import { serveCollectionFeed } from "@cloudhal/astro-renderer/collection-feed-route"; +import { getSource } from "@cloudhal/astro-renderer/runtime"; + +interface MiddlewareContext { + request: Request; +} + +export async function onRequest( + context: MiddlewareContext, + next: () => Promise, +): Promise { + const response = await next(); + return serveCollectionFeed(context.request, response, (headers) => + getSource(config, { request: context.request, response: { headers } }), + ); +} diff --git a/packages/astro-renderer/src/collection-feed-route.ts b/packages/astro-renderer/src/collection-feed-route.ts new file mode 100644 index 00000000..92362429 --- /dev/null +++ b/packages/astro-renderer/src/collection-feed-route.ts @@ -0,0 +1,92 @@ +/** + * RFC-0042 §5: the collection feed route. `collectionFeedXml` builds a collection's RSS 2.0 feed in + * a locale on top of `rssXml` (the site feed's rules: published, listed, `feed: false` left out); + * `serveCollectionFeed` turns a theme's 404 for `/rss.xml` into that feed. The `cloudhal()` + * integration runs it as a middleware on every target (`collection-feed-middleware.ts`), so a + * theme with its own route for the path keeps it, and old theme copies gain feeds on upgrade. + * Static builds write the files because `staticPaths().feeds` lists these paths for the starter's + * `[...path]/rss.xml` route, whose 404 this replaces. + */ +import { collectionLabel } from "@cloudhal/core-ts"; +import { localeUrl } from "@cloudhal/core-ts/locale-paths"; +import { addCacheTags, listCacheTag } from "./cache-tags.ts"; +import { collectionFeedPath, collectionForFeedPath } from "./collection-feeds.ts"; +import { rssXml } from "./feeds.ts"; +import { localeRequest } from "./locales.ts"; +import { loadAssetIndex } from "./media.ts"; +import { siteOrigin } from "./seo.ts"; +import type { GraphSource } from "./source.ts"; +import { loadTaxonomyIndex } from "./taxonomy.ts"; + +const SUFFIX = "/rss.xml"; + +/** + * A collection's feed in `locale`: its newest listed entries (expired and gated ones left out by + * the source), titled after its plural label, linking to the page at its prefix when there is one. + */ +export async function collectionFeedXml( + source: GraphSource, + input: { origin: string; locale: string; collection: string }, +): Promise { + const { origin, locale, collection } = input; + const site = await source.site(); + const feedPath = collectionFeedPath(site, collection) ?? `/${collection}${SUFFIX}`; + const prefix = feedPath.slice(0, -SUFFIX.length); + const index = await source.pageByPath(prefix, locale); + const description = index?.seo?.description; + return rssXml({ + site, + origin, + locale, + linkOrigin: localeUrl(site, locale, "", origin), + entries: await source.entries(collection, locale), + feedPath, + title: `${collectionLabel(site, collection, locale).plural} · ${site.name}`, + linkPath: index ? prefix : "/", + taxonomy: await loadTaxonomyIndex(source, locale), + assets: await loadAssetIndex(source), + ...(description ? { description } : {}), + }); +} + +/** + * Opens the request's source and lets it set caching headers on `headers` (the runtime's + * `getSource(config, { request, response: { headers } })`); undefined when unpublished. + */ +export type FeedSourceOpener = (headers: Headers) => Promise; + +/** + * The middleware's decision: `response` unchanged unless it is a 404 to a `GET`/`HEAD` of a + * collection's feed path (in the path's or host's locale), which is then answered with the feed, + * the source's cache headers and the site's list tag (like `rss.xml`). + */ +export async function serveCollectionFeed( + request: Request, + response: Response, + open: FeedSourceOpener, +): Promise { + if (response.status !== 404 || (request.method !== "GET" && request.method !== "HEAD")) + return response; + const url = new URL(request.url); + if (!url.pathname.endsWith(SUFFIX)) return response; + const headers = new Headers(); + const source = await open(headers); + if (!source) return response; + const site = await source.site(); + const lr = localeRequest(site, url.host, url.pathname); + if (lr.redirect !== undefined) return response; + const collection = + collectionForFeedPath(site, lr.path) ?? + collectionForFeedPath(site, lr.path, await source.allEntries(lr.locale)); + if (!collection) return response; + await response.body?.cancel(); + headers.set("content-type", "application/rss+xml; charset=utf-8"); + // Lists a collection's newest entries: any entry change purges it, as it does the site feed. + addCacheTags(headers, [listCacheTag(site.id)]); + const body = await collectionFeedXml(source, { + origin: siteOrigin(site, url), + locale: lr.locale, + collection, + }); + return new Response(body, { status: 200, headers }); +} diff --git a/packages/astro-renderer/src/collection-feeds.ts b/packages/astro-renderer/src/collection-feeds.ts new file mode 100644 index 00000000..dfbae1dd --- /dev/null +++ b/packages/astro-renderer/src/collection-feeds.ts @@ -0,0 +1,87 @@ +/** + * RFC-0042 §5: per-collection feeds. Each collection with a permalink prefix (RFC-0038: `/events` + * for `/events/{slug}`) has an RSS feed at `/rss.xml` (and `//rss.xml`), + * listing its newest listed entries: expiry and gates apply as in the site feed, `feed: false` + * answers 404. The renderer serves it after a theme's routes answered 404 + * (`collection-feed-route.ts`), so old theme copies get it and a theme's own route wins. + * + * This module holds the pure parts: which collections have a feed, their paths, and the + * `` a theme puts in its head. Worker-safe. + */ +import type { ContentEntry, Site } from "@cloudhal/core-contracts"; +import { + BUILT_IN_COLLECTION, + collectionLabel, + collectionPermalink, + collectionSettings, + permalinkPrefix, +} from "@cloudhal/core-ts"; +import { localeUrl } from "@cloudhal/core-ts/locale-paths"; + +const SUFFIX = "/rss.xml"; + +/** + * Where a collection's feed is served (unlocalized), or undefined when it has none: `feed: false`, + * or a permalink without a fixed prefix (`/{slug}`, whose feed would be the site's `/rss.xml`). + */ +export function collectionFeedPath(site: Site, collection: string): string | undefined { + if (!collectionSettings(site, collection).feed) return undefined; + const prefix = permalinkPrefix(collectionPermalink(site, collection)); + if (prefix === "" || prefix === "/" || prefix.includes("{")) return undefined; + return `${prefix}${SUFFIX}`; +} + +/** + * The collections with a feed: declared ones, the built-in `posts`, and collections inferred from + * `entries` (the entries a source lists), each in that order, without duplicates. + */ +export function feedCollections(site: Site, entries: readonly ContentEntry[] = []): string[] { + const keys = new Set([ + ...Object.keys(site.collections ?? {}), + BUILT_IN_COLLECTION, + ...entries.map((e) => e.collection), + ]); + return [...keys].filter((key) => collectionFeedPath(site, key) !== undefined); +} + +/** Every collection feed path (unlocalized), for static builds (`staticPaths().feeds`). */ +export function collectionFeedPaths(site: Site, entries: readonly ContentEntry[] = []): string[] { + return feedCollections(site, entries).map((key) => collectionFeedPath(site, key) as string); +} + +/** + * The collection a request path (without its locale prefix) addresses as a feed. `entries` are + * the entries the source lists, so an undeclared collection has a feed only once it has entries. + */ +export function collectionForFeedPath( + site: Site, + path: string, + entries: readonly ContentEntry[] = [], +): string | undefined { + if (!path.endsWith(SUFFIX)) return undefined; + return feedCollections(site, entries).find((key) => collectionFeedPath(site, key) === path); +} + +/** A feed for ``, as `HeadMeta.feeds` holds it. */ +export interface FeedLink { + readonly title: string; + readonly href: string; +} + +/** + * The `` of a collection's feed in `locale` (default: the site's), on the + * locale's URL (its prefix or its own domain): titled ` · `. Undefined + * when the collection has no feed (`feed: false`, no prefix). Add it to the head after + * `localizeHead`, whose feed links it would localize a second time. + */ +export function collectionFeedLink( + site: Site, + collection: string, + origin: string, + locale: string = site.defaultLocale, +): FeedLink | undefined { + const path = collectionFeedPath(site, collection); + if (!path) return undefined; + const { plural } = collectionLabel(site, collection, locale); + return { title: `${plural} · ${site.name}`, href: localeUrl(site, locale, path, origin) }; +} diff --git a/packages/astro-renderer/src/index.ts b/packages/astro-renderer/src/index.ts index b52c15d1..af29bf63 100644 --- a/packages/astro-renderer/src/index.ts +++ b/packages/astro-renderer/src/index.ts @@ -8,6 +8,7 @@ */ export * from "./archive.ts"; export * from "./cache-tags.ts"; +export * from "./collection-feeds.ts"; export * from "./entry-query.ts"; export * from "./fields.ts"; export * from "./gates.ts"; diff --git a/packages/astro-renderer/src/integration.ts b/packages/astro-renderer/src/integration.ts index f044a822..d4189b5d 100644 --- a/packages/astro-renderer/src/integration.ts +++ b/packages/astro-renderer/src/integration.ts @@ -61,6 +61,26 @@ const RESOLVED_CLIENT_ID = `\0${CLIENT_ID}`; const COMMENTS_CLIENT_ID = "virtual:cloudhal/comments-client"; const RESOLVED_COMMENTS_CLIENT_ID = `\0${COMMENTS_CLIENT_ID}`; +/** The virtual modules: the project config, and in dev the built analytics and comments clients. */ +function configPlugin(modules: { config: unknown; client: unknown; comments: unknown }) { + const ids: Record = { + [VIRTUAL_ID]: RESOLVED_ID, + [CLIENT_ID]: RESOLVED_CLIENT_ID, + [COMMENTS_CLIENT_ID]: RESOLVED_COMMENTS_CLIENT_ID, + }; + const bodies: Record = { + [RESOLVED_ID]: modules.config, + [RESOLVED_CLIENT_ID]: modules.client, + [RESOLVED_COMMENTS_CLIENT_ID]: modules.comments, + }; + return { + name: "cloudhal-config", + resolveId: (id: string) => (Object.hasOwn(ids, id) ? ids[id] : undefined), + load: (id: string) => + Object.hasOwn(bodies, id) ? `export default ${JSON.stringify(bodies[id])};` : undefined, + }; +} + export function cloudhal(config: CloudhalRendererConfig): AstroIntegration { let resolved: CloudhalRendererConfig = config; let client: AnalyticsClient | undefined; @@ -120,28 +140,19 @@ export function cloudhal(config: CloudhalRendererConfig): AstroIntegration { order: "pre", }); } + // RFC-0042 §5: a collection's feed where the theme's routes answer 404, on every target. + addMiddleware({ + entrypoint: "@cloudhal/astro-renderer/collection-feed-middleware", + order: "pre", + }); updateConfig({ vite: { plugins: [ - { - name: "cloudhal-config", - resolveId: (id: string) => - id === VIRTUAL_ID - ? RESOLVED_ID - : id === CLIENT_ID - ? RESOLVED_CLIENT_ID - : id === COMMENTS_CLIENT_ID - ? RESOLVED_COMMENTS_CLIENT_ID - : undefined, - load: (id: string) => - id === RESOLVED_ID - ? `export default ${JSON.stringify(resolved)};` - : id === RESOLVED_CLIENT_ID - ? `export default ${JSON.stringify(cloudflare ? null : (client ?? null))};` - : id === RESOLVED_COMMENTS_CLIENT_ID - ? `export default ${JSON.stringify(cloudflare ? null : (comments ?? null))};` - : undefined, - }, + configPlugin({ + config: resolved, + client: cloudflare ? null : (client ?? null), + comments: cloudflare ? null : (comments ?? null), + }), ], // Native module must stay external in Node SSR; workerd never loads it. ...(cloudflare ? {} : { ssr: { external: ["better-sqlite3"] } }), diff --git a/packages/astro-renderer/src/preview.ts b/packages/astro-renderer/src/preview.ts index 86465a0b..ffbcce43 100644 --- a/packages/astro-renderer/src/preview.ts +++ b/packages/astro-renderer/src/preview.ts @@ -1,5 +1,6 @@ import type { ContentEntry, + ContentListSection, ImageRef, Page, Section, @@ -18,6 +19,7 @@ import { viewPagePath, } from "./archive.ts"; import { renderComponentSection } from "./blocks.ts"; +import { evaluateEntryQuery, listsByField, sectionQuery } from "./entry-query.ts"; import { formatGateTime, gateNotice } from "./gates.ts"; import { renderBody, renderEntryArticle, renderMarkdown, safeHref, safeSrc } from "./markdown.ts"; import { assetCaption, imageAttributes, imgHtml, type MediaContext } from "./media.ts"; @@ -83,10 +85,55 @@ export function previewCollections(sections: readonly Section[]): string[] { * (`.ch-*` classes, `data-section-id`), so agents can preview a change without a build and the * markup matches what the site will render. */ +/** + * What a preview needs to list a `contentList` section by field (RFC-0042 §4): the site (field + * definitions, list fields, time zone) and the instant `now` compares with (default: the clock). + */ +export interface PreviewListing { + readonly site: Site; + readonly now?: number | undefined; +} + +/** + * A `contentList` section's entries in a preview: with `where`/`orderBy` (and `listing`), through + * the live renderer's evaluator (`evaluateEntryQuery`); otherwise the collection by the section's + * `sort`. A term `filter` is applied on the site only; a preview ignores it. + */ +function previewList( + section: ContentListSection, + entries: Record, + listing: PreviewListing | undefined, +): ContentEntry[] { + const all = entries[section.collection] ?? []; + if (listing && listsByField(section)) { + const { filter: _ignored, ...query } = sectionQuery(section); + return evaluateEntryQuery({ + site: listing.site, + candidates: all, + published: all, + taxonomies: [], + terms: [], + query, + now: listing.now ?? Date.now(), + }).entries; + } + return [...all] + .sort((a, b) => + section.sort === "title" + ? a.title.localeCompare(b.title) + : section.sort === "oldest" + ? (a.publishedAt ?? a.createdAt).localeCompare(b.publishedAt ?? b.createdAt) + : (b.publishedAt ?? b.createdAt).localeCompare(a.publishedAt ?? a.createdAt), + ) + .slice(0, section.limit); +} + export function renderSectionHtml( section: Section, entries: Record = {}, media?: MediaContext, + /** RFC-0042: lets a `contentList` with `where`/`orderBy` list as the site does. */ + listing?: PreviewListing, ): string { switch (section.kind) { case "hero": @@ -115,15 +162,7 @@ export function renderSectionHtml( }`; } case "contentList": { - const list = [...(entries[section.collection] ?? [])] - .sort((a, b) => - section.sort === "title" - ? a.title.localeCompare(b.title) - : section.sort === "oldest" - ? (a.publishedAt ?? a.createdAt).localeCompare(b.publishedAt ?? b.createdAt) - : (b.publishedAt ?? b.createdAt).localeCompare(a.publishedAt ?? a.createdAt), - ) - .slice(0, section.limit); + const list = previewList(section, entries, listing); const items = list .map( (e) => @@ -242,7 +281,7 @@ export function renderPagePreview( const ctx = previewMedia(site, media); const body = gateNoticeHtml(site, page, now) + - page.sections.map((s) => renderSectionHtml(s, entries, ctx)).join("\n"); + page.sections.map((s) => renderSectionHtml(s, entries, ctx, { site, now })).join("\n"); const title = page.seo?.title ?? page.title; return { title, path: page.path, html: document(site, title, page.seo?.description, body) }; } diff --git a/packages/astro-renderer/src/routing.ts b/packages/astro-renderer/src/routing.ts index a8c28f6e..e108c55c 100644 --- a/packages/astro-renderer/src/routing.ts +++ b/packages/astro-renderer/src/routing.ts @@ -18,6 +18,7 @@ import { parseArchivePath, termForSlug, } from "./archive.ts"; +import { collectionFeedPaths } from "./collection-feeds.ts"; import { routableEntries } from "./entry-views.ts"; import { PREVIEW_PARAM } from "./resolver.ts"; import { type SearchView, searchRequest } from "./search.ts"; @@ -261,7 +262,7 @@ export function redirectPaths(redirects: readonly Redirect[], served: readonly s export interface StaticPaths { /** For the catch-all page route: pages, entries, archive pages, then paths that redirect. */ readonly pages: string[]; - /** For the term feed route (`/rss.xml`). */ + /** For the term feed route (`/rss.xml`), then each collection's (`/rss.xml`). */ readonly feeds: string[]; } @@ -293,7 +294,14 @@ export async function staticPaths( ) .filter((p) => !taken.has(p)); const served = [...base, ...archivePages]; - const feeds = archives.flatMap((a) => (a.feedPath ? [a.feedPath] : [])); + // RFC-0042 §5: each collection's feed too (the starter's feed route answers 404 for it, and the + // renderer's middleware answers with the feed), so old theme copies write it as well. + const feeds = [ + ...new Set([ + ...archives.flatMap((a) => (a.feedPath ? [a.feedPath] : [])), + ...collectionFeedPaths(site, entries), + ]), + ]; // `…/page/1` answers with a 301 to the archive root (a meta-refresh page in static builds). const firstPages = archives.map((a) => `${a.path}/page/1`); return { diff --git a/packages/astro-renderer/src/seo.ts b/packages/astro-renderer/src/seo.ts index e372045b..d551545d 100644 --- a/packages/astro-renderer/src/seo.ts +++ b/packages/astro-renderer/src/seo.ts @@ -10,6 +10,8 @@ import { inlineMarkdownText, markdownText, safeSrc } from "./markdown.ts"; import { assetAlt, imageObject, type MediaContext, socialImageUrl } from "./media.ts"; import { type EntryTaxonomy, entryTaxonomy, type ShownTerm } from "./taxonomy.ts"; +/** RFC-0042 §5: the `` of a collection's feed. */ +export { collectionFeedLink, type FeedLink } from "./collection-feeds.ts"; /** RFC-0042 §2: a collection's JSON-LD preset in place of the generic `BlogPosting`. */ export { entryStructuredData, diff --git a/packages/astro-renderer/test/collection-feeds.test.ts b/packages/astro-renderer/test/collection-feeds.test.ts new file mode 100644 index 00000000..0357900f --- /dev/null +++ b/packages/astro-renderer/test/collection-feeds.test.ts @@ -0,0 +1,189 @@ +/** + * RFC-0042 §5: per-collection feeds, served by the renderer's middleware only where the theme's + * routes answered 404, over the stage 3 fixture (an `unlist` events collection, a `hide` offers + * collection) published in a real store. + */ +import type { Site } from "@cloudhal/core-contracts"; +import { beforeAll, describe, expect, it } from "vitest"; +import { collectionFeedXml, serveCollectionFeed } from "../src/collection-feed-route.ts"; +import { + collectionFeedLink, + collectionFeedPath, + collectionForFeedPath, + feedCollections, +} from "../src/collection-feeds.ts"; +import { staticPaths } from "../src/routing.ts"; +import { type GraphSource, SnapshotSource } from "../src/source.ts"; +import { EVENTS, HOUR, type Stage3, stage3, T } from "./stage3-fixture.ts"; + +const ORIGIN = "https://feeds.example"; +const notFound = () => new Response("not found", { status: 404 }); +const get = (path: string, method = "GET") => new Request(`${ORIGIN}${path}`, { method }); + +/** The middleware over `source`, with the site tag a published Worker response carries. */ +function serve( + source: GraphSource | undefined, + path: string, + response = notFound(), + method?: string, +) { + return serveCollectionFeed(get(path, method), response, async (headers) => { + headers.set("cache-tag", "cloudhal-site-test"); + headers.set("cloudflare-cdn-cache-control", "public, max-age=60"); + return source; + }); +} + +describe("collection feeds (RFC-0042 §5)", () => { + let fx: Stage3; + let published: SnapshotSource; + + beforeAll(async () => { + fx = await stage3(); + published = (await SnapshotSource.open(fx.store, fx.snapshotId)).at(T); + }); + + it("serves a collection's feed where the theme answered 404, with the site feed's headers", async () => { + const res = await serve(published, "/events/rss.xml"); + expect(res.status).toBe(200); + expect(res.headers.get("content-type")).toBe("application/rss+xml; charset=utf-8"); + expect(res.headers.get("cache-tag")).toBe(`cloudhal-site-test,cloudhal-list-${fx.site.id}`); + expect(res.headers.get("cloudflare-cdn-cache-control")).toBe("public, max-age=60"); + const xml = await res.text(); + expect(xml).toContain("Events · Stage Three"); + expect(xml).toContain(`${ORIGIN}/events/soon-gig`); + expect(xml).toContain(`${ORIGIN}/events/undated-gig`); + // only this collection's entries + expect(xml).not.toContain("/posts/"); + expect(xml).not.toContain("/offers/"); + // an event past its `unlist` expiry leaves the feed (its page stays) + expect(xml).not.toContain("ZQXU7"); + expect(xml).not.toContain("/events/past-gig"); + }); + + it("leaves the theme's own answer alone", async () => { + const own = new Response("theme", { status: 200 }); + expect(await serve(published, "/events/rss.xml", own)).toBe(own); + const gone = new Response(null, { status: 410 }); + expect(await serve(published, "/events/rss.xml", gone)).toBe(gone); + const redirect = new Response(null, { status: 301, headers: { location: "/x" } }); + expect(await serve(published, "/events/rss.xml", redirect)).toBe(redirect); + }); + + it("answers only feed paths of collections with a feed, to GET and HEAD", async () => { + for (const path of ["/events", "/events/feed.xml", "/nope/rss.xml", "/events/x/rss.xml"]) { + const miss = notFound(); + expect(await serve(published, path, miss), path).toBe(miss); + } + const post = notFound(); + expect(await serve(published, "/events/rss.xml", post, "POST")).toBe(post); + expect((await serve(published, "/events/rss.xml", notFound(), "HEAD")).status).toBe(200); + // an unpublished site keeps the 404 + const unpublished = notFound(); + expect(await serve(undefined, "/events/rss.xml", unpublished)).toBe(unpublished); + // the built-in collection has one too + expect((await serve(published, "/posts/rss.xml")).status).toBe(200); + }); + + it("drops entries past a `hide` expiry when their gate passes", async () => { + const today = await (await serve(published, "/offers/rss.xml")).text(); + expect(today).not.toContain("ZQXU8"); + expect(today).toContain("ZQXU9"); + // 21:00 UTC is midnight in Istanbul: the offer valid "all of today" is hidden then + const tonight = await (await serve(published.at(T + 9 * HOUR), "/offers/rss.xml")).text(); + expect(tonight).not.toContain("ZQXU9"); + expect(tonight).toContain("Offers · Stage Three"); + }); + + it("answers 404 for a collection with `feed: false`, and lists no path or link for it", async () => { + const off = await stage3({ events: { ...EVENTS, feed: false } }); + const source = (await SnapshotSource.open(off.store, off.snapshotId)).at(T); + const miss = notFound(); + expect(await serve(source, "/events/rss.xml", miss)).toBe(miss); + expect(collectionFeedLink(off.site, "events", ORIGIN)).toBeUndefined(); + const { feeds } = await staticPaths(source); + expect(feeds).not.toContain("/events/rss.xml"); + expect(feeds).toContain("/offers/rss.xml"); + }); + + it("lists every collection feed for static builds", async () => { + const { feeds } = await staticPaths(published); + expect(feeds).toEqual( + expect.arrayContaining(["/events/rss.xml", "/offers/rss.xml", "/posts/rss.xml"]), + ); + expect(new Set(feeds).size).toBe(feeds.length); + }); + + it("serves a locale's collection feed under its prefix, in its language", async () => { + const fy = await stage3(); + const site = await fy.store.updateSite({ ...fy.site, locales: ["en", "es"] }); + const [hello] = await fy.store.listContentEntries(site.id, "posts"); + await fy.store.createContentEntry({ + siteId: site.id, + collection: "events", + slug: "fiesta", + locale: "es", + status: "published", + title: "Fiesta ZQXU5", + body: { format: "markdown", value: "Una fiesta." }, + fields: { venue: "Sala" }, + tags: [], + ...(hello?.terms ? { terms: hello.terms } : {}), + author: { kind: "human", id: "user_stage3000001", name: "Halil" }, + } as never); + const snap = await fy.store.createSnapshot(site.id, { + actor: { kind: "human", id: "user_stage3000001", name: "Halil" }, + reason: "publish", + }); + await fy.store.publish((await fy.store.getSite(site.id)) as Site, snap.id); + const source = (await SnapshotSource.open(fy.store, snap.id)).at(T); + const es = await (await serve(source, "/es/events/rss.xml")).text(); + expect(es).toContain("es"); + expect(es).toContain(`${ORIGIN}/es/events/fiesta`); + expect(es).toContain(` { + const site = fx.site; + expect(collectionFeedLink(site, "events", ORIGIN)).toEqual({ + title: "Events · Stage Three", + href: `${ORIGIN}/events/rss.xml`, + }); + // an undeclared collection is labelled by its key + expect(collectionFeedLink(site, "notes", ORIGIN)?.title).toBe("notes · Stage Three"); + // a permalink without a prefix has no feed of its own (it would be the site's /rss.xml) + const rooted = { + ...site, + collections: { ...site.collections, pages: { permalink: "/{slug}" } }, + } as Site; + expect(collectionFeedPath(rooted, "pages")).toBeUndefined(); + expect(feedCollections(rooted)).not.toContain("pages"); + // an undeclared collection has a feed once it has entries + expect(collectionForFeedPath(site, "/notes/rss.xml")).toBeUndefined(); + expect(collectionForFeedPath(site, "/notes/rss.xml", [{ collection: "notes" } as never])).toBe( + "notes", + ); + }); + + it("titles the channel after the collection and links the page at its prefix", async () => { + const xml = await collectionFeedXml(published, { + origin: ORIGIN, + locale: "en", + collection: "events", + }); + // no page at /events in the fixture: the channel links the home page + expect(xml).toContain(`${ORIGIN}/`); + expect(xml).toContain("Stage Three"); + }); +}); diff --git a/packages/astro-renderer/test/derived-gates-fixture.ts b/packages/astro-renderer/test/derived-gates-fixture.ts new file mode 100644 index 00000000..802bc632 --- /dev/null +++ b/packages/astro-renderer/test/derived-gates-fixture.ts @@ -0,0 +1,119 @@ +/** + * RFC-0042 §3 route sweep fixture on the real clock (the Site Worker's sweep on workerd, + * `packages/cli/test/cloudflare-gates.test.ts`), applied on top of the starter seed relative to an + * instant `t`: + * + * - `shows` (datetime `endsAt`, `expiry: endsAt, unlist`): one show that ended at t−1h (`ZQXUA1`), + * one ending three days later (`ZQXUA2`); + * - `deals` (datetime `validUntil`, `expiry: validUntil, hide`): one deal that ended at t−1h + * (`ZQXUA3`), one valid for three more days (`ZQXUA4`); + * - a `/shows` page listing the shows. + * + * No list fields, so the gate index gains only the derived instants (t−1h, t+3d): the RFC-0008 + * fixture's next gate (t+1h) stays the next one. Markers follow `gates-fixture.ts` (a `U` keeps them + * out of generated ids). + */ +import type { ContentEntry, FieldDefinition } from "@cloudhal/core-contracts"; +import type { SiteGraphStore } from "@cloudhal/core-ts"; + +const HOUR = 3_600_000; +const writer = { kind: "human" as const, id: "user_derived00001", name: "Gate Keeper" }; + +/** An entry the sweep watches past its derived gate, with every string that would give it away. */ +export interface DerivedWatched { + readonly name: string; + readonly action: "unlist" | "hide"; + readonly entry: ContentEntry; + readonly path: string; + /** Id, path, title marker. */ + readonly leaks: readonly string[]; +} + +export interface DerivedGatesFixture { + /** Past their gate at any instant after `t`. */ + readonly watched: readonly DerivedWatched[]; + /** Still open for three days: listed, in the collection feeds. */ + readonly open: readonly ContentEntry[]; +} + +const datetime = (key: string, label: string): [FieldDefinition] => [ + { key, type: "datetime", labels: { en: { label } }, localized: false }, +]; + +export async function applyDerivedGatesFixture( + store: SiteGraphStore, + siteId: string, + t: number, +): Promise { + const site = await store.getSite(siteId); + const hello = (await store.listContentEntries(siteId)).find((e) => e.slug === "hello-world"); + if (!site || !hello) throw new Error("fixture: seed missing"); + await store.updateSite({ + ...site, + collections: { + ...site.collections, + shows: { + labels: { en: { singular: "Show", plural: "Shows" } }, + fields: datetime("endsAt", "Ends"), + expiry: { field: "endsAt", action: "unlist" }, + }, + deals: { + labels: { en: { singular: "Deal", plural: "Deals" } }, + fields: datetime("validUntil", "Valid until"), + expiry: { field: "validUntil", action: "hide" }, + }, + }, + }); + const at = (ms: number) => ({ at: new Date(ms).toISOString(), timeZone: "UTC" }); + const entry = (collection: string, slug: string, title: string, fields: object) => + store.createContentEntry({ + siteId, + collection, + slug, + locale: site.defaultLocale, + status: "published", + title, + summary: `About ${title}`, + body: { format: "markdown", value: `All about ${title}.` }, + fields, + tags: [], + ...(hello.terms ? { terms: hello.terms } : {}), + author: writer, + } as never); + const endedShow = await entry("shows", "ended-show", "Show ZQXUA1", { endsAt: at(t - HOUR) }); + const nextShow = await entry("shows", "next-show", "Show ZQXUA2", { endsAt: at(t + 72 * HOUR) }); + const endedDeal = await entry("deals", "ended-deal", "Deal ZQXUA3", { + validUntil: at(t - HOUR), + }); + const openDeal = await entry("deals", "open-deal", "Deal ZQXUA4", { + validUntil: at(t + 72 * HOUR), + }); + await store.createPage((await store.getSite(siteId)) ?? site, { + siteId, + path: "/shows", + title: "Shows", + locale: site.defaultLocale, + status: "published", + sections: [ + { + id: "sec_showlist0001", + kind: "contentList", + collection: "shows", + limit: 10, + sort: "newest", + }, + ], + } as never); + const watch = (name: string, action: DerivedWatched["action"], e: ContentEntry) => { + const path = `/${e.collection}/${e.slug}`; + const marker = /ZQXU\w+/.exec(e.title)?.[0] as string; + return { name, action, entry: e, path, leaks: [e.id, path, marker] }; + }; + return { + watched: [ + watch("show past its unlist expiry", "unlist", endedShow), + watch("deal past its hide expiry", "hide", endedDeal), + ], + open: [nextShow, openDeal], + }; +} diff --git a/packages/astro-renderer/test/derived-gates-sweep.test.ts b/packages/astro-renderer/test/derived-gates-sweep.test.ts index 3336461e..f1a53aed 100644 --- a/packages/astro-renderer/test/derived-gates-sweep.test.ts +++ b/packages/astro-renderer/test/derived-gates-sweep.test.ts @@ -3,7 +3,7 @@ * stage 3 fixture published once, swept at every derived instant −1 ms, exactly and +1 ms through * every route the blog starter serves. An entry past a `hide` expiry must not appear in any public * output (id, path, title); an entry past an `unlist` expiry keeps its own page but must not - * appear on any other route, list, feed or the sitemap. + * appear on any other route, list, feed (the collections' own feeds included) or the sitemap. */ import type { ContentEntry } from "@cloudhal/core-contracts"; import { entryPath } from "@cloudhal/core-ts"; @@ -63,6 +63,9 @@ describe("route sweep: derived gates (RFC-0042 §3)", () => { const label = `${new Date(now).toISOString()}`; const source = published.at(now); const swept = await sweep(source, candidates, [], fx.store); + // RFC-0042 §5: the collections' own feeds are swept too + for (const feed of ["/events/rss.xml", "/offers/rss.xml"]) + expect(swept.routes.get(feed), `${label}: ${feed}`).toBe("collection-feed"); for (const w of watched) { const past = !(now < w.at); const route = swept.routes.get(w.path); diff --git a/packages/astro-renderer/test/preview.test.ts b/packages/astro-renderer/test/preview.test.ts index 69c6980f..d8fb7635 100644 --- a/packages/astro-renderer/test/preview.test.ts +++ b/packages/astro-renderer/test/preview.test.ts @@ -167,3 +167,41 @@ describe("section URLs (H3)", async () => { expect(safeSrc("mailto:a@b.c")).toBeUndefined(); }); }); + +describe("previews list by field like the site (RFC-0042 §4)", async () => { + const { stage3, T, HOUR } = await import("./stage3-fixture.ts"); + const { contentListEntries } = await import("../src/entry-query.ts"); + const { SnapshotSource } = await import("../src/source.ts"); + const fx = await stage3(); + const events = { events: await fx.store.listContentEntries(fx.site.id, "events") }; + const section = fx.home.sections.find((s) => s.id === "sec_upcoming0001"); + if (section?.kind !== "contentList") throw new Error("fixture: no upcoming list"); + const titles = (html: string) => + [...html.matchAll(/([^<]+)<\/a>/g)].map((m) => m[1]); + + it("applies where/orderBy in page previews: upcoming only, soonest first", () => { + const html = renderPagePreview(fx.site, fx.home, events, undefined, T).html; + const list = html.slice(html.indexOf('data-section-id="sec_upcoming0001"')); + const shown = titles(list.slice(0, list.indexOf(""))); + // an entry without the field's value, or before `now`, is not upcoming + expect(shown).toEqual(["Gig soon", "Gig later"]); + // `now` is the preview's instant + const later = renderPagePreview(fx.site, fx.home, events, undefined, T + 3 * HOUR).html; + const rest = later.slice(later.indexOf('data-section-id="sec_upcoming0001"')); + expect(titles(rest.slice(0, rest.indexOf("")))).toEqual(["Gig later"]); + }); + + it("matches the live renderer's list for the same section and instant", async () => { + const source = (await SnapshotSource.open(fx.store, fx.snapshotId)).at(T); + const live = (await contentListEntries(source, [section])).sec_upcoming0001 ?? []; + const preview = titles( + renderSectionHtml(section, events, undefined, { site: fx.site, now: T }), + ); + expect(preview).toEqual(live.map((e) => e.title)); + }); + + it("keeps the old behaviour without a site: the collection by the section's sort", () => { + const shown = titles(renderSectionHtml(section, events)); + expect(shown).toHaveLength(4); + }); +}); diff --git a/packages/astro-renderer/test/sweep.ts b/packages/astro-renderer/test/sweep.ts index 80ddf3fa..7ae5cfe5 100644 --- a/packages/astro-renderer/test/sweep.ts +++ b/packages/astro-renderer/test/sweep.ts @@ -3,8 +3,12 @@ * one source, path by path. Shared by the time-gate sweep (`gates-sweep.test.ts`) and the derived * gate sweep (`derived-gates-sweep.test.ts`, RFC-0042 §3). */ +import type { Section } from "@cloudhal/core-contracts"; import type { SiteGraphStore } from "@cloudhal/core-ts"; import { archiveSummaries } from "../src/archive.ts"; +import { collectionFeedXml } from "../src/collection-feed-route.ts"; +import { collectionForFeedPath } from "../src/collection-feeds.ts"; +import { contentListEntries } from "../src/entry-query.ts"; import { archiveFeedXml, robotsTxt, rssXml, sitemapUrls, sitemapXml } from "../src/feeds.ts"; import { renderEntryArticle } from "../src/markdown.ts"; import { loadAssetIndex, type MediaContext } from "../src/media.ts"; @@ -61,13 +65,19 @@ export async function sweep( const before = out.length; const route = await resolveRoute(source, path); routes.set(path, route.kind); - if (route.kind === "page") { - const lists: Record>> = {}; - for (const s of route.page.sections) - if (s.kind === "contentList") lists[s.collection] = await source.entries(s.collection); + // RFC-0042 §5: a collection's feed answers where the starter's routes answer 404. + const feedOf = route.kind === "not-found" ? collectionForFeedPath(site, path, all) : undefined; + if (feedOf) { + routes.set(path, "collection-feed"); + out.push(await collectionFeedXml(source, { origin: ORIGIN, locale, collection: feedOf })); + } else if (route.kind === "page") { + // what `Sections` draws: a section listing by field gets its own query's entries + const lists = await contentListEntries(source, route.page.sections); + const listOf = (s: Section) => + s.kind === "contentList" && lists[s.id] ? { [s.collection]: lists[s.id] ?? [] } : lists; out.push( JSON.stringify(pageHead(site, route.page, ORIGIN, media)), - ...route.page.sections.map((s) => renderSectionHtml(s, lists, media)), + ...route.page.sections.map((s) => renderSectionHtml(s, listOf(s), media)), ); } else if (route.kind === "entry") { const terms = entryTaxonomy(index, route.entry); diff --git a/packages/cli/test/cloudflare-gates.test.ts b/packages/cli/test/cloudflare-gates.test.ts index c36bde51..6a9c1a52 100644 --- a/packages/cli/test/cloudflare-gates.test.ts +++ b/packages/cli/test/cloudflare-gates.test.ts @@ -8,6 +8,10 @@ * archives and their pages, term feeds, `rss.xml`, `sitemap.xml`, `robots.txt`, every media URL, * the 404 page, every gated node's own URL — is fetched, and no body or header may name a gated * node. Previews (live and snapshot links) show them, with the notice. + * - **Derived gates (RFC-0042 §3).** Entries past an `unlist` and a `hide` expiry (from their own + * date fields, `derived-gates-fixture.ts`) in the same sweep: the hidden one is a 404 named on no + * route, the unlisted one keeps its page and is named on no other route, the collections' own + * feeds (served by the renderer's middleware) included. * - **Cache cap (S10).** Published responses carry `max-age` up to the next gate; a gate a few * seconds ahead opens on its own, with no purge and no scheduler. * - **Features handshake (§7.7).** `GET /__cloudhal/features` answers `{ "gates": 2 }` to the admin @@ -40,6 +44,10 @@ import { createTestHarness } from "wrangler"; // The renderer is not a dependency of the CLI; its sources (and the sweep fixture) are read in place. import { staticPaths } from "../../astro-renderer/src/routing.ts"; import { SnapshotSource } from "../../astro-renderer/src/source.ts"; +import { + applyDerivedGatesFixture, + type DerivedGatesFixture, +} from "../../astro-renderer/test/derived-gates-fixture.ts"; import { applyGatesFixture, type GatesFixture } from "../../astro-renderer/test/gates-fixture.ts"; import { prepareAdminWorker } from "../src/cloudflare/admin.ts"; import { buildSiteWorker } from "../src/cloudflare/build.ts"; @@ -126,6 +134,7 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh let harness: Harness; let remote: SiteGraphStore; let fx: GatesFixture; + let dx: DerivedGatesFixture; let snapshotId: string; let candidates: string[]; @@ -165,6 +174,8 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh // The fixture is written straight into D1 once the Workers run, so `t` is "now" for them. remote = new SiteGraphStore({ ...target, gates: GATES_ENFORCED }); fx = await applyGatesFixture(remote, p.siteId, Date.now()); + // RFC-0042 §3: entries past an `unlist` and a `hide` expiry, derived from their fields. + dx = await applyDerivedGatesFixture(remote, p.siteId, Date.now()); const snap = await remote.createSnapshot(p.siteId, { actor: human, reason: "publish" }); await remote.publish((await remote.getSite(p.siteId)) as Site, snap.id); snapshotId = snap.id; @@ -177,6 +188,9 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh ...all.pages, ...all.feeds, ...fx.watched.map((w) => w.path), + ...dx.watched.map((w) => w.path), + "/shows/rss.xml", + "/deals/rss.xml", fx.tagPath, `${fx.tagPath}/rss.xml`, "/rss.xml", @@ -194,9 +208,14 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh if (p) await rm(p.tmp, { recursive: true, force: true }); }); - /** Fetches every candidate; returns every body and header as text, and each status. */ - async function sweep(): Promise<{ text: string; status: Map }> { + /** Fetches every candidate; returns every body and header as text, each status and each route's. */ + async function sweep(): Promise<{ + text: string; + status: Map; + byRoute: Map; + }> { const status = new Map(); + const byRoute = new Map(); const parts: string[] = []; for (const route of candidates) { const res = await siteFetch(route); @@ -204,12 +223,14 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh // what the site sent, never the URL asked for (the gated URLs are among the candidates) const body = await res.text(); // Astro's 404 page echoes the path asked for ("Path: /posts/…"): that is the request, not a leak. - parts.push( + const sent = [ JSON.stringify([...res.headers]), res.status === 404 ? body.split(route).join("") : body, - ); + ]; + parts.push(...sent); + byRoute.set(route, sent.join("\n")); } - return { text: parts.join("\n"), status }; + return { text: parts.join("\n"), status, byRoute }; } it("serves no gated node on any route, and everything that is open (S9)", async () => { @@ -249,6 +270,39 @@ describe("time gates on workerd: the starter's routes, the cache cap, the handsh expect(gated.headers.get("cloudflare-cdn-cache-control")).toBe("no-store"); }); + it("derived gates: a hidden entry is gone everywhere, an unlisted one keeps only its page (RFC-0042 §3)", async () => { + const { text, status, byRoute } = await sweep(); + for (const w of dx.watched) { + if (w.action === "hide") { + expect(status.get(w.path), w.name).toBe(404); + for (const leak of w.leaks) + expect(text.includes(leak), `${w.name} leaks "${leak}"`).toBe(false); + continue; + } + expect(status.get(w.path), w.name).toBe(200); + expect(byRoute.get(w.path), w.name).toContain(w.leaks[2] as string); + // every other route: lists, feeds (the site's and the collections'), the sitemap, menus + for (const [route, sent] of byRoute) + if (route !== w.path) + for (const leak of w.leaks) + expect(sent.includes(leak), `${w.name} listed on ${route}: "${leak}"`).toBe(false); + } + // the open entries stay listed, in their collections' feeds (RFC-0042 §5, served by the + // renderer's middleware on the Worker) and the sitemap + for (const e of dx.open) expect(text).toContain(`/${e.collection}/${e.slug}`); + expect(byRoute.get("/shows")).toContain("/shows/next-show"); + for (const feed of ["/shows/rss.xml", "/deals/rss.xml"]) { + expect(status.get(feed), feed).toBe(200); + expect(byRoute.get(feed), feed).toContain("application/rss+xml"); + } + expect(byRoute.get("/shows/rss.xml")).toContain("/shows/next-show"); + expect(byRoute.get("/deals/rss.xml")).toContain("/deals/open-deal"); + // the hidden entry's own URL is never cached + const hidden = await siteFetch("/deals/ended-deal"); + await hidden.body?.cancel(); + expect(hidden.headers.get("cloudflare-cdn-cache-control")).toBe("no-store"); + }); + it("previews show gated nodes with a notice, never cached", async () => { const live = await mintPreviewToken(secrets.CLOUDHAL_PREVIEW_SECRET, { site: p.siteId, diff --git a/packages/cli/test/e2e.test.ts b/packages/cli/test/e2e.test.ts index 7e979447..2cc62598 100644 --- a/packages/cli/test/e2e.test.ts +++ b/packages/cli/test/e2e.test.ts @@ -168,6 +168,12 @@ describe("cloudhal init → dev → snapshot → build", () => { const edited = await (await fetch(new URL("/about", server.url))).text(); expect(edited).toContain("About (edited live) · "); expect((await fetch(new URL("/nope", server.url))).status).toBe(404); + // RFC-0042 §5: a collection's feed, answered by the renderer where the theme's routes 404 + const feed = await fetch(new URL("/posts/rss.xml", server.url)); + expect(feed.status).toBe(200); + expect(feed.headers.get("content-type")).toContain("application/rss+xml"); + expect(await feed.text()).toContain("/posts/hello-world</link>"); + expect((await fetch(new URL("/nope/rss.xml", server.url))).status).toBe(404); // the seeded category's archive is served live too (one post: thin, so noindex) const archive = await fetch(new URL("/category/news", server.url)); expect(archive.status).toBe(200); @@ -913,6 +919,23 @@ describe("cloudhal init → dev → snapshot → build", () => { expect(await read("rss.xml")).not.toContain("Retro night"); expect(await read("sitemap.xml")).not.toContain("/events/retro"); expect(await read("sitemap.xml")).toContain("/events/launch"); + // RFC-0042 §5: the collection's own feed (the renderer answers the starter route's 404), per + // locale, without the past event; the entry and listing pages link it. + const feed = await read("events", "rss.xml"); + expect(feed).toContain("<title>Launch party"); + expect(feed).toContain("/events/launch"); + expect(feed).not.toContain("Retro night"); + expect(feed).not.toContain("/posts/hello-world"); + const trFeed = await read("tr", "events", "rss.xml"); + expect(trFeed).toContain("tr"); + expect(trFeed).toContain("/tr/events/launch"); + const feedLink = + //; + expect(await read("events", "launch", "index.html")).toMatch(feedLink); + expect(listing).toMatch(feedLink); + expect(await read("tr", "events", "launch", "index.html")).toMatch( + /type="application\/rss\+xml"[^>]*href="[^"]+\/tr\/events\/rss\.xml"/, + ); // A collection without the settings renders as before: its article and its JSON-LD. const postAfter = await read("posts", "hello-world", "index.html"); expect(article(postAfter)).toBe(article(postHtml)); diff --git a/packages/cli/test/old-theme-compat.test.ts b/packages/cli/test/old-theme-compat.test.ts index a8fe3568..8f5a27b9 100644 --- a/packages/cli/test/old-theme-compat.test.ts +++ b/packages/cli/test/old-theme-compat.test.ts @@ -6,8 +6,9 @@ * pre-0.3.11 theme served empty pages. The site also has a collection with typed fields * (RFC-0041): themes from before fields render its entries and simply do not show the values. * Its collection has RFC-0042's settings too (list fields, an `unlist` expiry, an `Event` preset) - * and a page lists it by field: old themes keep the generic `BlogPosting`, and get the expiry and - * the field listing on upgrade (both live in the renderer's sources and `ContentList.astro`). + * and a page lists it by field: old themes keep the generic `BlogPosting`, and get the expiry, the + * field listing (both live in the renderer's sources and `ContentList.astro`) and the collection's + * feed (the renderer's middleware) on upgrade. */ import { spawn } from "node:child_process"; import { cp, mkdtemp, readdir, readFile, rm, symlink } from "node:fs/promises"; @@ -222,6 +223,11 @@ describe.each(FROZEN)("a theme copied at %s still builds with today's packages", expect(listing).not.toContain("/events/retro"); expect(await read("sitemap.xml")).not.toContain("/events/retro"); expect(await read("rss.xml")).not.toContain("Retro night"); + // RFC-0042 §5: the collection's own feed comes with the renderer too (the theme's feed route + // answers 404 for it, the renderer's middleware answers with the feed) + const feed = await read("events", "rss.xml"); + expect(feed).toContain("Launch party"); + expect(feed).not.toContain("Retro night"); // No page of the build is empty (a theme that throws mid-stream leaves a bare document). const files = await htmlFiles(dist); diff --git a/scripts/size-baseline.json b/scripts/size-baseline.json index b93bef79..52f2f80d 100644 --- a/scripts/size-baseline.json +++ b/scripts/size-baseline.json @@ -352,7 +352,7 @@ "params": 5 }, "packages/astro-renderer/src/integration.ts::cloudhal": { - "lines": 181 + "lines": 172 }, "packages/astro-renderer/src/preview.ts::document": { "params": 5 diff --git a/starters/blog/src/pages/[...path].astro b/starters/blog/src/pages/[...path].astro index 8c8b759c..83c3b723 100644 --- a/starters/blog/src/pages/[...path].astro +++ b/starters/blog/src/pages/[...path].astro @@ -41,6 +41,7 @@ import { getSource, peerEditions } from "@cloudhal/astro-renderer/runtime"; import { searchAvailable } from "@cloudhal/astro-renderer/search"; import { archiveHead, + collectionFeedLink, entryHead, pageHead, searchHead, @@ -359,7 +360,23 @@ const localized = localizeHead(head, site, { alternates, ...(gated?.translation ? { translation: gated.translation } : {}), }); -const meta = fallback || notFound ? { ...localized, robots: "noindex,follow" } : localized; +// RFC-0042 §5: the feed of the entry's collection, or of each collection a page lists. +const listed = notFound + ? [] + : route.kind === "entry" + ? [route.entry.collection] + : route.kind === "page" + ? route.page.sections.flatMap((s) => (s.kind === "contentList" ? [s.collection] : [])) + : []; +const feeds = [ + ...localized.feeds, + ...[...new Set(listed)].flatMap((c) => collectionFeedLink(site, c, origin, locale) ?? []), +]; +const meta = { + ...localized, + feeds, + ...(fallback || notFound ? { robots: "noindex,follow" } : {}), +}; Astro.response.headers.set("content-language", shownIn); // The pairing's fonts as a Link header too (Early Hints), not only as in . const pairing = themeVariables(site, themeSettings).attributes["data-ch-type-pairing"]; diff --git a/starters/blog/src/pages/[...path]/rss.xml.ts b/starters/blog/src/pages/[...path]/rss.xml.ts index cd4a49b6..17e6908e 100644 --- a/starters/blog/src/pages/[...path]/rss.xml.ts +++ b/starters/blog/src/pages/[...path]/rss.xml.ts @@ -19,7 +19,9 @@ const SUFFIX = "/rss.xml"; /** * Static builds: one feed per served archive of a taxonomy with `feed: true`, in every locale on - * this host, and each other locale's site feed (`//rss.xml`, RFC-0033). + * this host, and each other locale's site feed (`//rss.xml`, RFC-0033). The paths include + * each collection's feed (`/rss.xml`, RFC-0042 §5): this route answers 404 for it and the + * renderer's middleware answers with the feed. */ export async function getStaticPaths() { const source = await getSource(config);