Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions .github/workflows/lighthouse.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,16 +105,29 @@ jobs:
# scores. The full reports hold screenshots and page details of the
# protected preview, and anyone signed in to GitHub can download a
# public repository's artifacts, so the full reports stay on the
# runner. The assertion results print in the Lighthouse CI step's log.
# runner. The assertion results print in the Lighthouse CI step's log,
# but they name only the category. So this step also keeps, for each
# category, the IDs of the scored audits below 1, and prints them.
# Audit IDs are Lighthouse's own names, not page content.
- name: Keep only the scores
if: always() && steps.page.outputs.path != ''
run: |
set -Eeuo pipefail
shopt -s nullglob
mkdir -p "${RUNNER_TEMP}/lighthouse-scores"
for report in .lighthouseci/lhr-*.json; do
jq '{finalUrl, categories: (.categories | map_values({score}))}' "${report}" \
> "${RUNNER_TEMP}/lighthouse-scores/$(basename "${report}")"
jq '. as $r | {
finalUrl,
categories: (.categories | map_values({score})),
failedAudits: (.categories | map_values([
.auditRefs[] | select(.weight > 0) | .id
| select($r.audits[.].score != null and $r.audits[.].score < 1)
]))
}' "${report}" > "${RUNNER_TEMP}/lighthouse-scores/$(basename "${report}")"
jq -r '"\(.finalUrl): \(.failedAudits | to_entries
| map(select(.value != []) | "\(.key) [\(.value | join(", "))]")
| if . == [] then "no failed audits" else join("; ") end)"' \
"${RUNNER_TEMP}/lighthouse-scores/$(basename "${report}")"
done

# Uploaded when the assertions fail too.
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ How the two sites share the app:
- **Headers.** The CSP and the other security headers in `next.config.mjs` cover both sites: one policy, since both self-host their fonts. bench now sends the site's security headers, including `X-Frame-Options: DENY` and `frame-ancestors 'none'`, so no other site can embed bench's pages in a frame. `vercel.json` holds bench's cron (`/bench/api/health`) and bench's CDN rule for its landing, chart, and methodology pages on bench.vortex.dev (header rules match the path as requested, before the rewrites). bench's HTML differs between its two URL forms, so the `/bench` form gets the CDN rule only outside production (`benchPrefixCdnRules` in `next.config.mjs`): a cached `/bench` page must never answer for bench.vortex.dev.
- **Toolchain.** One package: one `package.json`, one `bun.lock`, one `bunfig.toml`, one `tsconfig.json`. Next.js 16.4.1, React 19.3 and TypeScript 7. `tsconfig.json` has every strict flag either app had, including bench's `noFallthroughCasesInSwitch` and `noUncheckedSideEffectImports`; `src/css.d.ts` types the CSS side-effect imports. The root `biome.json` lints and formats both sites: an override keeps bench's files (`src/bench`, `src/app/(bench)`, the parity and deploy checks, `vitest.config.ts`) in bench's old Prettier style (single quotes, trailing commas, 100 columns) with bench's lint rules (see "bench" below), and `bun run check:ci` checks all of it. bench's tests run on Vitest from the root `vitest.config.ts`, scoped to bench's files; the site's run on `bun test`. bench's PostCSS config sits beside its stylesheets (`src/bench/styles/postcss.config.mjs`), and Turbopack applies it to those alone: with no config at the root, the site's CSS gets Turbopack's defaults.
- **CI.** Every check covers both sites and runs on every change; nothing is split by which site a change touches. `ci.yml`'s `Lint, build, verify` runs Biome on the whole repository, one build, the typecheck, knip, the site's unit tests, bench's Vitest suite with its Postgres integration tests (testcontainers, so the runner needs Docker), the migration-runner (`db/tests`), infrastructure (`infra/tests`), parity and deploy-check tests, `bun run verify`, and the Playwright smoke specs. Dependency review, the secret scan, REUSE and the non-blocking `bun audit` run beside it. `links.yml` checks the blog's external links with lychee each week and on a PR that changes `src/content/`; it is not a required check, so a broken link on another site never blocks a merge.
- **Lighthouse and axe.** `lighthouse.yml` audits both sites on the pull request's Vercel preview, because bench renders from its database and only a deployment has one. Its jobs run in parallel, each on its own runner, since a second audit or browser on the same CPU would skew Lighthouse's timings. Each one waits for the preview (Vercel's GitHub deployment in the `Preview` environment, through the `.github/actions/preview-proxy` action) and requests its page once to get past the cold start. Five Lighthouse jobs each audit one page, named by `LHCI_PAGE`: `/`, `/blog`, the newest post, `/bench` and one bench chart page (`lighthouserc.cjs` holds the pages, the thresholds and their rationale). The `axe` job runs `tests/a11y.spec.ts` (`--grep @axe`: WCAG 2.1 A and AA, and a serious or critical violation fails). The `report` job, the required check `Lighthouse and axe`, posts one score table on the PR from the Lighthouse jobs' reports and fails unless every job passed. The preview is reached through `scripts/preview-proxy.mjs`, which adds the `VERCEL_AUTOMATION_BYPASS_SECRET` header only to requests for the preview, so no other origin a page loads from sees it. Fork and Dependabot PRs get no secrets, so the jobs are skipped for them, which counts as passing. The smoke run in `ci.yml` leaves the axe specs out (`--grep-invert @axe`). To run axe yourself: `BASE_URL=https://vortex.dev BENCH_URL=https://bench.vortex.dev bunx playwright test --grep @axe`.
- **Lighthouse and axe.** `lighthouse.yml` audits both sites on the pull request's Vercel preview, because bench renders from its database and only a deployment has one. Its jobs run in parallel, each on its own runner, since a second audit or browser on the same CPU would skew Lighthouse's timings. Each one waits for the preview (Vercel's GitHub deployment in the `Preview` environment, through the `.github/actions/preview-proxy` action) and requests its page once to get past the cold start. Five Lighthouse jobs each audit one page, named by `LHCI_PAGE`: `/`, `/blog`, the newest post, `/bench` and one bench chart page (`lighthouserc.cjs` holds the pages, the thresholds and their rationale). The `axe` job runs `tests/a11y.spec.ts` (`--grep @axe`: WCAG 2.1 A and AA, and a serious or critical violation fails). The `report` job, the required check `Lighthouse and axe`, posts one score table on the PR from the Lighthouse jobs' reports and fails unless every job passed. The preview is reached through `scripts/preview-proxy.mjs`, which adds the `VERCEL_AUTOMATION_BYPASS_SECRET` header only to requests for the preview, so no other origin a page loads from sees it. Lighthouse blocks `vercel.live` and `pusher.com`, the hosts of Vercel's preview-only Toolbar: the Toolbar can set a third-party cookie that drops best practices to 0.78 on every page. Each Lighthouse job keeps each run's category scores and the IDs of its failed audits, and prints the IDs in its log. Fork and Dependabot PRs get no secrets, so the jobs are skipped for them, which counts as passing. The smoke run in `ci.yml` leaves the axe specs out (`--grep-invert @axe`). To run axe yourself: `BASE_URL=https://vortex.dev BENCH_URL=https://bench.vortex.dev bunx playwright test --grep @axe`.
- **Deploys.** The `vortex` Vercel project builds and deploys the app, both sites, through the git integration, on every commit. `bench-verify-deploy.yml` runs on each push to `main`: it waits for that commit's production deployment, as `lighthouse.yml` waits for a preview, then runs `scripts/verify-web-deploy.mjs` against bench under `/bench` on the deployment's own URL. It runs on push rather than on `deployment_status`, so pull requests show no skipped run of it. Schema migrations go through `bench-schema-deploy.yml`, which only runs manually from `main` because the AWS role trusts nothing else.

## Architecture Overview
Expand Down
17 changes: 15 additions & 2 deletions lighthouserc.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,8 @@ const urls = picked.filter(Boolean).map((p) => `${base}${p}`);
// Accessibility and SEO were 1.00 in every run since the contrast fixes of
// 2026-10-30. Best practices was 0.96 in every run, from one audit only: off
// Vercel, the Vercel Analytics script is a 404 and logs a console error. The
// site's layout now leaves the script out there.
// site's layout now leaves the script out there. On a preview, Vercel's
// Toolbar can fail two of its audits; `collect.settings` below blocks it.
//
// So every URL gets one bar: performance at 0.9; accessibility, best
// practices and SEO at 1.00, since their audits don't depend on timing and
Expand Down Expand Up @@ -145,7 +146,19 @@ module.exports = {
settings: {
preset: "desktop",
throttlingMethod: "simulate",
skipAudits: ["uses-http2"]
skipAudits: ["uses-http2"],
// Vercel adds its Toolbar to previews only, from vercel.live, which
// uses pusher.com. The proxy's `x-vercel-skip-toolbar` header does not
// always keep it out, and the Toolbar can set a third-party cookie.
// That fails the third-party-cookies and inspector-issues audits on
// every page. On 2026-11-20, best practices fell to 0.78 on all five
// pages of a preview in two attempts, and a later run of the same PR
// scored 1.00. A local build scored 1.00 too. CI kept only category
// scores then, so it did not record which audits failed. The Toolbar
// is not part of the app and production never loads it, so the audit
// blocks both hosts. Lighthouse ignores a blocked request in
// errors-in-console.
blockedUrlPatterns: ["*vercel.live*", "*pusher.com*"]
}
},
assert: {
Expand Down
Loading