Skip to content

perf(deploy): remove ineffective Docker cache pulls, bootstrap buildx early, and use official Hugo image - #6412

Merged
another-rex merged 1 commit into
google:masterfrom
another-rex:build-stage-build-startup
Nov 20, 2026
Merged

another-rex merged 1 commit into
google:masterfrom
another-rex:build-stage-build-startup

Conversation

@another-rex

@another-rex another-rex commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Overview

Removes startup contention and wasted pulls in deployment/build-and-stage.yaml:

  • Bootstraps osv-builder (buildx) immediately at build start (waitFor: ['-'], --bootstrap), avoiding a ~56s stall inside build-importer.
  • Removes 22 ineffective pull-* steps that caused a ~48s Docker daemon bottleneck at t=11s.
  • Switches build-hugo from the multi-GB gcr.io/oss-vdb/ci image (107–121s pull for a 99 ms Hugo run) to the official digest-pinned ghcr.io/gohugoio/hugo:v0.170.1 image.

(AI-assisted PR — part of a series of independent low-effort build-and-stage optimizations.)

Details

  • Remove 16 ineffective Go buildx pull-* steps: osv-builder uses the docker-container driver, which cannot read images pulled into the host Docker daemon. Also dropped unused --cache-from flags on the 15 Go buildx targets that do not emit BUILDKIT_INLINE_CACHE=1. Kept BUILDKIT_INLINE_CACHE=1 and --cache-from on build-website (buildx reads the cache manifest directly from GCR without needing pull-website).
  • Remove 6 ineffective legacy multi-stage pull-* steps (alpine-cve-convert, debian-cve-convert, combine-to-osv, cve5-to-osv, indexer, debian-convert): legacy docker build --cache-from cannot cache intermediate GO_BUILD stages from the final runtime image. Kept pull-worker-base and --cache-from gcr.io/oss-vdb/worker-base:latest, which is single-stage and achieves full cache hits.
  • Bootstrap buildx early: Changed setup-buildx to waitFor: ['-'] with --bootstrap so moby/buildkit initializes alongside setup and pull-worker-base.
  • Official Hugo image: Replaced gcr.io/oss-vdb/ci with ghcr.io/gohugoio/hugo:v0.170.1@sha256:4cc9f383086c0fba180efcf7132cf2f025969c462fedf5fcd2bf921915416797. Because the official image runs as non-root USER hugo:hugo, added a lightweight prep-hugo step (reusing already-cached gcr.io/cloud-builders/git) to create a writable website/dist/static/blog directory, and invoked entrypoint: 'hugo' with --noBuildLock.

Testing

  • Ran ghcr.io/gohugoio/hugo:v0.170.1@sha256:4cc9f383086c0fba180efcf7132cf2f025969c462fedf5fcd2bf921915416797 locally with --entrypoint hugo --noBuildLock --buildFuture against website/blog into a chmod 777 output directory (matching Cloud Build's root-owned /workspace setup): built 28 pages + 26 non-page files in 39 ms and verified generated index.html, posts/, and external link rendering.
  • Validated YAML syntax and full step dependency DAG with a Python check (all 78 steps valid, all 32 pushed images intact and gated on cloud-build-queue).
  • Verified clean zero-conflict merge alongside the sibling .dockerignore, GOMAXPROCS, and scheduling PRs.
  • Ran poetry run tools/lint_and_format.sh: passed.

agy

@another-rex another-rex added the wheel-of-fortune Intentionally blank label Nov 20, 2026
@another-rex
another-rex marked this pull request as ready for review November 20, 2026 04:32
@another-rex
another-rex merged commit 19ef18d into google:master Nov 20, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wheel-of-fortune Intentionally blank

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants