[chasetmartin]

Adds a pygeoapi_rebuild Dagster sensor. Once a product job (cohort or standalone) has succeeded and none are queued or running, it starts the die-pygeoapi-rebuild Cloud Build trigger, so the pygeoapi image bakes the new data. Several runs finishing on the same day give one build.

• The sensor calls the Cloud Build API directly with GCP_SERVICE_ACCOUNT_KEY and launches no Dagster run, so it uses no Dagster+ credits.
• It is stopped by default, so local and branch deployments never rebuild prod.
• Docs updated: SPEC §6.6, T5 in docs/pygeoapi-performance-plan.md, and orchestration/AGENTS.md.

Before turning it on:

• Create a manual-invocation Cloud Build trigger die-pygeoapi-rebuild in us-central1. It builds main with orchestration/pygeoapi/cloudbuild.yaml.
• Give the Dagster service account roles/cloudbuild.builds.editor, plus roles/iam.serviceAccountUser on the trigger's service account.
• After merging, turn the sensor on in the prod deployment.

🤖 Generated with Claude Code

[github-actions]

Your pull request is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
die-orchestration		View in Cloud	Sep 25, 2026 at 10:19 PM (UTC)

[jirhiker]

Is baking the data into the pygeoapi image a long or short term solution?

[chasetmartin]

@jirhiker If the entire package of data in parquet files keeps to a size similar to now (~80MB), I think it's a long term solution. Definitely open to discussion and continuing testing. Why I think this: 80MB is quite a minimal increase in image size. If the DIE pipeline runs weekly, the rebuild to update this data only needs to happen weekly. Saving the parquet into the storage on the image removes any network calls/latency when pygeoapi needs to get the data (compared to going to GCS or a GC Postgres DB).

If we want further performance gains for the new version of Weaver, we could build the list of most common requests from the client (like for loading the map), and save the gzipped responses into the image at build time so that those requests are just a static file read. I will explore this more because it's very relevant for the initial map loads.

If traffic grows the the point where CDN caching would also improve performance, we could turn on Google's Cloud CDN to cache and deliver repeat requests.