Repository navigation
perf(d1): route request reads through D1 read replicas - #303
Merged
Merged
Conversation
Crawler bursts queue every read on the single D1 primary and fail with 'D1 DB is overloaded'. A per-request session lets safe reads start on any replica, keeps mutating requests on the primary, and hands a write's bookmark to the visitor's next request.
Collaborator
Author
|
Checked by hand on a local
Local D1 has no replicas, so this run shows that nothing regressed. It does not show reads spreading across replicas. Check that after the operator step with |
This was referenced Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
❓ Type of change
📚 Description
Crawler bursts overload the single D1 primary. On 2026-09-29 at 07:50 UTC about 520 skill page requests made about 4,900 D1 reads in two minutes. The baseline is 150 to 200 reads a minute. The burst queued on the primary and threw
D1_ERROR: D1 DB is overloaded. Requests queued for too long.about 200 times (SKILLD-2Y, 1F, 27, 2P, 1S). The same error hit on 09-22 and 09-24.Each request already opened one D1 session, pinned to
first-primary. Now the session constraint depends on the request:GET,HEAD, andOPTIONSstartfirst-unconstrained, so they can run on the nearest replica.POST,PUT,PATCH, andDELETEstartfirst-primary, so a read before a write sees every other writer.d1-bookmarkcookie for one hour. The visitor's next safe request starts from that bookmark, so they read their own write. The OAuth callback is aGETthat writes, and it is covered the same way.The cookie is set only on a response whose
Cache-Controlsaysprivateorno-store. It is never set on anything Workers Cache or the CDN can share. A cache fill that writes insidewaitUntilafter the response has left sets nothing.Cron tasks, queue consumers, and ai-ready's db0 connector use the raw
env.DBbinding, so they stay on the primary.With replication off, D1 serves all three constraints from the primary. So this can merge and deploy before the setting changes, and it behaves exactly as before.
Loose ends:
$fetchthat writes during SSR sets its cookie on the internal response, and that cookie is lost. I found no SSR path that writes and then needs to read the write in a later request.📝 Migration
Production replication is
disabledtoday (read from the API on 2026-09-29; the primary is inOC). This PR does nothing until an operator turns it on. Run this after the deploy:The dashboard path is D1 > skilld-db > Settings > Enable Read Replication. To roll back, send
"mode": "disabled". Cloudflare says a full disable can take up to 24 hours, and the code stays correct during that time. Billing does not change: it is stillrows_readandrows_written.