Skip to content

fix(functions-compiler): retry dependency downloads the Deno loader drops - #667

Open
yurynix wants to merge 1 commit into
mainfrom
fix/functions-compiler-registry-retry
Open

yurynix wants to merge 1 commit into
mainfrom
fix/functions-compiler-registry-retry

Conversation

@yurynix

@yurynix yurynix commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Note

Description

@deno/loader 0.5.0 downloads every dependency through the global fetch, never retries (http_client.rs: // todo: implement retrying), and drops the failing download's error — so a single registry blip among the ~350 requests a large tree like Stagehand needs fails the whole compile, surfacing as Could not find constraint '<root package>' in the list of packages or a bogus ERR_MODULE_NOT_FOUND. This PR wraps globalThis.fetch once per process so GETs to the dependency hosts are retried on a network error, 429 or 5xx, and logs every failed attempt so the real cause is visible. The same loader runs in base44 publish and apper's bundler service, so this fixes user deploys as well as the flaky testStagehand package-matrix row in CI (3 of 29 Functions Compiler runs).

Related Issue

None (CI flake: Functions Compiler run on main after #665)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Other (please describe):

Changes Made

  • New packages/functions-compiler/src/dependency-fetch-retry.ts — installDependencyFetchRetry() wraps globalThis.fetch once (idempotent via an installed flag); createRetryingFetch() retries twice (250 ms, then 1 s) on a network error, 429 or 5xx. A 404 is a real answer and is not retried; non-GET requests and requests to any non-dependency host (including the host's own API calls) pass through untouched. Response bodies are cancelled between attempts so a retried request does not leave its body dangling.
  • Each failed attempt logs base44.bundler.dependency_fetch_failed with the URL, attempt number and HTTP status or error — warn while retries remain, error on the last one — since the loader swallows the final failure.
  • src/esbuild/deno-resolver.ts — installs the retry wrapper in setup(), before the Deno loader is created.
  • src/fetch-guard.ts — exports isDependencyHost (renamed from isAllowedHost), requestUrl and the FetchInput type so the guard and the retry wrapper share one host allowlist.
  • packages/functions-compiler/README.md — documents the wrapper, what it retries, and the new log event.

Testing

  • I have tested these changes locally
  • I have added/updated tests as needed
  • All tests pass (npm test) — verified in CI; the e2e test needs network access and a cold DENO_DIR

New tests:

  • test/dependency-fetch-retry.e2e.test.ts — fails is-number's metadata once (503) while bundling npm:is-odd@3.0.1 on a cold cache. Without the fix the build fails with Could not find constraint 'is-odd@3.0.1'; with it the bundle succeeds after exactly one retry.
  • test/dependency-fetch-retry.test.ts — retries are bounded and return the last answer; a 404, a POST and a non-dependency host each go through exactly once.

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (if applicable)
  • My changes generate no new warnings
  • I have updated docs/ (AGENTS.md) if I made architectural changes

Additional Notes

Patching globalThis.fetch is global by nature, but the wrapper is deliberately narrow — GET + dependency-host only — so a host embedding the compiler sees no change to its own traffic, and the existing fetch guard already scoped that host list. No release bump in this PR; that follows separately, as usual.


🤖 Generated by Claude | 2026-10-19 09:13 UTC | b65bd0b

…rops

@deno/loader fetches every npm/jsr dependency through the global fetch,
never retries, and swallows the failed download's error. One registry
blip deep in a large tree then fails the build as "Could not find
constraint '<root package>' in the list of packages" (or a missing
module for a lost tarball) — the Stagehand package-matrix row flaked on
exactly that.

Wrap globalThis.fetch once on the first compile: GETs to the dependency
hosts are retried on a network error, 429 or 5xx, and each failure is
logged as base44.bundler.dependency_fetch_failed. Other requests pass
through.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@yurynix

yurynix commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Verification

Reproduction against @deno/loader 0.5.0 (a standalone script that fails one request through globalThis.fetch, cold DENO_DIR, resolving npm:@browserbasehq/stagehand@3.7.3):

Injected failure Loader result
none resolves (~35 s)
503 on the gcp-metadata packument Could not find constraint '@browserbasehq/stagehand@3.7.3' in the list of packages. (the CI error)
TypeError: fetch failed on the same packument the same message
503 on the gaxios-7.3.1.tgz tarball ERR_MODULE_NOT_FOUND … stagehand/3.7.3/index.js (the #663 variant)
rerun on either failed cache, no injection resolves, so the cache isn't left broken

Regression test: dependency-fetch-retry.e2e.test.ts fails on main with Could not find constraint 'is-odd@3.0.1' in the list of packages. and passes on this branch.

Full functions-compiler suite on this branch, 8 local runs (4 on the shared Deno cache, 3 on a fresh DENO_DIR each, plus the first run below):

  • Stagehand passed in 7 of the 8.
  • In the first run it failed. I didn't capture that run's error message, so I can't say which failure mode it was.
  • One run timed out in workerd-runtime.e2e.test.ts > keeps duplicate in-flight targets until the last sibling settles. That test sends Miniflare dispatchFetch to localhost with no npm imports, which the wrapper passes straight through.
  • No real registry failure happened in the logged runs, so the retry itself was exercised only by injected failures.

Typecheck, Biome and knip are clean.

Flake rate on main before this change: 3 of 29 Functions Compiler runs failed on the Stagehand row since it landed on 2026-10-15 (39223401078, 38738000589, 38732605699).

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Package Preview Available!


Install this PR's preview build with npm:

npm i @base44-preview/cli@0.1.33-pr.667.b65bd0b

Prefer not to change any import paths? Install using npm alias so your code still imports base44:

npm i "base44@npm:@base44-preview/cli@0.1.33-pr.667.b65bd0b"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "base44": "npm:@base44-preview/cli@0.1.33-pr.667.b65bd0b"
  }
}

Preview published to npm registry — try new features instantly!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant