Skip to content

feat(functions-compiler): every build records a returned 5xx as one _b44_error line - #669

Open
egidijusj wants to merge 7 commits into
base44:mainfrom
egidijusj:feat/functions-compiler-error-records
Open

egidijusj wants to merge 7 commits into
base44:mainfrom
egidijusj:feat/functions-compiler-error-records

Conversation

@egidijusj

@egidijusj egidijusj commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Every build of a backend function now carries a small recorder in its generated Worker entry. When a function catches its own error and returns a 5xx, the recorder logs one _b44_error line saying what went wrong, so the Production Agent's Tail Worker has a reason to file, not just a status.

There is no build option. The off switch is the request: the recorder does nothing unless the request carries x-b44-feed-context. The backend sends that header only for production calls of apps that opted in, while its tail switch is on. Turning that switch off silences the record in every bundle, whatever built it.

Stacked on base44/cli#668 (source maps). Only the commits after #668's belong to this PR. #668 lives on a fork, so this PR can't use its branch as a base.

Why it's in every build

  • An app is ready the moment it opts in, with no rebuild. Behind a build flag, only builds that set the flag would carry the record.
  • The public CLI's own builds get it too. The CLI compiles shards locally with its own COMPILE_MODES (packages/cli/src/core/version/functions.ts), which would never have set a flag.
  • It costs little. About +3.0 KB raw / +1.2 KB gzipped per module (single-function: 41,400 → 44,422 bytes; per-app: 43,159 → 46,191). Without the header, it costs one headers.has per request and one context lookup per console.* call. Source maps (feat(functions-compiler): opt-in external source maps with app paths #668) keep their flag, because they cost compile time and memory and are useless unless the caller stores them.

Request and response changes

None. An earlier revision of this PR added an errorRecords option; it's gone. Nothing shipped with it, so no caller sends it and no compatibility shim is needed. The build banner is unchanged from main: the errorRecords field is gone, and the banner's compiler version tells a consumer whether a build carries the record.

The line, logged through the console patch (so per-app scripts stamp _b44_function):

console.error({ _b44_error: { v: 1, kind: "response", status, name, message, stack } }) // name/stack omitted when absent

Behaviour (src/error-records.ts), unchanged by this revision

  • Inert unless the request carries x-b44-feed-context. Presence is enough; the value isn't checked.

  • Handler Response with status ≥ 500: the same object is returned at once, never rebuilt, delayed or given a header. The recorder clones it and does the rest inside ctx.waitUntil. It sees the handler's own response, before telemetry or invocation logs re-wrap it.

  • Message, in order:

    1. an Error logged (any console.*) during this request whose message the JSON body repeats anywhere. The recorder keeps the last 8 Errors logged in the request and uses the newest one that matches, so a later unrelated warning doesn't hide it;
    2. otherwise the body: JSON only, from the clone, at most 2,048 bytes within 50 ms, then the reader is cancelled (over 2,048 bytes yields nothing). The first non-empty string among error, message, detail, details, error.message; error is skipped when details is present and error is exactly "Internal server error";
    3. otherwise the newest Error logged during this request.

    No message means no record.

  • Stack only from the logged Error whose message was chosen. It's minified: it's that Error's raw stack, read inside the Worker, and Cloudflare remaps only the exceptions it captures. The Production Agent remaps it with the same version's source_map from feat(functions-compiler): opt-in external source maps with app paths #668.

  • Scrub before cutting. Every string of 8+ characters from the request's env bindings and runtime secrets, longest first, becomes [redacted] in the name, message and stack. This includes strings nested in JSON-valued bindings, also when they start with whitespace (such as the private-data-sources manifest, which carries DB passwords). Binding objects (KV, services, Hyperdrive) are skipped.

  • Bounds: name ≤ 64 characters; message ≤ 500 UTF-8 bytes after toWellFormed(), cut on a code point; stack ≤ 4 KB, cut at a line end.

  • Never throws into the request. Any failure writes nothing.

  • Crash path untouched. The per-app entry's catch still does console.error(e) then throw e. A test pins that block's exact text.

This changes every build's output once

Snapshot change, in one line: each of the four pinned worker-entry snapshots (single-function and per-app, telemetry on and off) gains the recorder prelude. In each, exactly two existing lines change: the request store gains ..._b44ErrorStore(request), and the return becomes _b44RecordError(await handler(...)).

So every app's next build produces different bytes. Where a version's identity is the hash of the compiled output, as in the CLI's versions path, that mints one new version per app on its next publish. A package-version bump does the same anyway.

Byte comparison of the single-function lane

The README said bundle() stays bannerless "which keeps that lane byte-comparable with the engine apper still runs". I found nothing that still compares it.

  • apper's bundler service (infra/base44-userapp-bundler) pins and runs this package (0.1.7), so there's no second engine to compare against.
  • apper's own "byte-identical" notes (function_bundle.py) are about the bundle input, the flat main.ts submission, not the compiled output.
  • I found no hashing of the compiled module in apper's deploy path that decides redeploys.
  • bundle-banner.e2e.test.ts only asserts that bundle() has no banner, which still holds.

I kept bundle() bannerless, since it compiles one function and has nothing to name, and reworded the README. The caveat: this is from a local apper checkout I didn't update, so a newer apper commit could differ. No flag was kept for it.

Open question: client-sent x-b44-feed-context

The recorder treats the header's presence as the switch. Now that every app carries it, it matters whether the dispatcher strips a client-supplied x-b44-feed-context before calling a function. I couldn't find the header anywhere in the local apper checkout, so I can't verify this. The impact of a spoofed header is small: it only records a 5xx the function really returned, with secrets scrubbed, into that app's own logs. Please confirm the dispatcher drops it from client requests.

Tests

test/error-records.e2e.test.ts runs every case in workerd (Miniflare) and reads records through a real Tail Worker, the way production reads them. With no build option, the header is the only switch: every positive case also asserts there's no record without the header, and the cases that compared an option-off build now compare header-absent vs header-present on the same build.

  • 5xx JSON in each field shape: error, message, detail, details, error.message, first non-empty wins. Both SSO shapes: {error:"Internal server error", details} gives details, and {error:"Internal server error"} gives the placeholder. A near-miss isn't skipped. A 503 reports 503. The per-app entry works too and stamps _b44_function;
  • no record for: a 5xx text body, a 4xx, a 2xx, a 5xx with no message field, a non-Response return, a body over 2 KB, and a streamed body that misses the 50 ms deadline. The client still gets the whole body in the last two;
  • a recorder failure (a Response subclass whose clone() throws): the response stands and no record is written;
  • logged Errors: a matching one gives name, message and stack; it still wins when an unrelated warning is logged after it; the 8-Error bound is pinned (7 later Errors keep the match, 8 push it out); an unrelated one is ignored; the newest is the fallback for a text body; an Error from an earlier request doesn't leak;
  • non-ASCII cut to ≤ 500 bytes on a code point; a lone surrogate becomes U+FFFD; env secrets scrubbed in message and stack, longest first, with short values kept; JSON bindings scrubbed, including whitespace-led ones; stack ≤ 4 KB on a line end; name ≤ 64;
  • the response is identical (status, headers, body) with and without the header, and its headers arrive while the handler's stream is still open;
  • an uncaught throw, in both entries: the same status, body and Tail exceptions with and without the header, and no record.

test/worker-entry.test.ts:

  • the recorder is in both entries under every telemetry × runtimeSecrets combination;
  • it sits inside telemetry and invocation logs;
  • the crash block is unchanged;
  • the banner keys are exactly functions, telemetry, runtimeSecrets, compiler;
  • the four pinned snapshots are regenerated.

Results:

  • full compiler suite: 488/488;
  • typecheck across all packages: clean;
  • lint and knip: clean;
  • the CLI's tests/core/version-functions.spec.ts, its function-compiling publish path: 4/4.

Notes for review

  • Version and changelog. No version bump or changelog entry in this PR, per this package's convention: the Manual Functions Compiler Publish workflow bumps the version in its own release commit, and the root CHANGELOG.md is the CLI's. The release notes should say: every build now carries the _b44_error record, inert until the backend sends x-b44-feed-context. The README says the same.
  • What gets no record:
    • responses the wrapper builds rather than the handler: the per-app "failed to initialize" 500 and the 503/404 routing answers;
    • the per-app crash response that X-B44-Capture-Logs turns an uncaught throw into, which is the crash path;
    • actor entries.
  • Overlap with base44/cli#662 (edge errors on outbound fetches): it also edits worker-entry.ts and its snapshots, so whichever lands second needs a rebase and a snapshot regeneration.
  • "Not delayed" in Miniflare. Miniflare's dispatchFetch hands a streamed body to the caller only once it closes, so the test times the headers.

🤖 Generated with Claude Code

`sourceMap: true` on `bundle`/`bundleApp` returns `map` beside `module`:
an esbuild map of the final compile (not the per-function probes) with no
sourcesContent and no sourceRoot. The module keeps the trailing
`//# sourceMappingURL=_bundled.mjs.map` comment Cloudflare reads to remap
stack traces, and otherwise matches the option-off build byte for byte.

The map's sources name the request's files: per-app `fn_<index>/` is
stripped and checked against that function's files, and a function's
entry becomes its optional `entryPath` (its real stored path). Every
other source stays in place, so the mappings' indices hold, behind a
`base44-internal:` prefix.

Off by default; the public CLI never sets it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@egidijusj
egidijusj force-pushed the feat/functions-compiler-error-records branch from f570f53 to 9d49b63 Compare October 20, 2026 06:42
egidijusj and others added 3 commits October 20, 2026 10:22
…e_map

Cloudflare remaps only the stacks of uncaught exceptions it captures;
a stack the Worker logs itself (the per-app entry's console.error before
rethrowing, or user logs) stays minified. Run a throwing function in
workerd with a Tail Worker, for both entries, and map the exception's
frames back to the app's files by hand — local workerd does no remapping
— and pin that the logged per-app crash line stays minified. Say in the
README what is remapped automatically and what a consumer must remap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`errorRecords: true` on `bundle`/`bundleApp` bakes a recorder into both
generated entries. It is inert unless the request carries
`x-b44-feed-context`. For a handler `Response` with status >= 500 it
returns that same response at once and, on a clone inside
ctx.waitUntil, logs one
`{ _b44_error: { v: 1, kind: "response", status, name?, message, stack? } }`
through the console patch.

The message comes from an Error logged during the request whose message
the JSON body repeats, else the body's first non-empty error / message /
detail / details / error.message (2 KB, 50 ms; the SSO
"Internal server error" placeholder skipped when details is present),
else the last logged Error. The stack comes only from that Error. Env and
runtime secrets of 8+ characters are scrubbed, longest first, before
name (64 chars), message (500 bytes) and stack (4 KB) are cut. Any
failure writes nothing; the crash path is unchanged.

Off by default; the public CLI never sets it, and flag-off entries and
banners keep their bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It is the raw Error.stack read inside the Worker, and Cloudflare remaps
only the exceptions it captures, never a logged string. Say so, and that
a consumer remaps it with the same version's source_map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y recent logged Error

Two review findings on the error recorder:

- A JSON-valued binding with leading whitespace (`\n{"db":{...}}`) was
  not parsed, so the secrets nested in it were logged unredacted.
  Detect JSON after trimStart(); the original string stays a candidate.
- A single saved Error was overwritten on every log, so a later
  unrelated warning lost the matching Error's name and stack. Keep the
  last 8 Errors logged in the request, use the newest one whose message
  the body repeats, and the newest one only as the fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@egidijusj
egidijusj marked this pull request as ready for review October 20, 2026 07:45
yurynix
yurynix previously approved these changes Oct 20, 2026
The error recorder now replaces a matched secret with `[redacted]`
instead of `[secret]`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the `errorRecords` option. Both generated entries now always append
the recorder, route the handler's response through `_b44RecordError`
and seed `_b44ErrorStore(request)`, including the public CLI's own shard
builds and bundleApp's probe compiles. An app is ready for error
reporting the moment it opts in, with no rebuild.

The off switch is the request: the recorder does nothing unless it
carries `x-b44-feed-context`, which the backend sends only for
production calls of opted-in apps while its tail switch is on. Runtime
behaviour is unchanged, crash path included.

The banner loses its `errorRecords` field; `compiler` says which builds
carry the record. This changes every build's output once: about +3 KB
raw (+1.2 KB gzipped) per module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@egidijusj
egidijusj force-pushed the feat/functions-compiler-error-records branch from b2dd271 to de77dbf Compare October 20, 2026 08:38
@egidijusj egidijusj changed the title feat(functions-compiler): one _b44_error record for a returned 5xx feat(functions-compiler): every build records a returned 5xx as one _b44_error line Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants