Repository navigation
Stop replays serving truncated answers as empty, and check the recordings - #326
Merged
Merged
Conversation
…ings A response cut off after its headers -- still streaming when a test's page closed -- is recorded as a 200 with no body. Replayed, it was a successful *empty* answer the server never gave, and the page quietly showed nothing; 24 such answers sat in 15 recordings, and the audit (#321) found a test that skipped itself for weeks on one. - The harness treats them as the cut-off requests they were and aborts them, as it already did for requests that never got a response. - Recording keeps each test's context open until its backend requests have finished (15s ceiling). The HAR is written when the context closes, which is what caught them mid-stream: truncated answers 18 -> 0 on re-recording, and requests still open at a test's end 243 -> 193. - One test left its page while it was still loading; it waits for the page first. - scripts/check-har.mjs refuses any recording with a truncated answer and no complete copy, and reports the cut-off count. It runs in CI and after every recording. The rules it shares with the harness -- what the backend is, what counts as truncated -- live in e2e/support/har-entry.mjs, so the two cannot disagree. Seven spec files re-recorded. Full suite 269 passed, none failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A response cut off mid-stream now counts as truncated when it had no length to declare, not only when its Content-Length promised a body: with no length, a chunked or JSON response that kept no body was cut off. Eight such answers, in six recordings the re-recording had not touched, were still replayed as successful empty ones; five of the six are re-recorded. - The three left are answers the app itself abandons mid-load, so a recording always catches them cut off. The check names them, each with why, rather than refusing them; anything else is a test ending too soon. - While recording, the test's time limit grows by the settle ceiling, which runs in teardown and counted against it. - The harness comment says a truncated answer falls to the pooled recordings and is aborted only if none has a complete copy; stray doc comments moved to their functions; the check's main guard survives an import with no argv. Re-recorded specs and the release suite: 62 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ned answer - A test limit of 0 means none; adding the settle ceiling turned it into 15s. - Each abandoned answer is pinned to its own recording, so a new cut-off in a neighbouring one is not let through; and the wording no longer promises an abort where another test's complete copy may answer instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second guard from #321.
The problem. A response still streaming when a test's page closed is recorded as a 200 with no body. Replayed, that was a successful empty answer the server never gave, and the page quietly showed nothing. There were 24 such answers in 15 recordings. The audit found a test that had been skipping itself for weeks because of one.
Changes
e2e/support/backend.ts) treats a truncated answer (2xx, headers promise a body, none kept) as the cut-off request it was, and aborts it, as it already did for requests that never got a response.npm run check:har(scripts/check-har.mjs) refuses any recording with a truncated answer and no complete copy, and reports the cut-off count. It runs in CI (unit job) and after everye2e:record. The rules it shares with the harness, meaning what the backend is and what counts as truncated, live ine2e/support/har-entry.mjs, so the two can't disagree. There are unit tests intools/har-check/.record-e2e.mjsstill exits with the recording's status first, and a killed recording (null status) still fails.Verification
check:harclean.🤖 Generated with Claude Code