Repository navigation
feat(cli): cloud uploads send a big file in parts, so a cut connection resends one part - #7375
Conversation
There was a problem hiding this comment.
Verdict: APPROVE · 0 blocker · 1 important · 1 nit · head 9af5d20b
- important
uploadParts.ts:52-68+upload.ts:385: the saved plan is removed only aftercompletesucceeds. Ifcompleterefuses for good (for example the content-type refusal thatnameRefusedFilehandles), every later run of the same bytes resumes the same asset, with its original filename and content type, and gets the same refusal until the multipart upload expires on the server. Renaming the file doesn't help either, because the key is checksum+size. Fix: forget the saved plan whencompletereturns a non-409 4xx. - nit
uploadParts.ts:160: a mint failure (HyperframesApiError401/404) inside theputWithRetryclosure isn'tisFinal, so it is retried 4 times with backoff. On a part, therepeatedCutmessage still says "Every try restarts the file".
Scope
Read upload.ts, uploadParts.ts, uploadPut.ts and the test end to end; skimmed _gen. Ran uploadParts.test.ts locally: 8/8 pass. I checked the "fails without the fix" claim by disabling if (plan) return sendInParts: the 1.2 s cut test fails with "cut at 1 s on two tries in a row", and the cut-part and resume tests fail too. mergeStateStatus CLEAN. No prior reviews.
The base branch was changed.
# Conflicts: # packages/cli/src/cloud/upload.ts # packages/cli/src/cloud/uploadPut.ts
There was a problem hiding this comment.
Verdict: APPROVE — 1 resolved, 0 open, 0 new · 9af5d20b → b554a017. A 4xx refusal from complete now forgets the saved plan; 401/403/408/409/429, 5xx and cancels keep it so the upload can resume. The new test covers both sides. Since my last review the merge from main changed no files under src/cloud, and CI is green.
…n never feeds the single PUT
There was a problem hiding this comment.
Verdict: APPROVE — 0 resolved, 0 open, 0 new · b554a017 → 8249651a. A resumed upload now goes straight to sendInParts, so a plan read from disk can no longer reach putBytesToPresignedUrl. Behavior is otherwise the same: the guard dropped with "upload_url" in created was only there for the old union type, and CreateAssetUploadResponse.upload_url is required. CI and CodeQL are green.
Port the adopted upstream fixes from f5bde95 to the current upstream tip: heygen-com#7375 multipart cloud upload, heygen-com#7382 minParallelChunks, heygen-com#7349 HLG header SEI, heygen-com#7369 skip unmounted compositions in lint/check, heygen-com#7381 filesystem-path media in the cloud archive, heygen-com#7383 Safari cut-point pre-roll revert, heygen-com#7373 clock steering between audio time updates, heygen-com#7384 injector test isolation, heygen-com#7378 preview seek retention, heygen-com#7390 virtual timer schedule per worker, heygen-com#7394 CDN script failure naming, heygen-com#7395 menu row radius, heygen-com#7119 decode timeout caller test, heygen-com#7397 authored tween order, heygen-com#7402 capture page clip loading, heygen-com#7412 muted video audio track at cuts, heygen-com#6150 template root variable test, heygen-com#7442 held tail seek once. Deliberate local differences: heygen-com#7067's audio-start hold is not carried into clock.ts, mountedSources comes from heygen-com#6827 as a helper only, cloud/_gen/types.ts omits the image-request types the fork does not generate, and the lint symlink test skips through symlinkOrSkip where the account cannot create links. Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
What
hyperframes clouduploads a file over 16 MiB in parts when the cloud offers them, so a cut connection costs one part instead of the whole file.multipart: trueon create). With a plan it sends each part to its own presigned URL through the same PUT and retry as today, keeps each part's ETag, and tellscompletewhich parts make the file.completesucceeds, and also whencompleterefuses the file itself (a 4xx other than 401, 403, 408, 409 or 429), so a refused file is not resumed into the same refusal. A cut, a 5xx or an abort keeps it for the next try._gen/client.tsand_gen/types.tsgain the part routes and fields the server adds.Why
Users on links that cut long connections could not finish a big upload: every retry restarted the file, and the CLI ended with "The connection was cut at N s on two tries in a row". Parts are short enough to finish before the cut.
Stack
On top of #7374, which moves the PUT code to
uploadPut.ts; review that one first.Test plan
uploadParts.test.ts, against a local HTTP server that cuts every connection after 1.2 s: a 17 MiB file still arrives in 5 parts of up to 4 MiB andcompletegets all 5 ETags; a cut part is sent again alone; a restart resends only the missing parts; progress counts finished and live parts. Small files, a plan-less server and a server refusing the field keep the single PUT. 186 tests insrc/cloud.complete(400 invalid_parameter) makes the next upload of the same file start fresh; a 503 oncompletemakes it resume. Without the forget on refusal, that test fails (the next upload resumes the stale plan).upload.ts, the 1.2 s cut test fails (ECONNRESET). A control case in the suite shows the same link failing a single PUT of the same file with "cut at ... on two tries in a row".src/cloudunit tests (186), 3 runs in a row;tsc --noEmitforcliexit 0.bun run format:checkexit 0;fallow auditexit 0; oxlint exit 0; comment ratchet and citations exit 0.