Repository navigation
fix: staging folders beside a project path no longer fail past 260 characters on Windows - #6492
miguel-heygen wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head eebf9fe2
- blocker —
renderhits the same failure on the same project.packages/producer/src/services/render/artifactTransaction.ts:149→renderDirOwner.ts:45still doesmkdtempSyncbeside the output, i.e.<project>\renders\.<project>_<date>_<time>.hf-transaction-XXXXXX. Your 226-char repro project gives a ~300-char path there, and since the project name shows up twice, render breaks at a shallower depth than snapshot. Fix: movecreateStagingDirsomewhere producer can import it (e.g.@hyperframes/core), use it increateOwnedRenderDir, and add a test there that pins it.core/src/fonts/fontCompression.ts:74follows the same pattern but sits under the cache dir, so it's lower risk.
Root cause and scope
The root cause checks out against Node v22 src/node_file.cc: MKDir calls ToNamespacedPath (which adds the \\?\ prefix) and Mkdtemp doesn't. So every mkdtemp under a deep project path fails, not just these three.
The other mkdtemp sites in cli/producer/engine all live under tmpdir(). On win32 the render work dir also goes under tmpdir (resolveRenderWorkDirPrefix). The helper and its tests look right: 0o700, EEXIST-only retry, mkdtemp mock.
CI was still running at review time. The Windows long-path evidence comment the body promises hasn't been posted yet.
|
Windows evidence (windows-latest, long paths enabled), same project layout, snapshot --at 1 on a project folder of 226 characters (its snapshot artifacts are about 316 characters deep): on main |
| "cli/src/media-use/lib/lut-preset-provider.mjs", | ||
| "producer/src/server.ts", | ||
| "producer/src/regression-harness.ts", | ||
| "producer/src/services/render/renderDirOwner.ts", |
There was a problem hiding this comment.
blocker (still open from the last review): renderDirOwner.ts is allowlisted as "not beside a project or output path", but it is. artifactTransaction.ts:149 calls createOwnedRenderDir(join(parent, .${stem}.hf-transaction-)) next to the render output, which is <project>\renders\.<project>_<date>_<time>.hf-transaction-XXXXXX by default. On the 226-character repro project, render still fails at renderDirOwner.ts:45 with the same mkdtemp ENOENT. Fix: have createOwnedRenderDir take (parent, prefix) and call createStagingDir, then remove this allowlist entry. Its other caller, createRenderWorkDir, already splits the prefix into dirname(prefix).
There was a problem hiding this comment.
✅ Fixed in 7ef0c9e: the allowlist entry is gone and createOwnedRenderDir now calls createStagingDir. The new suffix mismatch is tracked in a separate thread.
Edit accuracy: accurate 271 (base branch 271), smooth 149 of thoseThe gate passes. Quarantined, measured but not gated (0) Unstable (1)
|
| /** A render's private temp dir, stamped with the process that owns it so a later sweep can tell it was abandoned. */ | ||
| export function createOwnedRenderDir(prefix: string): string { | ||
| const dir = mkdtempSync(prefix); | ||
| const dir = createStagingDir(dirname(prefix), basename(prefix)); |
There was a problem hiding this comment.
blocker: createStagingDir uses an 8-hex suffix, but RENDER_TEMP_DIR (core/src/projectFolders.ts:20) only matches a 6-character mkdtemp suffix (-[A-Za-z0-9]{6}$). With this change, hf-render-0a1b2c3d, work-<uuid>-0a1b2c3d and .x.hf-transaction-0a1b2c3d all stop matching (checked in node). As a result:
createRenderWorkDir's sweep andhyperframes clean(listAbandonedRenderDirs) never reclaim a killed render's work dir, so its frames leak.isProjectIgnoredDirstops hiding these dirs from the previewfileWatcherandprojectSignature.- studio-server's
safePathwalk lists them.
Fix: widen the regex suffix to {6,8} (or pass a 6-char suffix here), and add a test that RENDER_TEMP_DIR matches what createOwnedRenderDir creates.
…aracters on Windows
7ef0c9e to
cc16106
Compare
|
Windows evidence for render (windows-latest, long paths enabled): |
|
Closing: this conflicts with main, has had no update for over 5 days, and no open issue or report names who it hurts. Reopen if someone hits it; the branch is kept. |
|
Superseded by #7338: GitHub would not reopen this PR after its branch was rebuilt on current main. |
What
Every staging folder the CLI, engine, producer, core and studio-server make beside a project or output path now comes from one
createStagingDirin@hyperframes/core/atomic-file(a random 6-letter-or-digit suffix, the same shapemkdtempmade, andmkdirSync) instead ofmkdtempSync. 26 call sites move: snapshot capture files, cloud download, registry install, template pull, measured frames, audio mixing and encoding, the URL downloader, font compression, waveform cache, HDR frames, the render transaction folder, and the distributed plan, assemble and stitch steps. A test fails on any new baremkdtempthat is not under the system temp root; the few allowed exceptions are listed in it with the reason.Why
On Windows,
hyperframes snapshot --at 1failed for a project whose folder path is long, withENOENT: no such file or directory, mkdtemp '<project>\.hyperframes\artifacts\snapshots\<id>\.capture-XXXXXX'.mkdtempon Windows does not get the long-path form thatmkdirgets, so it fails once its full path passes 260 characters.renderhits the same failure at the render transaction folder. Every site that stages next to a target in a project has the same failure on a deep project, so they share one helper.Supersedes the earlier version of this change, rebuilt on current main so it also covers call sites added since. The helper keeps
mkdtemp's name shape, so the patterns that recognise leftover render folders and template pull folders still match them.Tests
noBareMkdtemp.test.tson main's sources lists 26 offenders (cli/src/capture/captureFile.ts:21and 25 more) and fails; with the change it passes.atomicFile.test.tspins the helper: private folder, 6-character suffix, a taken name is retried, five collisions give up, any other error is not retried (it fails when the suffix length is changed).snapshot --at 1failed on main and exited 0 with the change;renderexited 0) is in the comments; Windows CI runs on this head.