[miguel-heygen]

What changed

Opening a project for the first time starts its history by copying every file into the history store. That copy now takes a fraction of the time on a project with thousands of files, and the history it writes is the same.

Design: none (no UI change)

Why

A project's first open waits for this copy. It took one file at a time, with about seven file operations per file (a stat, two folder creations, a clone, a read to hash the clone, another stat, a rename) and a folder check on the main thread after each.

How

• Files up to 64 KB are read once, hashed, written to a temp file and renamed into place. Larger files keep the clone they had; a clone saves real disk there and almost none on a small file.
• Folders are made when a write finds them missing, not before every write.
• Up to 4 files copy at once, shared across every project opening in the process, so several first opens together still leave room for other file work. 16 at once was no faster and made other reads wait longer (numbers below).
• The folder check runs once when the copy ends. Nothing names the copies until the log is written after it.

Unchanged: the on-disk format, what the store holds, and the order of the baseline. Small files get their exact mode set after the write (one more call per file), since a write applies the umask and the clone it replaces did not.

What I measured

One rig, before (main) and after (this branch), on GitHub's macos-latest and windows-latest runners, 3 rounds each, alternating. Each project has 4,005 files: 4,000 small scripts, an HTML file, two files with the same bytes, one 200 KB file and one executable script. The rig opens one project's history for the first time, then 11 more at once in one process, while timing a small stat on the side.

	macOS before	macOS after	Windows before	Windows after
One project's first open	9.2, 9.2, 10.7 s	2.6, 2.0, 2.4 s	20.3, 19.5, 19.1 s	8.2, 8.2, 8.0 s
11 at once, median done	30.0, 28.8, 28.1 s	15.1, 13.6, 20.2 s	189.8, 129.7, 131.9 s	56.2, 48.8, 48.3 s
11 at once, all done	30.4, 29.1, 28.3 s	25.7, 23.0, 31.8 s	190.2, 130.1, 132.5 s	102.5, 87.4, 88.3 s
Longest side stat, one open	89, 93, 99 ms	104, 91, 121 ms	190, 201, 242 ms	424, 418, 166 ms
Longest side stat, 11 at once	743, 746, 49 ms	1004, 256, 876 ms	1447, 1223, 971 ms	397, 353, 363 ms

History contents are identical before and after on each OS: the baseline of every file and every blob's bytes and permissions hash to the same fingerprint (478a308d… on macOS, 98cba7a6… on Windows), 4,005 files each.

Floor and ceiling: a single first open is 2.7 to 5.2 times faster. With 11 at once, half the projects are ready in about half the time, but on macOS the last one finishes in about the same time (one round slower), because the 4 shared slots keep the copies from running wider. The side stat waited longer on a single Windows open (up to 424 ms against 242 ms) and on a macOS batch, and shorter on a Windows batch.

fails without the fix: firstCopy.test.ts, "copies a few files at a time across every project opening together" fails when the limit is lifted; "keeps the same contents and permissions, small files and large" fails when permissions are dropped; blobStore.test.ts, "counts a file once when two copies of it are filed together" fails with an unguarded count. Each break fails exactly one test.

"loses nothing when the process dies midway: the next open finishes it" kills a real process with SIGKILL during the copy, with a half-written temp file on disk, then opens again and checks every file is in the baseline with its exact bytes, every blob matches its hash, and no temp file is left. It fails if the log is written before the copy ends, or if half-written temp files aren't swept.

"keeps the same contents and permissions, small files and large, under a strict umask" runs under umask 077 and fails without the explicit mode (expected 64 to be 73); every history test also passes run under umask 077.

blobStore.test.ts changes one line of an existing test: it counts refusals, not rename attempts, because a first rename can now find the folder missing and try once more.

What I did NOT exercise

• Linux in the rig; the unit tests run there.
• A file system where clones are real savings for small files (APFS clones): small files are now copied, so a project of many small files uses a little more disk in history than before.
• Files changing during the first copy, beyond what the existing tests cover. The bytes hashed are the bytes filed, as before.
• The 4-slot limit at other values; 16 was no faster in a separate bench and made other reads wait longer.

[github-actions]

Edit accuracy: accurate 267 (base branch 267), smooth 100 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (8)

• seqplainkeys-none-center-r0-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-center-r0-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): fail / pass / pass, passes
• seqplainkeys-none-center-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-pct-r0-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-pct-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-px-r0-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): pass, passes
• seqplainkeys-none-px-r30-nested-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run
• seqplainkeys-none-px-r30-root-z100 (fixed by studio: undo/redo on a keyframed plain element sometimes leaves the box 230-400px off (edit-accuracy seqplainkeys-none-*) #6989): not run

[heygengenesis]

Requesting changes at 09236877 for one blocker. The rest looks right.

Blocker: small-file blobs lose permission bits under a restrictive umask. writeFile(temp, bytes, { mode }) applies the process umask to mode. On main, put clones (COPYFILE_FICLONE_FORCE), which keeps the source mode exactly. Only the rare stream fallback was masked. Now every file of 64 KB or less goes through the masked path, and writeTo clones the blob back on restore. So with umask 077 or 027 (common on hardened Linux and shared hosts), a 0755 script is stored as 0700 or 0750, and restoring it changes the file's mode.

Reproduced in a sandbox with umask 0077 on a reflink-capable file system:

• main: projectHistory.test.ts › "keeps the file's mode" passes.
• this head: that existing test fails (expected 448 to be 493), and so does the new firstCopy.test.ts › "keeps the same contents and permissions, small files and large" (expected 64 to be 73).
• this head with umask 022: all 259 history tests pass.

Suggested fix: after the write, set the mode explicitly, e.g. chmod(temp, mode & 0o7777), or write through a handle and handle.chmod. That costs one more call per file. You could skip it when (mode & ~process.umask()) === (mode & 0o7777). Please also make one test pin the umask, for example by running it under process.umask(0o077) with a restore in finally, so CI catches this. CI's default 022 hides it today.

Checked and OK

• The 'fails without the fix:' line names real tests. I applied each break (slot limit raised to 1000, unguarded total += size, mode dropped from writeFile), and each fails the test it names.
• inFirstCopySlot hands a slot directly to the next waiter and only decrements when none is waiting, so it never goes above 4 or leaks a slot.
• The SIGKILL test is sound: open sweeps incoming-* leftovers, and the log is written only after allSettled. Moving the whereFolder() check to the end is safe because nothing names the blobs until persistLog.
• putBytes hashes the bytes it writes, so a write racing the read can't file bytes under the wrong hash. The file() guard fixes the double count when two puts of the same bytes run at once.
• __fixtures__/dieMidFirstCopy.ts is not a tsup entry, so it isn't bundled.

Non-blocking

• allSettled now waits for every copy before reporting the first failure. On main the copy stopped at the first one. That's fine, just worth knowing for a project that is moved away mid-copy: the remaining copies run and mostly find no file.

[heygengenesis]

mode here is masked by the process umask, but the clone this replaces kept the source mode exactly. With umask 077, a 0755 script is stored as 0700, and writeTo clones it back on restore. Set the mode explicitly after the write (chmod(temp, mode & 0o7777) or handle.chmod).

[heygengenesis]

Approving at d5a1f47d. This resolves the blocker from my last review.

• After the write, chmod(temp, mode & 0o7777) sets the stored copy's mode exactly, as the clone path does.
• The renamed test runs under process.umask(0o077) and restores the umask in onTestFinished. I deleted the chmod line and the test failed with expected 64 to be 73; with the line restored it passes. That makes it a valid 'fails without the fix:' test.
• I ran all history tests under umask 077 and under 022, and both runs passed (259 passed, 2 skipped).
• Nothing else changed since 09236877, so my earlier checks still apply.

CI on this head was still running when I posted this.