bug: squad externalize corrupts binary files under .squad/ and deletes the originals #1489

Summary

squad externalize moves every file under .squad/ by reading it as a utf-8 string and writing the string back out. Any file that isn't valid UTF-8 text gets its bytes ≥ 0x80 replaced with U+FFFD during the read — and because externalize deletes the source after the copy, the corruption is silent and unrecoverable. squad internalize copies back through the same lossy path, so the round trip never restores the original.

Steps to reproduce

Verified on dev @ e57b904 (built dist, squad-cli 0.11.0), Windows 11, Node v22.21.0. APPDATA was overridden so the external dir lands in a sandbox.

mkdir -p myproj/.squad/assets && cd myproj
echo "# Team" > .squad/team.md
# 1 KB binary file: byte values 0x00-0xFF repeated
node -e "const b=Buffer.alloc(1024); for(let i=0;i<1024;i++) b[i]=i%256; require('fs').writeFileSync('.squad/assets/diagram.png', b)"
sha256sum .squad/assets/diagram.png
# 785b0751fc2c53dc14a4ce3d800e69ef9ce1009eb327ccf458afe09c242c26c9  (1024 bytes)

squad externalize
# ✅ Externalized 2 items. State now at: .../squad/projects/myproj

After externalize:

ls .squad/assets/
# ls: cannot access '.squad/assets/': No such file or directory   ← original gone

sha256sum <external>/projects/myproj/assets/diagram.png
# 36c2611dbed66212ce63c6bb7957c2199c66386fee7f6fafb2f7a3b151bb7a83  (2048 bytes)

dd if=<external>/.../assets/diagram.png bs=1 skip=128 count=12 | od -An -tx1
#  ef bf bd ef bf bd ef bf bd ef bf bd                              ← U+FFFD from byte 0x80 on

Every byte ≥ 0x80 became the 3-byte UTF-8 replacement character (1024 → 2048 bytes: 512 low bytes preserved + 512 high bytes × 3). The first 128 bytes survive, which makes the corruption easy to miss at a glance.

Root cause

Verified on dev @ e57b904:

• packages/squad-cli/src/cli/commands/externalize.ts:73-76 (file branch) and :200-215 (copyDirRecursive) — both copy via storage.readSync(src) → storage.writeSync(dest, content).
• packages/squad-sdk/src/storage/fs-storage-provider.ts:193 — readSync is readFileSync(path, 'utf-8'), which decodes invalid sequences to U+FFFD. The string round-trip is lossy for any non-UTF-8 content.
• externalize.ts:71 / :78 — the source is deleted (deleteDirSync / deleteSync) right after the lossy copy, so the only intact copy is destroyed.
• runInternalize (externalize.ts:158-165) uses the same copyDirRecursive, so internalize corrupts in the same way.

Impact

Core squad state is markdown/JSON today, so text-only projects don't notice. It bites:

• any binary a user or agent keeps under .squad/ — diagrams or screenshots referenced from decisions.md, PDFs, archives;
• any future binary state: SqliteStorageProvider (in-tree, currently unwired) defaults its database to .squad/squad.db — the moment that backend becomes reachable, externalize destroys the whole team state db in one command.

Likely also strips POSIX exec bits on scripts (writeSync creates files with default mode) — not verified here, Windows box.

Suggested fix direction

Copy bytes, not strings: copyDirRecursive and the file branch can use the provider's copy/copySync (copyFileSync underneath, already on FSStorageProvider) instead of read/write, and only delete the source after the copy is verified. A regression test with a non-UTF-8 fixture would lock it in.

Can pick this up if useful — same area as the deleteDir/stub work in #1487.