macOS guests 6/6: experimental machine-image builds

[chruffins]

Summary

PR 6 of 6. Experimental machine-image builds for macOS. Base: #514. Not needed for v0; this can land later.

• Opt-in: macos_builds: true (requires macos_only) lets POST /builds accept machine_base_image (a SHA256-pinned installed base) and a versioned recipe (hypeman-macos-build.json, version 1).
• Job: the recipe runs as root in an isolated builder VM made from the base. The builder is stopped strictly (acknowledged shutdown and confirmed VMM exit, no forced fallback), then exported and destroyed. The result is streamed to hypeman's own registry under builds/{id}, and the build waits for the existing image-readiness gate.
• Disk use: the layer is generated twice from the export (once to hash, once to stream) instead of being spooled to disk. With macOS guests 2/6: OCI machine-image integration #499's sparse unpack, a build writes the compressed layer and the disk's real data once each.
• Recovery: after a restart, interrupted machine jobs are rebuilt from their staged source. The source is kept until the image is ready, and each attempt reaps the previous attempt's builder and workspaces. Linux builder services aren't started in macos_only mode, and Linux jobs stay untouched on disk.
• Errors report only the failed step and its exit code, or host admission errors such as the base's identity being in use; guest output stays masked.
• Uploads are authenticated before taking one of 4 upload slots, and capped at 65 MiB.

Linux code touched

• lib/builds/manager.go: machine jobs dispatch to macos_build_manager.go; the staged machine source is removed once the image is ready.
• lib/instances/stop.go: a strict stop variant used only for export; a normal stop takes the same path as before.
• cmd/api: macos_builds config, the upload limiter, and multipart parsing for machine builds. Linux build requests are parsed as before.

Limitations

The recipe format is internal (version 1). Recipe output isn't streamed. No clean-template sanitation policy, toolchain provenance or storage quota yet. registry.url must be hypeman's own registry.

Testing

• Unit tests cover admission, recipe extraction limits, strict stop and quarantine, malformed exports, masking, publication ordering and verification, streaming without a spool, and recovery of queued, building and pushing jobs.
• Native QA (Apple silicon, macOS 27.0.1): end-to-end builds through the HTTP API, recipe-step failure, cancellation, API crash during provisioning, publication and conversion (each rebuilt), and host admission errors.

Stack

Each PR is one commit on the one below it. macOS logic lives in its own files (macos*.go, *_darwin.go, exec_lifecycle.go, port_forward.go); shared Linux files get small hooks such as if stored.MacOS != nil.

1. macOS guests 1/6: VZ runtime, machine images and instance lifecycle #498 VZ runtime, machine images and instance lifecycle
2. macOS guests 2/6: OCI machine-image integration #499 OCI machine images
3. macOS guests 3/6: Darwin guest service, readiness and exec #500 Darwin guest agent, readiness and exec
4. macOS guests 4/6: snapshots, schedules and stopped forks #506 snapshots, standby and stopped clones
5. macOS guests 5/6: guest-port forwarding #514 guest-port forwarding
6. macOS guests 6/6: experimental machine-image builds #503 machine-image builds (not needed for v0; can land later)

[chruffins]

Actual full-size transport QA passed against lib/registry.New(...).Handler() with an empty filesystem-backed destination: streamed the real 28,150,700,918-byte (26.22GiB) compressed installed macOS machine layer plus config over loopback HTTP using monolithic POST /blobs/uploads/?digest=sha256:.... No disk seeding at destination. Server verified streaming digests, then the test independently rehashed persisted blobs, published the original manifest last and fetched it to verify the original manifest SHA256; tag was404 before publication.

Runner/supervisor exit0,47.96s total, sampled peak process-group RSS17.98MiB, minimum free disk491.61GiB. Bounded by1.5GiB RSS/60GiB free disk/20min deadline, nice10/GOMAXPROCS1/GOMEMLIMIT256MiB; no VM ran during upload. Sampled RSS does not include OS cache or account for physical APFS allocation.

This demonstrates a viable streamed transport for the concrete publisher, not a completed PR5 publisher/build API. Cache namespace avoided image conversion; registry auth middleware, remote credentials/redirect/retry/resume and ambiguous commit reconciliation were not exercised. Default go-containerregistry remote.Write PATCH still buffers upload data in0.20.6 even with a disk BlobPutHandler; it was not fixed or passed here. Separately, the earlier pulled artifact passed real normal-API cold boot/lifecycle/recovery; it was not freshly pulled from this upload destination, so these are not an end-to-end build→upload→pull→boot result.

[chruffins]

Simplification update at ed21906: standalone production MachineBuildRunner replaced by internal MachineBuildBackend executing machine-specific phases through ordinary CreateBuild/runBuild. Normal manager owns queue, persisted request, deadline, private source staging/hash verification, status/log completion, provenance and image-readiness gate. Machine resources inherit base; unsupported Linux builder/cache/secret options fail admission; recovered source is verified before start. Source helper is shared without adding a Linux source-size cap. Normal-manager synthetic tests prove ready/failed status, source consumption and no publication after provision failure. Build race suite passed 3 runs excluding exactly five existing Linux config-volume tests requiring unavailable mkfs.ext4; focused image/machine tests, CGO0 machine/source tests and vet pass. CGO0 VZ OCI round-trip remains unsupported. No concrete VZ/GuestService driver, streamed publisher, recipe, secret/log stream or public HTTP machine mode: still a foundation.

All PRs remain draft. This is source-level/synthetic validation, not a new live boot or production build proof. Default Codex independent review was attempted but is still blocked by authentication (HTTP401). Published atomically after checking reviewer heads with explicit per-ref force-with-lease; prior heads preserved locally.

[chruffins]

Published 951e8f68, restacked on #500 (base updated), with explicit lease against prior ed219069. Still draft.

Implemented concrete GuestService/VZ machine driver, bounded tar.gz/versioned JSON recipe admission, instance-locked strict stop/export, normal-job cancellation/quarantine, and scoped-auth streaming publisher with independent persisted blob verification and manifest-last digest verification. Darwin shutdown now schedules power-off after acceptance so vsock teardown cannot race the reply.

Native isolated QA passed: ordinary queued build → root provisioning → source sanitation → acknowledged no-force shutdown + owned VMM exit → matching export → authenticated28.85GB streaming publication → shared image-readiness gate → fresh empty destination/cache HTTP pull → cold boot → root exec marker + source-absence check. Run483.74s, sampled test-process peak36.875MiB, minimum free313.17GiB. First attempt correctly refused export/publication when shutdown acknowledgement was lost. Final cold-boot test cleanup used ordinary short-timeout forced fallback; that stop was not used for export.

Evidence limits: activation remains internal Go CreateBuild, not HTTP POST /builds/SDK. Base was a manually installed isolated development template, not an automatically provisioned or credential-sanitized production image. Identity exclusivity preserved; no rekey/fork/parallel claim. Common safe recipe logs, clean-base policy, storage quotas, native repeat/cancel/recovery and Linux runtime regressions remain gates.

Repeated focused race regressions and supported full build suite pass; exactly five existing mkfs.ext4-dependent tests excluded on Darwin. Selected CGO0 synthetic tests and vet pass. Default independent autoreview remains401 authentication-blocked, not a completed clean review. All QA VMMs stopped/storage closed; original API and identity slot preserved. See updated docs/macos-builds.md for contracts and remaining gates.

[chruffins]

Published 0983495e, explicit lease against 951e8f68, still draft/base#500.

Default-off macos_builds now wires the concrete backend into the actual server provider. Requires macos_only and an explicit host registry URL. Existing authenticated POST /builds accepts machine_base_image (installed SHA256-pinned base), source, timeout_seconds, memory_mb, cpus and tags. Source64MiB/fields64KiB/full HTTP body65MiB (before OpenAPI validation); duplicate/unknown/Linux/cache/secret fields and malformed numbers rejected. Linux source parser remains unchanged; Linux servers explicitly reject the machine selector. OpenAPI and generated Go multipart request types updated; external language SDK release pipelines were not operated.

**Real HTTP/provider QA passed:**401 missing/invalid auth,403 read-only scope,202 POST /builds, ordinary queue → real guest provisioning/sanitation → acknowledged no-force export → verified scoped streaming publication → shared readiness → Ready. POST /instances cold-booted the returned image_ref; normal authenticated exec WebSocket verified UID0, the new marker and source absence. New output cold boot used converted cache; previous native run separately proved fresh pull. Private development template remains unsuitable for external distribution without credential sanitation. All QA storage closed/API stopped/slot released; original instance preserved.

Repeated focused API/config/provider/scopes race tests, full config/provider/scopes race tests and vet pass. Default independent autoreview remains401-blocked; no clean review claim. Still gates: external SDK releases, safe recipe logs, clean-base policy, storage quotas, native cancellation/recovery/repeat and Linux runtime validation. Updated docs/macos-builds.md includes opt-in configuration and curl example.

[github-actions]

-->

✱ stlc build

✅ go code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

116 files generated at b333a46 (pushed)

go get github.com/kernel/hypeman-go-staging@b333a4643223baee3c964cbf623058f64e190d40

✅ python code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

232 files generated at 557cf64 (pushed)

✅ typescript code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

138 files generated at 9d343e5 (pushed)

Diagnostics: ❗ 0 new / 1 total error, 💡 0 new / 5 total note

Level	Code	Message	Targets

Build metadata

Build	bd_76Xxls6j-green-rill
Timestamp	2026-10-15T00:32:02.530Z
stlc	8413509
Spec hash	e557b7c00082
Config hash	659c3687c3f0

---

This comment is auto-generated by stlc and is kept up to date as you push.
If you push new commits, re-run this workflow to update this comment.
Last updated: 2026-10-15 00:32:28 UTC