Skip to content

Add a developer-only macOS sandbox surrogate - #296

Merged
jsturtevant merged 9 commits into
mainfrom
jsturtevant-dev-sandbox-host
Nov 12, 2026
Merged

jsturtevant merged 9 commits into
mainfrom
jsturtevant-dev-sandbox-host

Conversation

@jsturtevant

@jsturtevant jsturtevant commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Run macOS SDK sandboxes in a signed Rust surrogate so Python and dotnet do not need to be re-signed. Follow HIP-0004 with one sandbox per surrogate, a bounded reusable pool, postcard frames over an inherited socket, proxied tools, and file-backed snapshots.

This is developer-only support. macOS SDKs use this path by default; Linux Rust callers can opt in to the process API. Other SDK defaults stay unchanged. Windows transport is not included.

Native stack: #296 → #291 (Python wheels) → #293 (NuGet packages).

@jsturtevant
jsturtevant added this pull request to stack #297 November 12, 2026 03:28
@jsturtevant jsturtevant changed the title Add a developer-only macOS sandbox process host Add a developer-only macOS sandbox surrogate Nov 12, 2026
Run sandbox operations over an inherited Unix socket with bounded frames, typed errors, safe disconnect handling, and signed macOS build recipes.

Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Keep tool callbacks on the requesting thread, preserve cancellation and error types, and retain snapshot backing files safely across process reuse. Cover execution, disconnects, callbacks, snapshots, and interruption with regression tests.

Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Stage the signed worker beside native modules so Python applications do not need hypervisor entitlements. Keep other platforms on native execution and clean only the staged helper files.

Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Keep the public SDK API unchanged while moving macOS execution into the signed worker. Include the helper in the existing NuGet package and document the developer-only process boundary.

Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
@jsturtevant
jsturtevant force-pushed the jsturtevant-dev-sandbox-host branch from b9cbb1e to 3f07e04 Compare November 12, 2026 22:04
@jsturtevant
jsturtevant marked this pull request as ready for review November 12, 2026 22:07
@jsturtevant
jsturtevant requested a max review from Copilot November 12, 2026 22:07
Exercise macOS execution through the existing Python unit tests, examples, and fuzzing. Build native ARM64 guests on the HVF runner instead of reusing Linux artifacts.

Signed-off-by: James Sturtevant <jsturtevant@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved fork handling can hang child processes and invalidate parent connections, while delayed argument-size checks expose the host to guest-triggered memory pressure.

10 open issues
What changed in this PR

Adds a developer-only surrogate layer for macOS SDKs, moving sandbox execution into a signed Rust helper so Python and .NET hosts do not need to be re-signed. This provides the foundation for the linked wheel and NuGet packaging work.

Changes:

  • Introduces a Unix worker with bounded pooling, framed messaging, tool callbacks, interruption, and file-backed snapshots.
  • Routes macOS native bindings through the worker while preserving other SDK defaults.
  • Adds helper staging, build recipes, cleanup coverage, and CI integration.
File Description
src/​sdk/​python/​wasm_backend/​src/​lib.rs Selects worker-backed sandboxes and snapshots on macOS.
src/​sdk/​python/​wasm_backend/​pyproject.toml Includes the staged helper in backend wheels.
src/​sdk/​python/​wasm_backend/​Cargo.toml Adds the macOS worker dependency.
src/​sdk/​python/​pyo3_common/​src/​lib.rs Selects worker interrupt handles on macOS.
src/​sdk/​python/​pyo3_common/​Cargo.toml Adds the shared bindings’ macOS worker dependency.
src/​sdk/​python/​Justfile Stages helpers for development and wheels; removes staged files during cleanup.
src/​sdk/​python/​hyperlight_js_backend/​src/​lib.rs Selects worker-backed JavaScript sandboxes on macOS.
src/​sdk/​python/​hyperlight_js_backend/​pyproject.toml Includes the helper in JavaScript backend wheels.
src/​sdk/​python/​hyperlight_js_backend/​Cargo.toml Adds the macOS worker dependency.
src/​sdk/​dotnet/​ffi/​src/​lib.rs Selects worker sandbox, snapshot, and interrupt types on macOS.
src/​sdk/​dotnet/​ffi/​Cargo.toml Adds the macOS worker dependency.
src/​sdk/​dotnet/​core/​PInvoke/​HyperlightSandbox.PInvoke.csproj Builds and stages the signed macOS helper.
src/​sandbox_worker/​tests/​process.rs Covers process lifecycle, callbacks, snapshots, pooling, and cancellation.
src/​sandbox_worker/​src/​server.rs Executes sandbox operations and proxies callbacks inside the helper.
src/​sandbox_worker/​src/​protocol.rs Defines framed requests, responses, compatibility checks, and remote errors.
src/​sandbox_worker/​src/​main.rs Adds the private worker executable entry point.
src/​sandbox_worker/​src/​lib.rs Exposes Unix process APIs and the server entry point.
src/​sandbox_worker/​src/​client.rs Implements worker connections, pooling, snapshots, and sandbox operations.
src/​sandbox_worker/​README.md Documents developer-only scope, configuration, and lifecycle behavior.
src/​sandbox_worker/​Justfile Adds helper build, signing, test, and lint recipes.
src/​sandbox_worker/​Cargo.toml Defines the worker crate and its dependencies.
src/​sandbox_worker/​build.rs Generates a protocol and dependency compatibility fingerprint.
src/​hyperlight_sandbox/​src/​tools.rs Exposes registered tool names for callback proxying.
scripts/​tests/​test_python_clean.py Checks helper cleanup without removing Python package files.
Justfile Integrates worker build, lint, and test recipes.
Cargo.toml Registers the worker crate in the workspace.
Cargo.lock Records worker dependencies and updated dependency relationships.
.gitignore Ignores generated helpers staged in Python packages.
.github/​workflows/​ci.yml Runs worker lint and integration checks on Unix runners.

Review effort: Max

Comment thread src/sandbox_worker/src/client.rs Resolved
Comment thread src/sandbox_worker/src/server.rs Resolved
Comment thread src/sandbox_worker/Justfile Resolved
Comment thread src/sandbox_worker/src/client.rs Resolved
Comment thread src/sandbox_worker/src/client.rs Resolved
Comment thread src/sandbox_worker/src/client.rs Resolved
Comment thread src/sandbox_worker/src/client.rs Resolved
Comment thread src/sandbox_worker/src/protocol.rs Resolved
Comment thread src/sandbox_worker/src/protocol.rs Resolved
Comment thread src/sandbox_worker/src/server.rs Resolved
@jsturtevant
jsturtevant force-pushed the jsturtevant-dev-sandbox-host branch from 5ff28d8 to ccd02b8 Compare November 12, 2026 22:47
Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
Signed-off-by: James Sturtevant <jsturtevant@gmail.com>
@jsturtevant
jsturtevant merged commit 090bc88 into main Nov 12, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants