feat(sandbox): add storage-preserving suspend and resume lifecycle #2652

Problem Statement

Some harnesses use an OpenShell sandbox only as an execution environment while the harness, session controller, and agent orchestration run outside the sandbox. These harnesses know when a session is idle and expect to release the execution environment's compute resources until the next interaction without losing the session's working data.

Today, the practical choices are to keep the sandbox compute running while idle or delete the sandbox. Keeping it running consumes container, pod, or VM resources unnecessarily. Deleting it is the wrong lifecycle operation because it removes the sandbox identity and may remove the persistent workspace that the harness expects to reuse.

OpenShell needs first-class suspend and resume operations with a clear storage-preserving contract:

• Suspend shuts down the sandbox compute: the container, pod, or VM is stopped or terminated.
• The sandbox record and persistent workspace storage remain intact while suspended.
• Resume restores compute for the same sandbox and remounts/reuses the same workspace storage.
• The external harness can suspend on session idle and resume before the next execution request.

This is not a request to checkpoint or preserve RAM, process state, open connections, or accelerator state. The contract on resume is that the environment, primarily the writable workspace, is preserved. Any processes that were running when the suspend occurred are not resumed automatically and an external system must restart them.

Proposed Design

Add public sandbox lifecycle operations and expose them consistently through the CLI, gRPC API, and SDKs:

openshell sandbox suspend [NAME]
openshell sandbox resume [NAME]

Model the durable gateway lifecycle as:

Ready -> Suspending -> Suspended -> Resuming -> Ready

The operations should be idempotent. Suspend should only complete after compute is confirmed stopped. Resume should only complete after the backend is running and the sandbox supervisor has reconnected. Exec, connect, forwarding, and exposed-service routing should remain unavailable while the sandbox is suspended or transitioning.

The gateway should retain the sandbox identity and associated durable configuration, including policy, provider attachments, settings, service definitions, and workspace ownership. Ephemeral sessions and live connections should be revoked during suspend and recreated after resume.

Use driver-native lifecycle behavior:

• Docker and Podman: stop the existing managed container on suspend and start that same container on resume. Preserve its writable layer and attached workspace volumes. Delete remains the operation that removes the container and storage.
• Kubernetes: scale the managed Sandbox workload to zero while retaining the Sandbox resource and PVC, then scale it back to one. For Agent Sandbox APIs, use spec.operatingMode: Suspended|Running where supported and the version-appropriate replica field otherwise.
• VM: stop the VM compute and release transient host allocations while retaining its sandbox state directory and writable workspace overlay, then start a VM using that same retained overlay. This does not require a memory snapshot.

At the compute-driver boundary, define stop/suspend and resume as idempotent operations keyed by the stable sandbox ID. A missing backend resource should be reported distinctly from an already stopped or running resource. Delete must remain authoritative for removing compute and persistent storage.

Gateway persistence should treat Suspending, Suspended, and Resuming as durable intent so gateway restarts do not accidentally restart a suspended sandbox. Reconciliation must not interpret an expected stopped container or absent scaled-to-zero pod as a sandbox failure or orphan.

An automatic idle policy can be added separately. The initial feature should provide explicit operations so an external harness can make the idle/active decision using its own session knowledge.

Alternatives Considered

• Keep compute running while idle: preserves continuity but wastes CPU, memory, and pod/VM capacity for execution environments that may be inactive most of the time.
• Delete and recreate the sandbox: releases compute but conflates temporary idleness with destruction, churns sandbox identity and configuration, and can remove or complicate recovery of workspace storage.
• Checkpoint and restore full runtime state: useful for other workloads, but substantially more complex and backend-specific. External harnesses primarily need durable workspace continuity and can restart processes, so compute stop/start is a smaller and broadly implementable primitive.
• Driver-specific commands only: would force harnesses to know the active compute backend. A gateway-level lifecycle API gives harnesses one portable contract across containers, Kubernetes, and VMs.

Related issues cover adjacent or narrower scopes:

• Design checkpoint, pause, and resume support #1823 proposes a broader checkpoint/pause/resume design that may preserve serialized runtime state.
• feat(kubernetes): expose agent-sandbox operatingMode (suspend/resume) for idle scale-to-zero with PVC retention #2035 proposes the Kubernetes scale-to-zero and PVC-retention realization.
• feat(driver-vm): snapshot/restore (suspend/resume) for idle sandboxes #1551 discusses VM memory snapshot/restore for faster resume.

This issue defines the cross-driver public lifecycle whose required durable state is sandbox identity plus workspace storage, not memory checkpoint state.

Agent Investigation

Codebase exploration identified an existing internal ComputeDriver.StopSandbox seam and native backend mechanisms that fit this design: container stop/start, Agent Sandbox scale-to-zero with PVC retention, and VM state/overlay retention. The gateway already persists sandbox protobuf records and has lifecycle reconciliation patterns that can be extended with durable transition phases. Public gRPC methods, SDK mappings, CLI commands, authorization metadata, restart recovery, and cross-driver tests are required to make the behavior reliable and portable.