feat(proto): add opaque driver-specific passthrough field to SandboxTemplate #1492

Problem Statement

The gateway currently acts as a strict gatekeeper between the public API and compute drivers. Every driver-specific feature (node selectors, tolerations, GPU pinning, image pull secrets) requires:

1. A typed field on SandboxTemplate
2. Explicit forwarding logic in build_platform_config()
3. A coordinated release of gateway + client

This creates a bottleneck: driver capabilities can only be exposed to callers after gateway changes, even when the gateway doesn't need to understand or validate those fields.

With multiple drivers (Kubernetes, Docker, Podman, VM) having fundamentally different capabilities, forcing all driver-specific config through typed gateway fields creates either API bloat or lowest-common-denominator interfaces.

Prior Art: Kubernetes CSI StorageClass Parameters

This proposal follows a well-established pattern in Kubernetes. The Container Storage Interface (CSI) uses StorageClass.parameters as an opaque map[string]string passthrough:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "4000"
  throughput: "400"
  encrypted: "true"

The Kubernetes control plane (kube-controller-manager, scheduler) does not interpret, validate, or schema-check parameters. It passes them verbatim to the CSI driver, which is responsible for validating and consuming them. Each CSI driver documents its own supported keys (type, iops, fsType, skuName, etc.) independently of the Kubernetes release cycle.

This decoupling has proven highly successful:

• 50+ CSI drivers ship independently of Kubernetes releases
• No API bloat — Kubernetes never needed typed fields for iops, diskEncryptionSetID, skuName, etc.
• Driver velocity — new driver capabilities are exposed to users without any core API changes
• Clear ownership — the provisioner validates; the core passes through

The passthrough field proposed here mirrors this model: the gateway (analogous to kube-controller-manager) passes through an opaque struct to the compute driver (analogous to a CSI plugin), which validates and consumes driver-specific keys.

Proposed Design

Proto changes

Note: The field name platform_extensions used below is illustrative. The exact naming (e.g. driver_parameters, driver_config, platform_extensions) is open for discussion.

openshell.proto — public API:

message SandboxTemplate {
  // ... existing typed fields ...

  // Opaque driver-specific configuration provided by the caller.
  // Passed through to the compute driver without gateway interpretation.
  // The driver validates accepted keys via ValidateSandboxCreate.
  google.protobuf.Struct platform_extensions = <next_field_number>;
}

compute_driver.proto — driver API:

message DriverSandboxTemplate {
  // ... existing fields ...
  google.protobuf.Struct platform_config = 11;        // gateway-computed
  google.protobuf.Struct platform_extensions = 12;    // caller-provided, untouched by gateway
}

Gateway behavior

The gateway does NOT inspect, validate, or merge the passthrough field. It forwards the Struct as-is to the driver's corresponding field. This is a pure passthrough — the gateway remains agnostic to driver-specific schemas.

Driver validation

Two options for validating the passthrough field, in order of preference:

Option A — Driver-side validation only (preferred):
The driver is solely responsible for validating the passthrough field contents during ValidateSandboxCreate. Unknown or malformed keys are rejected with a descriptive error. The gateway has zero knowledge of what keys any driver accepts. This is the simplest model and mirrors CSI exactly.

Option B — Semi-structured pre-validation:
Drivers declare their accepted keys (and optionally value schemas) via GetCapabilities or a similar discovery RPC. The gateway can perform lightweight pre-validation (reject keys not in the driver's declared set) before forwarding to the driver. This catches obvious errors earlier in the request path but adds coupling between gateway and driver capability declarations.

Option A is preferred for initial implementation. Option B can be layered on later if operational experience shows that early rejection improves the user experience without excessive coupling.

Ownership model

This mirrors the CSI ownership model where Kubernetes owns the StorageClass API, the CSI driver owns the parameters schema, and the user (or operator) sets parameters knowing which provisioner they target.

Merge semantics

None at the gateway level. The driver receives both platform_config and the passthrough field as separate fields. If a driver wants to merge them, that's the driver's implementation choice. There are no override conflicts at the gateway layer.

Use Cases

CLI / TUI Surface

CLI/TUI support for the passthrough field is optional and orthogonal to this proposal. Several approaches are possible:

One natural pattern is a SandboxClass concept (analogous to Kubernetes StorageClass): a named, reusable bundle of driver-specific parameters that users reference by name rather than specifying raw keys on every sandbox creation. For example, openshell sandbox create --class gpu-a100 would resolve to a stored set of driver parameters (node selector, tolerations, resource requests) without the user needing to know the individual keys. This keeps the CLI ergonomic while the underlying passthrough remains a generic Struct.

Alternatively, the CLI could accept raw key-value pairs (--driver-param key=value), read them from a file, or a combination. The exact UX is left to a follow-up discussion — the important point is that the proto-level passthrough mechanism does not depend on any particular CLI surface.

Alternatives Considered

1. Typed fields for everything

Every driver-specific feature gets a typed field on SandboxTemplate + forwarding in build_platform_config().

• Con: Gateway becomes a bottleneck; proto changes needed per feature; lowest-common-denominator API across drivers.

2. Merge into platform_config at gateway level

Caller provides the passthrough, gateway merges keys into the single platform_config before sending to driver.

• Con: Override semantics are confusing; security risk (caller can override gateway-computed keys like grpc_endpoint); gateway needs allowlist logic.

3. Status quo (no passthrough)

Operators fork the gateway or wait for upstream changes.

• Con: Fork maintenance burden; slow feature velocity for driver-specific capabilities.

Compatibility

• Wire-safe: protobuf silently ignores unknown fields. Old gateways receiving the new field from new clients will ignore it (no error).
• Old clients that don't set the field are unaffected.
• Drivers that don't read the field are unaffected (simply unused).

Definition of Done

• Add passthrough field to SandboxTemplate in openshell.proto
• Add corresponding field to DriverSandboxTemplate in compute_driver.proto
• Gateway: forward the field through to DriverSandboxTemplate in driver_sandbox_template_from_public()
• K8s driver: read keys from the passthrough field
• K8s driver: validate known keys in validate_sandbox_create()
• Document supported keys per driver
• (Optional) CLI/TUI surface for setting driver parameters