Sandbox relay drops with TLS/h2 disconnection under sequential real workload (not reproducible in isolation) — v0.0.91 #2482

Agent Diagnostic

• Checked latest OpenShell release before testing: v0.0.91 (24 Jul 2026) — no newer release available.
• Searched for known/duplicate issues: found bug: ExecSandbox hangs indefinitely after the command exits when a supervisor session resets mid-exec #1990 ("h2 protocol error", "relay stream: inbound errored", "supervisor session: ended") — same symptom signature. Fix (PR fix(server): prevent exec relays from hanging on idle connections #1992, merged 26 Jun 2026: SSH keepalive, HTTP/2 keepalive, post-exit timeout) is already included in v0.0.91, which we run. No other open issue matching this exact pattern was found.
• Skills/tools loaded: read openshell-driver-docker README (network/host_gateway_ip config), gateway.toml, capability_broker docs (our own orchestration layer, ruled out as cause), tcpdump for packet-level capture, journalctl -k for kernel-level OOM check.
• Hypotheses tested with REAL workload runs (not isolated synthetic tests, which never reproduce this):
  1. CLI/gateway version mismatch (host CLI 0.0.89 vs gateway 0.0.91) → aligned both to 0.0.91 → no change.
  2. Resource starvation → added explicit --memory 8Gi --cpu 4 to sandbox create (previously unset) → no change, and journalctl -k shows no OOM-killer event in the failure window.
  3. OCSF/security policy denial → checked sandbox logs for DENIED entries in the failure window → none found.
  4. Rapid sequential exec calls racing the relay → added 100ms sleep between every sequential policy update/sandbox exec call → no change.
• Packet-level capture (tcpdump on openshell-net bridge + gateway port 18080) during a live failure: the sandbox-side connection lives ~3ms, sends a clean FIN, then 13 microseconds later the gateway writes one more 39-byte frame on that same connection, and the sandbox immediately responds RST. Full details in the Logs/Description fields below.
• Conclusion: the agent could not resolve this from the calling side — it appears to be internal to the gateway's relay/multiplexer, not something fixable via CLI flags, resource limits, or client-side timing.

Description

The gateway relay drops mid-session with an abrupt TLS/h2 disconnection ("h2 protocol error: error reading a body from connection" / "peer closed connection without sending TLS close_notify") during real sequential sandbox operations (policy update + multiple sandbox exec calls against the same sandbox). Expected: the relay stays up for the duration of normal sequential operations. Actual: it drops reproducibly under real workload, never under isolated single-command tests.

Reproduction Steps

1. openshell sandbox create --from --name --policy <policy.yaml>, wait for Ready.
2. Run 2+ sequential openshell policy update <sb> --add-endpoint ... --wait calls.
3. Run several sequential openshell sandbox exec -n <sb> -- sh -c '...' calls writing small (KB-sized) files.
4. On real multi-step runs (not single isolated commands), the relay drops with the h2/TLS error above.

Environment

• OS: Ubuntu 26.04 LTS, kernel 6.17.0-1018-nvidia (aarch64)
• Platform: NVIDIA DGX Spark (GB10, unified memory)
• Docker: 29.2.1 (build a5c7197)
• OpenShell CLI: v0.0.91
• OpenShell Gateway image: ghcr.io/nvidia/openshell/gateway:latest, digest matching v0.0.91
• Driver: openshell-driver-docker, dedicated bridge "openshell-net" (172.28.0.0/24, host_gateway_ip configured)
• Verified against latest release (v0.0.91) — same behavior.

Logs

Sandbox-side connection to gateway (172.28.0.3 -> 172.28.0.1:18080), captured with:
  tcpdump -i any -s 0 -w capture.pcap 'net 172.28.0.0/24 or port 18080'

~15 small HTTP/2 frames exchanged in ~3ms, then:
  13:09:34.587410  sandbox -> gateway  Flags [F.]   (clean half-close)
  13:09:34.587446  gateway -> sandbox  Flags [P.]   len=39  (one more frame, sent AFTER sandbox FIN)
  13:09:34.587459  sandbox -> gateway  Flags [R]           (immediate reset)

Total connection lifetime: ~3ms. No DENIED policy entries, no OOM-killer entries in host journalctl -k for this window.

Agent-First Checklist

• I pointed my agent at the repo and had it investigate this issue
• I loaded relevant skills (e.g., debug-openshell-cluster, debug-inference, openshell-cli)
• I checked the latest OpenShell release and either reproduced the issue there or explained why I cannot upgrade/test it
• I searched existing issues for possible duplicates or explained why I could not
• My agent could not resolve this — the diagnostic above explains why