feat(network): corporate HTTP CONNECT upstream for sandbox egress in restricted environments #2232

Problem Statement

OpenShell sandboxes route ordinary egress through the supervisor's built-in policy proxy (OPA/L7 enforcement, credential injection, SSRF checks). After a connection is allowed by policy, the proxy currently opens a direct TCP connection to the resolved destination.

This works when the compute platform (Docker, Podman, VM, or Kubernetes pod/network namespace) can reach the public Internet directly. It breaks in restricted egress environments where outbound traffic must traverse a mandatory HTTP forward proxy (HTTP CONNECT), for example:

• Enterprise or IDC Kubernetes clusters with no default route to the public Internet
• Hosts that only permit egress to an internal proxy address (e.g. http://10.x.x.x:8080)
• Regulated networks where all external HTTP/HTTPS must go through a audited forward proxy

In these deployments, policy may correctly allow https://github.com, but the sandbox still fails with connection timeouts because the built-in proxy attempts a direct connect that the network blocks.

This is a general restricted egress / mandatory forward proxy problem, not specific to one geography. It affects Helm/Kubernetes deployments in particular as OpenShell moves toward enterprise use, and it applies to both the existing combined supervisor topology and the newer sidecar network supervisor topology: the sidecar enforces policy and container boundaries, but still needs a supported path to reach external destinations when direct egress is unavailable.

Proposed Design

Add an optional upstream HTTP CONNECT proxy configuration for the supervisor network proxy. This is a transport-layer hop after OpenShell policy has already allowed the connection. It must not widen the policy surface (denied destinations stay denied).

Behavior

1. Policy unchanged (first hop): Existing OPA/network policy evaluation runs as today (host/port, process/binary identity, L7 rules, SSRF/declared-endpoint checks). Denied requests never reach upstream configuration.

2. Upstream routing (second hop, optional): When configured, connections to public destinations (non-RFC1918, non-loopback, non-link-local, not on the always-blocked list) are established by sending CONNECT host:port to the configured upstream proxy and tunneling traffic through the returned connection.

3. Internal/cluster traffic unchanged: Destinations that resolve to internal addresses (ClusterIP Service CIDR, pod CIDR, loopback, link-local, host-gateway aliases used for in-cluster services) continue to connect directly, bypassing the upstream proxy. This preserves access to gateway callbacks, inference routes, and in-cluster dependencies (e.g. Astra/catalog endpoints) without forcing them through a corporate proxy.

4. No client-visible policy bypass: Processes still use the sandbox-local proxy (http_proxy / injected env pointing at the supervisor proxy). Upstream configuration is supervisor-internal only.

Configuration surface (proposal)

Because OPENSHELL_* environment keys are reserved and cannot be injected via CreateSandbox API environment maps, upstream settings should be owned by the gateway / compute driver, not sandbox clients:

• Environment (supervisor container): e.g. OPENSHELL_UPSTREAM_PROXY=http://proxy.example.com:8080
  • Accept http://host:port, https://host:port, or host:port (default port 3128 if omitted)
• Gateway TOML / Helm values (preferred for operators): e.g. under [openshell.drivers.kubernetes] (and inherited gateway defaults where appropriate), rendered into sandbox pod env by the Kubernetes driver
• Optional no_proxy analogue for upstream routing: if needed, document whether existing internal-IP detection is sufficient or whether an explicit bypass list is required for operator-controlled exceptions

Kubernetes / Helm notes

• Sandbox pods need L3/L4 egress to the upstream proxy address (separate from OpenShell policy). Cluster NetworkPolicy (or equivalent) remains the operator's responsibility to prevent bypass; this feature does not replace infra-level egress controls.
• Must work in combined and sidecar supervisor topologies (network supervisor container in sidecar mode performs the outbound connect).

Observability & failure modes

• Log when upstream proxy mode is enabled (without logging credentials).
• Surface clear errors when upstream CONNECT fails (timeout, non-200 response, connection refused) distinct from OPA policy denials.
• Document interaction with TLS interception/placeholder credential rewriting (upstream is reached only after policy allow; TLS to origin may still be terminated/re-encrypted by the supervisor as today).

Testing & documentation

• Unit tests for upstream URL parsing and public-vs-internal routing decisions.
• Integration/e2e test with a minimal mock HTTP CONNECT proxy that accepts CONNECT and forwards (or returns 200 with a stub tunnel).
• Docs: gateway config reference, Kubernetes restricted-egress deployment guide (upstream proxy + optional NetworkPolicy example).

Alternatives Considered

1. Rely on Kubernetes NetworkPolicy / service mesh egress gateway only

   • NetworkPolicy can block disallowed destinations but cannot redirect traffic to a forward proxy.
   • Service mesh (Istio/Envoy) operates at pod network namespace level and does not integrate with OpenShell's in-sandbox network namespace, OPA decisions, or credential/L7 rewriting. It adds operational complexity and duplicate proxy hops.

2. Configure a cluster-wide HTTP_PROXY on sandbox pods

   • Pointing user http_proxy at the corporate proxy bypasses OpenShell's built-in policy proxy and breaks the intended security model (binary binding, L7 rules, placeholder credential handling).

3. Require operators to open direct egress from sandbox pods

   • Often impossible in enterprise/IDC environments; defeats the purpose of restricted egress clusters.

4. Sidecar topology alone

   • Valuable for enforcement and non-root agent containers, but does not solve mandatory forward-proxy egress when the platform cannot reach public IPs directly.

Why upstream proxy in the supervisor is preferable: It preserves OpenShell's existing policy enforcement point and adds one optional transport hop only for allowed public destinations—minimal conceptual surface, aligned with how other enterprise runtimes expose HTTP_PROXY for themselves while keeping sandbox policy authoritative.

Agent Investigation

Exploration of current upstream main architecture (NVIDIA/OpenShell):

• Sandboxes route ordinary egress through the supervisor network proxy with OPA/L7 enforcement (openshell-supervisor-network; see docs on policy-enforced egress).
• After policy allow, outbound TCP currently uses direct TcpStream::connect to resolved destination addresses.
• Kubernetes supports combined and sidecar supervisor topologies (docs/kubernetes/topology.mdx, supervisor.topology Helm value). Sidecar mode adds nftables/pod-local enforcement but still performs policy-enforced egress to external destinations from the network supervisor.
• OPENSHELL_* keys in sandbox environment / template.environment are reserved for the controller; clients cannot inject supervisor-owned settings via CreateSandbox API (grpc/validation.rs).
• Official Helm chart / gateway TOML do not currently document a corporate upstream proxy setting for sandbox egress.
• A natural implementation locus is the supervisor network proxy connect path, gated after existing allow/SSRF/L7 checks, with driver/Helm-owned injection into sandbox workload env.

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request