Sandbox SSH accepts any auth

@drew

Summary

The SSH server running inside the sandbox pod accepts both password-less (auth_none) and public-key authentication without checking the user or key. Every request is accepted. The design assumes authorization is enforced at the gateway (token + sandbox_id) and that the HMAC handshake proves the connection came from the gateway. If the gateway were bypassed or the handshake weakened, the SSH server would accept any connection.

Source Code

In crates/navigator-sandbox/src/ssh.rs, the SshHandler implementation of russh::server::Handler has auth_none (lines 202-203) returning Ok(Auth::Accept) and auth_publickey (lines 206-211) also returning Ok(Auth::Accept) for any user and any public key.

Originally by @drew on 2026-02-19T08:59:43.265-08:00

It's impossible to SSH directly to a sandbox from outside the Gateway, which is handling authentication via mTLS. To be verified by a review agent.

🔒 security-review-agent

Security Review

Determination: Legitimate concern (defense-in-depth hardening)

Summary

The claim is factually correct — auth_none and auth_publickey in crates/navigator-sandbox/src/ssh.rs (lines 256–265) both unconditionally return Auth::Accept. However, this is a deliberate, documented design choice (architecture/sandbox-connect.md, line 69). Three prior authentication layers (mTLS, session token, NSSH1 HMAC handshake) must all succeed before the SSH protocol begins. The real concern is not the SSH auth itself but two defense-in-depth gaps that could matter if an adjacent layer were compromised.

Severity Assessment

Impact: Low (properly configured) / Medium (if handshake secret is empty)
Exploitability: Requires cluster-internal network access at minimum; external exploitation is not possible. An in-cluster attacker would also need the HMAC handshake secret unless it's left empty.
Affected components:
- crates/navigator-sandbox/src/ssh.rs — SshHandler::auth_none (line 256), SshHandler::auth_publickey (line 260), verify_preface (lines 181–206)
- crates/navigator-sandbox/src/lib.rs — ssh_handshake_secret.unwrap_or_default() (line 407)
- crates/navigator-server/src/sandbox/mod.rs — conditional secret injection (line 918)

Attack Scenario

The SSH auth issue alone is not directly exploitable. The realistic attack path involves the empty handshake secret gap:

Attacker gains code execution in any pod within the same Kubernetes cluster (e.g., via a compromised workload)
Attacker discovers sandbox pod IPs on the flat cluster network and connects to port 2222
If NEMOCLAW_SSH_HANDSHAKE_SECRET is not configured, the HMAC check uses an empty-string key — the attacker can compute a valid NSSH1 preface (NSSH1 <token> <timestamp> <nonce> <hmac>)
The SSH server accepts the connection (auth_none → Accept)
Attacker lands in a sandboxed shell — confined by Landlock, seccomp, network namespace, and privilege dropping — limiting blast radius but still gaining access to the sandbox environment

Existing Mitigations (defense in depth)

Layer	Mechanism	Bypass Difficulty
External network	Sandbox pods not exposed via NodePort/LB	Requires cluster-internal access
Gateway mTLS	`WebPkiClientVerifier` — mandatory client cert (`tls.rs:42–44`)	Requires cluster CA-signed cert
Session token	Token lookup + revocation + sandbox-id match (`ssh_tunnel.rs:50–61`)	Requires valid `CreateSshSession` token
NSSH1 HMAC	HMAC-SHA256 with shared secret, timestamp, nonce (`ssh.rs:181–206`)	Requires handshake secret
Process sandbox	Landlock + seccomp + netns + privilege dropping	OS kernel enforcement

Remediation Plan

Make handshake secret mandatory: In crates/navigator-sandbox/src/lib.rs (line 407), fail startup if NEMOCLAW_SSH_HANDSHAKE_SECRET is empty or unset when SSH is enabled, rather than defaulting to "". Similarly enforce this on the server side in crates/navigator-server/src/sandbox/mod.rs (line 918).
Add Kubernetes NetworkPolicy: Create a NetworkPolicy resource that restricts ingress to sandbox pods on port 2222 to only the gateway pod. This eliminates the in-cluster lateral movement vector entirely.
Add NSSH1 nonce replay detection: Implement a short-lived cache of seen nonces (TTL matching the 300-second timestamp skew) to prevent replay attacks within the validity window.

Additional Notes

The unconditional Auth::Accept in the SSH handler itself does not need to change — adding SSH-level auth would be redundant given the three prior layers and would complicate the connection flow for no security gain.
The process-level sandbox (Landlock, seccomp, netns, setuid to unprivileged user, PR_SET_NO_NEW_PRIVS) provides strong containment even in the worst case, significantly limiting blast radius.
Items 1 and 2 in the remediation plan are the highest priority; item 3 is a nice-to-have hardening measure.

🔒 security-review-agent

Responding to the comment about mTLS preventing direct SSH access:

This is correct — external clients cannot SSH directly to a sandbox. The gateway enforces mandatory mTLS via WebPkiClientVerifier (crates/navigator-server/src/tls.rs:42–44), and sandbox pods have no externally-exposed ports (no NodePort or LoadBalancer for port 2222). All SSH traffic must traverse:

External client --mTLS--> Gateway (NodePort 30051) --NSSH1 HMAC--> Sandbox (port 2222)

However, the analysis did identify that in-cluster traffic (from other pods on the flat Kubernetes network) can reach sandbox port 2222 directly, bypassing the gateway entirely. This is mitigated by the NSSH1 HMAC handshake, but only when NEMOCLAW_SSH_HANDSHAKE_SECRET is properly configured. The remediation plan above addresses this gap with a mandatory secret and Kubernetes NetworkPolicy.

🏗️ build-plan

Implementation Plan

Issue type: fix
Complexity: Low-Medium
Confidence: High — clear paths for all three remediations, no architectural changes needed

Summary

Harden the SSH connection path between gateway and sandbox pods by (1) making the HMAC handshake secret mandatory at startup, (2) adding a Kubernetes NetworkPolicy to restrict sandbox SSH ingress to the gateway pod only, and (3) adding nonce replay detection to the NSSH1 handshake. These are defense-in-depth improvements — the primary auth layers (mTLS, session tokens) remain unchanged.

Scope

crates/navigator-sandbox/src/lib.rs: Fail startup if NEMOCLAW_SSH_HANDSHAKE_SECRET is empty/unset when SSH is enabled (currently silently defaults to "" at line 407)
crates/navigator-sandbox/src/ssh.rs: Add NonceCache (HashMap+Mutex) to run_ssh_server, pass to verify_preface, reject replayed nonces; spawn background reaper task
crates/navigator-sandbox/src/ssh.rs (tests): Add unit tests for verify_preface — valid preface, replay rejection, expired timestamp, bad HMAC, malformed input
crates/navigator-server/src/lib.rs: Validate ssh_handshake_secret is non-empty at server startup (around line 92, alongside existing database_url validation)
crates/navigator-server/src/sandbox/mod.rs: Remove conditional guard on secret injection (line 918) — always inject NEMOCLAW_SSH_HANDSHAKE_SECRET since startup validation guarantees it's non-empty
deploy/helm/navigator/templates/networkpolicy.yaml: New NetworkPolicy restricting sandbox pod ingress on port 2222 to gateway pods only
deploy/helm/navigator/values.yaml: Add networkPolicy.enabled: true default and sshHandshakeSecret field
deploy/kube/manifests/navigator-helmchart.yaml: Wire the handshake secret into HelmChart values
deploy/docker/cluster-entrypoint.sh: Generate a random handshake secret at cluster deploy time (similar to existing SSH_GATEWAY_HOST pattern)

Implementation Steps

Step 1: Make handshake secret mandatory (sandbox side)

In crates/navigator-sandbox/src/lib.rs:407, replace ssh_handshake_secret.unwrap_or_default() with a validation that returns an error if the secret is None or empty when ssh_listen_addr is Some. This ensures the sandbox process fails fast with a clear error message rather than silently running with no HMAC protection.

Step 2: Make handshake secret mandatory (server side)

In crates/navigator-server/src/lib.rs, add a startup check that config.ssh_handshake_secret is non-empty (after the existing database_url check around line 92). In crates/navigator-server/src/sandbox/mod.rs:918–919, remove the if !ssh_handshake_secret.is_empty() guard so the env var is always injected into sandbox pod specs.

Step 3: Wire secret through Helm/cluster deploy

Add sshHandshakeSecret to deploy/helm/navigator/values.yaml
In deploy/docker/cluster-entrypoint.sh, generate a random secret via openssl rand -hex 32 and inject it into the HelmChart values (similar to how SSH_GATEWAY_HOST is handled)
Update deploy/kube/manifests/navigator-helmchart.yaml to pass the secret through

Step 4: Add Kubernetes NetworkPolicy

Create deploy/helm/navigator/templates/networkpolicy.yaml that:

Selects sandbox pods via navigator.ai/managed-by: navigator
Allows TCP ingress on port 2222 only from pods matching app.kubernetes.io/name: navigator in the release namespace
Uses namespaceSelector to handle cases where gateway and sandbox are in different namespaces
Is gated behind networkPolicy.enabled in values.yaml (default: true)

Step 5: Add NSSH1 nonce replay detection

In crates/navigator-sandbox/src/ssh.rs:

Define NonceCache as Arc<Mutex<HashMap<String, Instant>>>
Create the cache in run_ssh_server() before the accept loop
Spawn a background tokio task that reaps expired entries every 60 seconds (TTL = handshake_skew_secs)
Pass the cache (via Arc::clone) to each connection handler
In verify_preface(), after HMAC verification succeeds, check the cache — reject if the nonce exists, otherwise insert it
No new dependencies needed (HashMap, Mutex, Instant from std)

Step 6: Add tests

verify_preface unit tests (in crates/navigator-sandbox/src/ssh.rs #[cfg(test)] module):
- Accepts a valid preface with correct HMAC, fresh timestamp, unique nonce
- Rejects replayed nonce (second call with same nonce returns false)
- Rejects expired timestamp (skew > handshake_skew_secs)
- Rejects invalid HMAC signature
- Rejects malformed preface (wrong part count, missing magic)
- Rejects empty secret (if we decide verify_preface should guard this too)
Startup validation tests (in respective #[cfg(test)] modules):
- Sandbox: run_sandbox fails when ssh_listen_addr is Some but secret is missing
- Server: startup rejects empty ssh_handshake_secret
Env injection test (in crates/navigator-server/src/sandbox/mod.rs tests):
- NEMOCLAW_SSH_HANDSHAKE_SECRET is always present in sandbox pod env

Step 7: Run pre-commit and verify

Run mise run pre-commit to ensure all checks pass (build, lint, tests, formatting).

Test Plan

Unit tests: verify_preface correctness (valid, replay, expired, bad HMAC, malformed) in crates/navigator-sandbox/src/ssh.rs; startup validation in crates/navigator-sandbox/src/lib.rs and crates/navigator-server/src/lib.rs
Integration tests: N/A — the changes are isolated to startup validation and a single function; Helm template correctness is verified by helm template during pre-commit
E2E tests: N/A — no changes under e2e/; the NSSH1 handshake is exercised by existing sandbox connect E2E tests once deployed

Risks & Open Questions

Backward compatibility: Making the secret mandatory is a breaking change for any deployment that currently runs without it. The Helm chart and cluster entrypoint changes mitigate this for standard deployments, but custom deployments will need to set the new required env var. This is acceptable — running without the secret was never intentionally supported.
NetworkPolicy CNI dependency: NetworkPolicy enforcement requires a CNI plugin that supports it (Calico, Cilium, etc.). k3s (dev clusters) supports this by default. Production deployments should verify their CNI.
Nonce cache memory: With a 300-second TTL and typical sandbox connection rates (< 1/sec), the cache will hold at most a few hundred entries. No memory concern.

Documentation Impact

architecture/sandbox-connect.md: Update to document the mandatory handshake secret requirement and the NetworkPolicy
Potentially architecture/plans/: None needed — this is a focused security hardening, not a new feature

Revision 1 — initial plan

fix(security): harden sandbox SSH with mandatory HMAC secret, Network…

🏗️ build-from-issue-agent

Implementation Complete

PR: #127

What was built

Three defense-in-depth hardening measures for sandbox SSH: mandatory HMAC handshake secret (server and sandbox refuse to start without it), Kubernetes NetworkPolicy restricting port 2222 ingress to the gateway pod, and NSSH1 nonce replay detection. No UX changes — the secret is auto-generated for cluster deployments.

Tests

Unit: 7 tests added (6 for verify_preface, 1 for env injection)
Integration: N/A
E2E: N/A

Docs updated

architecture/sandbox-connect.md: mandatory secret, nonce replay detection, NetworkPolicy

The issue will auto-close when the PR is merged.

fix(security): harden sandbox SSH with mandatory HMAC secret, Network…

feat(sandbox): L7 protocol-aware inspection with TLS termination (!29)

fix(security): harden sandbox SSH with mandatory HMAC secret, Network…

feat(sandbox): L7 protocol-aware inspection with TLS termination (!29)

fix: restore cluster_pods network policy for nemoclaw and openclaw (N…

Revert "fix: restore cluster_pods network policy for nemoclaw and ope…

Merge pull request NVIDIA#25 from LobsterTrap/fix/flannel-iptables-an…

Navigation Menu

Repository navigation

Sandbox SSH accepts any auth #25

Summary

Source Code

Security Review

Summary

Severity Assessment

Attack Scenario

Existing Mitigations (defense in depth)

Remediation Plan

Additional Notes

Implementation Plan

Summary

Scope

Implementation Steps

Step 1: Make handshake secret mandatory (sandbox side)

Step 2: Make handshake secret mandatory (server side)

Step 3: Wire secret through Helm/cluster deploy

Step 4: Add Kubernetes NetworkPolicy

Step 5: Add NSSH1 nonce replay detection

Step 6: Add tests

Step 7: Run pre-commit and verify

Test Plan

Risks & Open Questions

Documentation Impact

Implementation Complete

What was built

Tests

Docs updated

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Sandbox SSH accepts any auth #25

Description

Summary

Source Code

Activity

johntmyers commented on Mar 4, 2026

johntmyers commented on Mar 5, 2026

Security Review

Summary

Severity Assessment

Attack Scenario

Existing Mitigations (defense in depth)

Remediation Plan

Additional Notes

johntmyers commented on Mar 5, 2026

johntmyers commented on Mar 5, 2026

Implementation Plan

Summary

Scope

Implementation Steps

Step 1: Make handshake secret mandatory (sandbox side)

Step 2: Make handshake secret mandatory (server side)

Step 3: Wire secret through Helm/cluster deploy

Step 4: Add Kubernetes NetworkPolicy

Step 5: Add NSSH1 nonce replay detection

Step 6: Add tests

Step 7: Run pre-commit and verify

Test Plan

Risks & Open Questions

Documentation Impact

johntmyers commented on Mar 5, 2026

Implementation Complete

What was built

Tests

Docs updated

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions