feat: Secure Docker daemon inside sandboxes #113

Problem

AI agents working on projects that involve container builds (e.g., nemoclaw itself, any Dockerized application) cannot run docker build, docker run, or interact with a Docker daemon from inside a sandbox. This blocks a significant class of development workflows — most notably nemoclaw development from within a nemoclaw sandbox.

Today there is no Docker daemon, socket, or runtime installed in the sandbox image, and multiple isolation layers actively prevent it:

• Seccomp: Blocks AF_NETLINK in proxy mode, which Docker bridge networking requires
• Landlock: Allowlisted filesystem paths do not include Docker runtime paths (/var/run/docker.sock, /var/lib/docker, etc.)
• Capabilities: Sandbox pods have SYS_ADMIN, NET_ADMIN, SYS_PTRACE but not full privileged mode
• Network namespace: The child process is isolated in a netns where all traffic routes through the HTTP CONNECT proxy

Context

The sandbox security model implements defense-in-depth with four layers: network namespace isolation, seccomp BPF, Landlock filesystem restrictions, and an HTTP CONNECT proxy with OPA policy evaluation. Any solution must integrate with — not circumvent — this model.

The sandbox already runs inside a Kubernetes pod on a k3s cluster inside a Docker container. This means we're looking at Docker-in-Kubernetes-in-Docker, which adds nesting complexity.

Proposed Approach (needs spike)

Several strategies are worth investigating:

1. Rootless Docker / Rootless Podman — Run a rootless container runtime inside the sandbox user namespace. This avoids needing privileged mode and may work within the existing capability set. Podman's daemonless architecture may be simpler to integrate.

2. Sysbox runtime — A purpose-built OCI runtime for running system workloads (including Docker) inside containers without privileged mode. Would require changes to the k3s node configuration.

3. Nested containerd with BuildKit — Since k3s already runs containerd, expose a scoped containerd socket or run a sandboxed BuildKit instance that the agent can use for image builds without a full Docker daemon.

4. Remote Docker / build delegation — Instead of running a daemon inside the sandbox, proxy Docker API calls to a dedicated build service running on the host cluster. The sandbox would have a Docker CLI that talks to a remote dockerd over a Unix socket or TCP, with policy enforcement at the proxy layer.

Security Considerations

• Container builds should not be able to escalate privileges beyond the sandbox boundary
• Network policy enforcement must still apply to containers built/run inside the sandbox
• Filesystem isolation (Landlock) rules need updating to allow Docker runtime paths while maintaining the security boundary
• Seccomp filter needs AF_NETLINK allowance (at minimum for the daemon process, ideally not for the agent process itself)
• Image pulls from inside the sandbox must route through the proxy and respect OPA policies
• The solution should not require granting the sandbox pod full privileged mode

Acceptance Criteria

• A sandbox can run docker build and docker run for basic development workflows
• The sandbox security boundary is maintained — container escapes from inner Docker do not compromise the host
• Network policy enforcement applies to traffic originating from inner containers
• The approach is documented in architecture/plans/
• Existing sandbox workflows (non-Docker) are unaffected
• Works with the BYOC (Bring Your Own Container) flow

Unlocks

• Nemoclaw self-hosted development (building and testing nemoclaw from inside a nemoclaw sandbox)
• Agent workflows involving container image builds, multi-service integration testing, and Docker Compose-based projects
• Broader applicability for any agent task that requires a container runtime