feat(gpu): derive Docker sandbox requirements from CDI specs by elezar · Pull Request #4204 · NVIDIA/OpenShell

elezar · 2026-10-05T15:22:22Z

Summary

Derive Docker GPU sandbox access from the selected CDI specifications so device nodes, library mounts, and supplemental group requirements reach the workload's policy boundary. This combines the core resolver, sandbox consumer, and Docker producer into one reviewable change.

Related Issue

Related to #1606 (currently closed). Supersedes #2775, #2776, and #2265. The standard-sbin PATH repair remains a separate follow-up in #2846.

Changes

Add a versioned CDI context and Linux resolver, with fixtures for NVIDIA multi-GPU, Orin, and Spark layouts.
Resolve selected opaque device IDs and validate device-node types, safe paths, conflicting access, writable single-file mount opt-ins, and supplemental GIDs.
Carry Docker-selected device IDs through the protected workload boundary configuration and mount daemon-reported CDI spec directories read-only.
Derive workload filesystem permissions from CDI metadata and verify that the container runtime supplied required supplemental groups before launching agent processes.
Keep non-GPU workloads free of CDI context and spec mounts, and clean runtime state after failed provisioning.
Document the Docker CDI lifecycle and policy behavior.
Preserve Cargo's complete vendor source mapping so offline RPM dependency resolution includes the pinned CDI Git dependency.

Testing

mise run pre-commit passes after rebasing onto origin/main at 07a05f856.
cargo test --locked -p openshell-core -p openshell-policy -p openshell-sandbox -p openshell-driver-docker passes.
cargo test --locked -p openshell-supervisor-process -p openshell-sandbox-backend passes on the combined branch.
Resolver, boundary, and Docker unit coverage included.
RPM vendor archive extraction and offline workspace dependency resolution pass with empty Git and registry caches (verified before this conflict-free rebase; commit patches unchanged).
mise run ci: attempted; stops on an existing assertion in test:packaging-assets because tests/ansible/roles/openshell_packaged_gateway/tasks/main.yaml contains /var/lib/openshell-qualification/gateway.toml. Both files are unchanged from current origin/main.
Full Fedora RPM build and Docker GPU E2E validation remain pending CI.

Checklist

Follows Conventional Commits.
All four commits are signed off (DCO).
Relevant crate and published runtime/policy documentation updated.

Signed-off-by: Evan Lezar <elezar@nvidia.com>

github-actions · 2026-10-05T15:22:36Z

Label test:e2e-gpu applied, but pull-request/4204 does not exist yet. A maintainer needs to comment /ok to test 966d8eab99d30c0615105d07379de02af8695f28 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

github-actions · 2026-10-05T15:22:37Z

Label test:e2e applied, but pull-request/4204 does not exist yet. A maintainer needs to comment /ok to test 966d8eab99d30c0615105d07379de02af8695f28 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

github-actions · 2026-10-05T15:25:53Z

Label test:e2e applied for 966d8ea. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

github-actions · 2026-10-05T15:25:53Z

Label test:e2e-gpu applied for 966d8ea. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute GPU E2E after building the required supervisor image once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

github-actions · 2026-10-05T15:26:20Z

🌿 Preview your docs: https://nvidia-preview-pr-4204.docs.buildwithfern.com/openshell

drew · 2026-10-05T15:36:20Z

The new CDI spec bind mounts need to participate in the existing GPU trust exemption. With resource admission enabled by default, admit_container_resources() currently rejects these binds as unlabelable effective mounts, so GPU provisioning fails with OuterFenceRejected before the container starts.

We can treat GPU attachments and their driver-managed CDI spec projections as trusted. Please have admission recognize and allow the exact expected CDI mounts by validating their daemon-reported source directory, reserved workload destination, and read-only mode. Keep rejecting arbitrary host binds, including on GPU sandboxes, rather than skipping admission for the entire container.

Please also add a GPU provisioning/admission regression test with resource admission enabled; the current GPU test configuration disables it.

drew

Reviewed at 966d8ea. I found two validation issues, detailed inline: library-parent expansion bypasses broad-path rejection, and implicitly writable mounts skip the new opt-in checks. Please add regression cases for both.

The overall design keeps Docker responsible for device injection and resolves policy requirements inside the workload namespace.

The Docker troubleshooting skill (skills/debug-openshell-cluster/SKILL.md) also needs a companion update covering CDI projection and supplemental-group failures.

Validation: static review only; I did not run tests. CI was still running when checked, and the PR description reports Docker GPU E2E validation as pending.

drew · 2026-10-05T15:36:45Z

+        for (path, access) in self.mount_paths {
+            match access {
+                CdiAccess::ReadOnly => {
+                    read_only_paths.insert(shared_library_parent(&path).unwrap_or(path));


Validate the derived library parent before adding it. Before this PR, GPU enrichment used a fixed allowlist. Now a read-only CDI destination such as /libexample.so passes normalize_policy_path() and then expands here to /, granting read access far beyond that library within the outer sandbox boundary. validate_cdi_requirements() only checks that the resulting directory exists, so it does not catch this. Apply the broad-path rejection to the derived parent as well, and add a regression case for a library directly beneath / (and other rejected roots).

drew · 2026-10-05T15:36:45Z

+    if read_write_requested {
+        Ok(CdiAccess::ReadWrite)
+    } else {
+        Ok(CdiAccess::ReadOnly)


Do not infer read-only access from the absence of rw. This classifies options: [bind] or [rbind] as read-only, although a bind mount can preserve a writable source mount. Consequently, these mounts never reach the single-file and exact-path allowlist checks. For example, a writable CDI directory mounted at /tmp/device-data remains writable under an existing /tmp read-write policy, despite the new contract rejecting writable CDI directories. Require explicit read-only semantics or conservatively validate the mount as writable, with regression coverage for omitted ro/rw options. See Linux bind-mount semantics.

elezar added 4 commits October 5, 2026 17:15

feat(core): add CDI policy resolver

75a30bb

Signed-off-by: Evan Lezar <elezar@nvidia.com>

fix(rpm): retain vendored Git source mappings

d55f918

Signed-off-by: Evan Lezar <elezar@nvidia.com>

feat(sandbox): enforce CDI policy in workload boundary

0651b9c

Signed-off-by: Evan Lezar <elezar@nvidia.com>

feat(docker): project CDI metadata into GPU workloads

966d8ea

Signed-off-by: Evan Lezar <elezar@nvidia.com>

elezar requested review from a team, derekwaynecarr, mrunalp and sjenning as code owners October 5, 2026 15:22

elezar added test:e2e Requires end-to-end coverage test:e2e-gpu Requires GPU end-to-end coverage labels Oct 5, 2026

This was referenced Oct 5, 2026

fix(supervisor): preserve standard sbin paths in sandbox shells #2846

Open

feat(core): add CDI policy resolver #2775

Closed

feat(sandbox): derive policy from CDI context #2776

Closed

feat(docker): supply CDI context for GPU sandboxes #2265

Closed

elezar added test:e2e Requires end-to-end coverage test:e2e-gpu Requires GPU end-to-end coverage labels Oct 5, 2026

drew reviewed Oct 5, 2026

View reviewed changes

Navigation Menu

feat(gpu): derive Docker sandbox requirements from CDI specs - #4204

feat(gpu): derive Docker sandbox requirements from CDI specs#4204
elezar wants to merge 4 commits into
mainfrom
codex/1606-cdi-policy-docker

elezar commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

drew commented Oct 5, 2026

Uh oh!

drew left a comment

Uh oh!

drew Oct 5, 2026

Uh oh!

drew Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

elezar commented Oct 5, 2026

Summary

Related Issue

Changes

Testing

Checklist

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

drew commented Oct 5, 2026

Uh oh!

drew left a comment

Choose a reason for hiding this comment

Uh oh!

drew Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

drew Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants