Skip to content

test(conformance): migrate sandbox template tests to template stories - #4608

Open
politerealism wants to merge 1 commit into
NVIDIA:codex/4119-suite-layoutfrom
politerealism:test/3954-sandbox-template-stories/politerealism
Open

politerealism wants to merge 1 commit into
NVIDIA:codex/4119-suite-layoutfrom
politerealism:test/3954-sandbox-template-stories/politerealism

Conversation

@politerealism

Copy link
Copy Markdown
Contributor

Summary

Move the sandbox template tests from the legacy e2e crate into a new sandbox_templates conformance area. Template create, inspect, list and use isn't driver-specific, so it belongs with the other conformance stories and runs against any prepared gateway. Stacked on #4538, which relocates the suites.

Related Issue

Contributes to #3954. No separate issue: this migrates existing coverage.

Changes

  • Add a sandbox_templates area (cli/tests/sandbox_templates/) with five stories, and list it in the conformance README. The first original test checked the template definition and a sandbox created from it in one test, so it's split to make a failure point at one behavior:
    • sandbox_template_round_trips_its_definition (new name): name, label, environment and resources survive create and get as JSON, and the template is listed. Resources are metadata here; enforcing them is driver-specific.
    • sandbox_create_from_template_uses_reusable_workload: name kept. A sandbox created from a template sees its environment, checked with sandbox exec and not the printed output of sandbox create.
  • The three negative tests keep their names and now assert state as well as exit status, instead of matching message text:
    • sandbox_template_get_after_delete_returns_not_found: the template is also gone from the listing.
    • sandbox_template_create_rejects_duplicate_name: the original template is unchanged (the second create used a different label).
    • sandbox_create_from_missing_template_returns_not_found: no sandbox was created. The same flags succeed for an existing template in the from-template story, so the rejection is about the template.
  • The CLI exposes no structured error, so those limits are documented in the area. A passing story deletes what it created; cleanup runs only after a failure and reports anything other than "not found". The helpers it uses (run, succeeds, fails, finish) are copied in the workspaces area; I can move a shared version into the support crate if you want one.
  • Delete e2e/rust/tests/sandbox_templates.rs and its Cargo target. The conformance archive and e2e_run_openshell_conformance run every test target in the conformance package, so the new binary runs in every lane that runs conformance, replacing the Podman tmachine archive coverage the old test had.

Testing

  • mise run pre-commit passes
  • The five stories pass against a rootless Podman gateway: cargo test --manifest-path e2e/suites/conformance/Cargo.toml --test sandbox_templates
  • Deliberate-failure checks on the same gateway, each failing exactly the intended story: a wrong CPU value, the wrong original label, a template still listed after delete, a wrong environment value, and the --template flag dropped so the missing-template create succeeds.
  • tmachine guests, run on a throwaway branch that combines the three migrations: the full conformance archive passes in the Ubuntu rootful Docker guest (tmachine test ubuntu-docker-rootful binaries conformance) and in the Ubuntu K3s guest (tmachine test ubuntu-k3s k3s conformance). The first Docker run is what exposed the provider-profile dependency fixed in the workspaces PR, and it also failed once on the existing lifecycle stop_start (ContainerExited: Container exited while waiting for Ready), which passed on the rerun. Each PR alone was not run in tmachine.
  • Not run: macOS.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable)

Move the sandbox template tests from the legacy e2e crate into a new
`sandbox_templates` conformance area. Template create, inspect, list and
use is not driver-specific, so it belongs with the other conformance
stories and runs against any prepared gateway.

The first original test checked the template definition and a sandbox
created from it in one test. Split it so a failure points at one
behavior:
- sandbox_template_round_trips_its_definition (new name): name, label,
  environment and resources survive create and get as JSON, and the
  template is listed. Resources are metadata here; enforcing them is
  driver-specific.
- sandbox_create_from_template_uses_reusable_workload: name kept. A
  sandbox created from a template sees its environment, checked with
  `sandbox exec` instead of the printed output of `sandbox create`.

The three negative tests keep their names and now assert state as well as
the exit status, instead of matching message text:
- get after delete: the template is also gone from the listing
- duplicate name: the original template is unchanged
- missing template: no sandbox was created

The CLI exposes no structured error, so those limits are documented in
the area. A passing story deletes what it created; cleanup runs only
after a failure and reports anything other than "not found".

Remove the old test file and its Cargo target, and list the area in the
conformance README.

Contributes to NVIDIA#3954

Signed-off-by: politerealism <burdcat17@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@politerealism
politerealism marked this pull request as ready for review October 15, 2026 16:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant