Skip to content

test(conformance): migrate workspace lifecycle tests to workspace stories - #4607

Open
politerealism wants to merge 1 commit into
NVIDIA:codex/4119-suite-layoutfrom
politerealism:test/3954-workspace-stories/politerealism
Open

politerealism wants to merge 1 commit into
NVIDIA:codex/4119-suite-layoutfrom
politerealism:test/3954-workspace-stories/politerealism

Conversation

@politerealism

Copy link
Copy Markdown
Contributor

Summary

Move the workspace lifecycle tests from the legacy e2e crate into a new workspaces conformance area. Workspace create, inspect, scope and delete behavior isn't driver-specific, so it belongs with the other conformance stories and runs against any prepared gateway. Stacked on #4538, which relocates the suites.

Related Issue

Contributes to #3954. No separate issue: this migrates existing coverage.

Changes

  • Add a workspaces area (cli/tests/workspaces/) with three stories, and list it in the conformance README. The original workspace_full_crud_lifecycle checked seven behaviors in one test, so it's split to make a failure point at one behavior:
    • workspace_full_crud_lifecycle: create, get, listed as Active, delete when empty, gone afterwards. The name is kept so filters keep working.
    • template_is_scoped_to_its_workspace (new name): a resource is listed in its own workspace and in the all-workspaces view with its owner, and isn't visible in the default workspace.
    • workspace_terminating_rejects_creates: name kept. It now also covers the deletion guard, because a refused delete is what moves a workspace to Terminating, a phase in the public WorkspacePhase enum. While Terminating, new resources are rejected and existing ones can still be deleted; retrying the delete then succeeds. This is server behavior; the user docs don't describe the phase.
  • The scoped resource is a sandbox template, not a provider. The original used a provider of type openai, which needs a provider profile that only the wrapper lanes import. A freshly provisioned tmachine gateway has none, so those stories failed there (found by running the suite in the Ubuntu rootful Docker guest). A template is workspace-scoped, is covered by the same delete guard, and needs no profile, credential or image. This drops the provider list --all-workspaces check; the all-workspaces view is still checked, for templates.
  • Assertions check exit status and listed state from --output json, not message text. The CLI has no structured error and no structured workspace get, so those limits are documented in the area. Because a rejection by exit status could have another cause, the terminating story first creates and deletes the same template while the workspace is Active, so the later rejection can only be about the Terminating state.
  • Listings are paginated and capped at 100 pages. A missing workspaces or templates array in the output is an error, so a "not listed" check can't pass on a malformed listing.
  • A story removes its own resources. Cleanup runs only when a story fails, removes its templates and workspace, and reports anything other than "not found". The shared runner doesn't track workspaces, so this lives in small helpers (run, succeeds, fails, finish) in the area, and sandbox_templates has its own copy. I can move a shared version into the support crate if you want one.
  • Delete e2e/rust/tests/workspace_lifecycle.rs, its Cargo target and its entry in the Podman archive exclusion list.

Coverage after this change

The conformance archive and e2e_run_openshell_conformance run every test target in the conformance package, so the new workspaces binary runs in every lane that runs conformance. The old test was excluded from the Podman tmachine archive, so Podman gains coverage. The Docker and Kubernetes wrapper lanes run conformance first, so the coverage moves there. Kubernetes operator-workspace mode skips conformance (default workspace not allowlisted), as it did for the old test's lane, and the area documents that it isn't covered there.

Testing

  • mise run pre-commit passes
  • The three stories pass against a rootless Podman gateway: cargo test --manifest-path e2e/suites/conformance/Cargo.toml --test workspaces
  • Deliberate-failure checks on the same gateway, each failing exactly the intended story: the terminating story expecting Active, the CRUD story expecting Terminating, the scoping story looking in the default workspace, and the listing key missing from the JSON (fails the two stories that read workspaces).
  • tmachine guests, run on a throwaway branch that combines the three migrations: the full conformance archive passes in the Ubuntu rootful Docker guest (tmachine test ubuntu-docker-rootful binaries conformance) and in the Ubuntu K3s guest (tmachine test ubuntu-k3s k3s conformance). The first Docker run is what exposed the provider-profile dependency fixed in the workspaces PR, and it also failed once on the existing lifecycle stop_start (ContainerExited: Container exited while waiting for Ready), which passed on the rerun. Each PR alone was not run in tmachine.
  • Not run: macOS.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable)

@copy-pr-bot

copy-pr-bot Bot commented Oct 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…ries

Move the workspace lifecycle tests from the legacy e2e crate into a new
`workspaces` conformance area. Workspace create, inspect, scope and
delete behavior is not driver-specific, so it belongs with the other
conformance stories and runs against any prepared gateway.

The original `workspace_full_crud_lifecycle` checked seven behaviors in
one test. Split it so a failure points at one behavior:
- workspace_full_crud_lifecycle: create, get, list as Active, delete when
  empty, gone afterwards (name kept so filters keep working)
- template_is_scoped_to_its_workspace (new name): a resource is listed in
  its own workspace and in the all-workspaces view with its owner, and not
  in the default workspace
- workspace_terminating_rejects_creates: name kept. It now also covers the
  deletion guard, because a refused delete is what moves the workspace to
  Terminating, which is a public phase

The scoped resource is a sandbox template, not a provider. The original
used a provider of type `openai`, which needs a provider profile that
only the wrapper lanes import. A freshly provisioned tmachine gateway has
none, so those stories failed there. A template is workspace-scoped, is
covered by the same delete guard, and needs no profile, credential or
image. This drops the `provider list --all-workspaces` check; the
all-workspaces view is still checked, for templates.

Assertions check exit status and listed state from `--output json`
instead of matching message text. The CLI exposes no structured error and
no structured `workspace get`, so those limits are documented in the area.
The paginated listings are searched, so a busy gateway does not hide an
entry, and a missing collection in the output is an error.

A passing story removes what it created. Cleanup runs only after a
failure and reports anything other than "not found". The shared runner
does not track workspaces, so this stays in the area helpers.

Remove the old test file, its Cargo target and its Podman archive
exclusion, and list the area in the conformance README.

Contributes to NVIDIA#3954

Signed-off-by: politerealism <burdcat17@gmail.com>
@politerealism
politerealism force-pushed the test/3954-workspace-stories/politerealism branch from f5ff370 to a2554c4 Compare October 15, 2026 18:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant