refactor(e2e): align suites with testing strategy

[elezar]

Summary

Reorganize conformance, feature, and driver suites under e2e/suites and move shared Rust tooling to e2e/support/rust. Conformance stories are implemented directly in area modules rather than through a separate scenario crate and registry, aligning suite ownership and authoring with the testing-strategy proposal in #3460.

Based directly on main a9a635ab4. The shared-gateway Podman fixture work is in #4464, stacked on this PR.

Related Issue

Closes #4119. Part of #3954.

Changes

• Move behavioral suites from tests/suites to e2e/suites; retain target provisioning and artifact construction under tests.
• Extract the CLI runner, process executor, binary resolution, output parsing, and port utilities into openshell-e2e-support. Existing Rust e2e tests re-export the common helpers, and driver/feature suites use the shared dependency.
• Preserve test(conformance): run driver suites with cargo #3866's API cleanup and remove crates/openshell-conformance and its scenario registry. Organize conformance into file_transfer, lifecycle, policy_advisor, and smoke area binaries with story modules containing the tests and assertions.
• Split stop/start persistence and stopped-sandbox deletion into independent lifecycle stories. Keep pure policy-draft assertions together in draft_assertion.rs and preserve focused Cargo/nextest selection by story name.
• Preserve main's upload-to-existing-directory checks in the relocated round-trip story, including symlink, trailing-slash, rename, and overwrite cases.
• Update lockfiles, archive and wrapper paths, source checks, branch CI, and suite documentation. Correct Podman fixture inputs to use their relocated e2e/suites paths; retain the existing Ansible profile execution sequence.

Broader coverage migrations, result/waiver reporting, CI stream selection, disruption testing, and standalone distribution remain follow-ups.

Testing

Reduced-parent verification on 2026-10-13:

• nix develop --command mise run --jobs 1 pre-commit passes.
• Shared tooling tests (24), pure policy assertions (3), conformance all-target compilation, and legacy Podman target compilation pass.
• The three retained commits are unchanged; the Podman fixture rewrite is isolated as one commit in the stacked follow-up.

The combined tree (including #4464) passed the full configured unit suite and pre-commit on 2026-10-13 before splitting. Earlier conformance archive validation built all four area binaries. Live conformance and Keycloak runs have not been repeated on the current main base; full mise run ci and an x86_64 CI pass are not claimed.

Checklist

• Follows Conventional Commits.
• Commits are signed off (DCO).
• Suite documentation, testing guidance, CI guidance, and architecture inventory updated.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4120.docs.buildwithfern.com/openshell

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

Label test:e2e applied, but pull-request/4120 is at 33424e8 while the PR head is 846fa93. A maintainer needs to comment /ok to test 846fa93f597f89d0fa72678fe2881045918e54bb to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

[elezar]

/ok-to-test 846fa93

[elezar]

/ok-to-test b10ab39

[elezar]

/ok-to-test 6148ef6

[politerealism]

Thanks for this, @elezar. I've been working through the Podman exclusion list in #3712 and read the driver-suite part of this PR. Nothing here is a blocker, just a few ideas from that angle, mostly about what the next tests that need the same machinery would want.

1. The gateway fixture is shaped around user namespaces. Profile maps to --userns/--uidmap/--gidmap, and the baseline check rejects a config that already contains userns, uidmap or gidmap. A couple of other tests need the same "change the driver config, restart, check, restore" cycle with different keys (for example driver_config_volume needs allow_driver_config and enable_bind_mounts, and podman_corporate_proxy rewrites proxy settings). It might be nice if the overlay and the baseline check weren't tied to the userns keys, so a second test could reuse the fixture. I wouldn't ask for that in this PR; I'd be happy to take it as a follow-up and use driver_config_volume as the second consumer once this lands.

2. Maybe keep the host-control calls in one small module. The config path, the systemd unit name, the fixture directory and the sudo use are constants inside user_namespaces/gateway.rs. RFC 0016 talks about separating portable recovery assertions from environment-specific disruption actuators. Keeping "read config, write config, restart" in one place would let a second target (say, a Kubernetes gateway) swap it later without touching the tests. A plain module seems enough for now; no need for a trait until there's a second implementation.

3. The fixture pulls a registry image. DEFAULT_IMAGE is nvcr.io/nvidia/base/ubuntu:24.04. From the RFC thread, I understood the direction to be that images come from target preparation and tests refer to them by configured name. Using the preloaded or configured workload image here might avoid a registry pull at test time.

Mostly I wanted to check the direction with you before I start on driver_config_volume, so I don't duplicate or fight what you're building here. Does this match your plans for the fixture?

[elezar]

Thanks for this, @elezar. I've been working through the Podman exclusion list in #3712 and read the driver-suite part of this PR. Nothing here is a blocker, just a few ideas from that angle, mostly about what the next tests that need the same machinery would want.

1. The gateway fixture is shaped around user namespaces. Profile maps to --userns/--uidmap/--gidmap, and the baseline check rejects a config that already contains userns, uidmap or gidmap. A couple of other tests need the same "change the driver config, restart, check, restore" cycle with different keys (for example driver_config_volume needs allow_driver_config and enable_bind_mounts, and podman_corporate_proxy rewrites proxy settings). It might be nice if the overlay and the baseline check weren't tied to the userns keys, so a second test could reuse the fixture. I wouldn't ask for that in this PR; I'd be happy to take it as a follow-up and use driver_config_volume as the second consumer once this lands.

Yes, I think the "change config, restart, check, restore" workflow is the way to think about it. I haven't spent the right number of cycles cleaning that up here. I'll look at it a bit more closely before continuing. The first step for me was to move these out of Ansible.

2. Maybe keep the host-control calls in one small module. The config path, the systemd unit name, the fixture directory and the sudo use are constants inside user_namespaces/gateway.rs. RFC 0016 talks about separating portable recovery assertions from environment-specific disruption actuators. Keeping "read config, write config, restart" in one place would let a second target (say, a Kubernetes gateway) swap it later without touching the tests. A plain module seems enough for now; no need for a trait until there's a second implementation.

Getting the orgainisation cleaner is definitely something we want to look into, although we could also lean towards pulling out the common code once we have another consumer. Not going to push too hard if you think that this makes sense already.

3. The fixture pulls a registry image. DEFAULT_IMAGE is nvcr.io/nvidia/base/ubuntu:24.04. From the RFC thread, I understood the direction to be that images come from target preparation and tests refer to them by configured name. Using the preloaded or configured workload image here might avoid a registry pull at test time.

I'll look into this. That's a good callout. At the very least we should pull and prepare the input images beforehand to better allow the tests to operate in non-connected environments. The choice of ubuntu here is also most likely arbitrary.

Mostly I wanted to check the direction with you before I start on driver_config_volume, so I don't duplicate or fight what you're building here. Does this match your plans for the fixture?

Are you pulling driver_config_volume out as a driver-specific or a feature-specific test? In theory we have three drivers that support volumes via driver config so it COULD be a feature-specific tests. With that said, the driver-specific difference may be large enough to not make it worth it to try an unify.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.