[matthewgrossman]

Stack

Part of a stack of four; each PR targets the one below it. The tmachine move (#4726) has merged, so the stack uses its tests/tmachine/ paths.

1. test(testkit): add the testkit library for end-to-end suites #4724: the testkit library.
2. test(sandbox): run a testkit suite in Branch E2E #4822: the first testkit suite, run in Branch E2E against a Docker gateway. The simplest route to testkit tests in CI.
3. feat(runner): add the test runner, plans and tmachine Session API, and run a plan on CI #4773: the test runner, plans, and tmachine Session API, with a plan running on CI.
4. test(suites): port the universal conformance suites onto testkit and run them from the plan #4774: ports the old suites onto testkit and adds them to the plan.

Still to come, on top of #4774: moving the release lanes onto plans and deleting the old suites, once the plan has run the ported suites green.

Summary

Runs a testkit test in CI. A new testkit-e2e job in Branch E2E reuses the Docker E2E workflow to start a Docker-backed gateway and run mise run e2e:testkit, a plain cargo test of one new suite. There is no VM, runner, plan, or Nix archive.

The first test is adapted from the sandbox-lifecycle suite in @elezar's draft #4629.

Related Issue

Part of #3954

Changes

• tests/suites/sandbox/: a testkit suite workspace with long_running_exec_delete, plus a README with guidance on what a test may assert.
• mise run e2e:testkit: runs the suite under e2e/with-docker-gateway.sh. Same task locally and in CI.
• testkit-e2e job in branch-e2e.yml (label test:e2e): calls e2e-docker-test.yml with a one-entry suite matrix, after the binaries and images are built. It is not in core-e2e-result.needs, so it doesn't gate merges yet.
• fmt and clippy for the new workspace in Branch Checks and the mise tasks. TESTING.md describes the lane.

Deliberately cut for now

This is the simplest thing that runs testkit tests in CI. Left out on purpose, to come back in later PRs:

• tmachine and the test runner/plans. No VM guests, installed packages, per-environment capabilities, or plan-driven matrix. A separate branch (test/3954-plan-driven-ci/matthewgrossman) explores that.
• Lockfile alignment check and --locked Nix test archives. Not needed while the suite builds with plain cargo. They matter once testkit is built into Nix archives.
• Gating. The job isn't part of the Core E2E result.
• Other drivers and architectures. Docker only, on the arm64 runner the existing Docker E2E lane uses. No Podman, Kubernetes, k3s, rpm, or amd64 runs.
• JUnit upload and the tmachine disk cache key. There are no VM disks here, and nextest isn't used, so there is nothing to upload yet.
• Suite layout. The suite lives in tests/suites/sandbox for now. The planned conformance/features layout lands with the migration, when the old tests/suites/conformance goes away.
• Stable test IDs and requires. Not needed until a runner selects tests.

Testing

• Checks appropriate to the affected code and behavior pass
  • mise run e2e:testkit passes locally against a Docker gateway (1/1). fmt, clippy, and the lockfile check pass.
  • Not run yet: the testkit-e2e job itself. It needs Branch E2E (test:e2e) or a manual dispatch.
• E2E tests added/updated

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (TESTING.md)

🤖 Generated with Claude Code

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4822.docs.buildwithfern.com/openshell

[matthewgrossman]

Dropped from the stack: the plan-driven runner in #4773 runs the same smoke test on CI through the installed package, so this standalone Docker lane is redundant. The mise run e2e:testkit task moved into #4773 for local runs.