Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
50a6c0f
test(testkit): add the testkit workspace for end-to-end suites
matthewgrossman Oct 19, 2026
94539c4
ci(tests): format, lint and test testkit in Branch Checks
matthewgrossman Oct 19, 2026
21c4aa9
docs(tests): describe how to write and run testkit tests
matthewgrossman Oct 19, 2026
69bdada
fix(testkit): give each sandbox its own cleanup timeout
matthewgrossman Oct 19, 2026
489720f
docs(testkit): show how to exclude tests by capability
matthewgrossman Oct 19, 2026
675e15b
docs: list tests/testkit in the project structure tables
matthewgrossman Oct 19, 2026
1f1c9e0
refactor(testkit): drop test IDs and capabilities for now
matthewgrossman Oct 19, 2026
35e356e
docs(testkit): describe what testkit provides today
matthewgrossman Oct 19, 2026
237601d
refactor(testkit): find the gateway the way the CLI does
matthewgrossman Oct 19, 2026
9ae29ab
fix(testkit): keep labels when creating a sandbox with a policy
matthewgrossman Oct 21, 2026
2c1d7a9
fix(testkit): use the registered gateway's credentials for an endpoint
matthewgrossman Oct 21, 2026
5714c29
docs(tests): list testkit's gateway variables in precedence order
matthewgrossman Oct 21, 2026
64f54cc
fix(testkit): apply other attributes to the generated test
matthewgrossman Oct 21, 2026
d6adcd6
fix(testkit): show usage for testkit-ready --help
matthewgrossman Oct 21, 2026
f2c07e7
test(testkit): cover cleanup after pass, failure, and panic
matthewgrossman Oct 21, 2026
fbf6029
fix(testkit): reject #[should_panic] on testkit tests
matthewgrossman Oct 21, 2026
c2dc21c
test(testkit): keep only the macro's misuse checks as unit tests
matthewgrossman Oct 21, 2026
5cf4df2
refactor(testkit): connect through the SDK's mTLS support
matthewgrossman Oct 22, 2026
6427db9
fix(testkit): bound readiness and cleanup with timeouts
matthewgrossman Oct 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/branch-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,7 @@ jobs:
cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all -- --check
cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all -- --check
cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all -- --check
cargo fmt --manifest-path tests/testkit/Cargo.toml --all -- --check

cargo-lockfiles:
name: Cargo lockfiles
Expand Down Expand Up @@ -277,6 +278,7 @@ jobs:
cargo clippy --locked --manifest-path examples/governance-interceptor/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path examples/isolation-backend-v8/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path tests/testkit/Cargo.toml --all-targets -- -D warnings

rust-test:
name: Rust test (${{ matrix.system }})
Expand Down Expand Up @@ -325,6 +327,7 @@ jobs:
run: |
cargo nextest run --locked --profile ci --workspace --features openshell-server/test-support
cargo nextest run --locked --config-file .config/nextest.toml --profile ci --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml
cargo nextest run --locked --config-file .config/nextest.toml --profile ci --manifest-path tests/testkit/Cargo.toml

rust-postgres-test:
name: Rust PostgreSQL tests
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ Do not rely on this file for a full inventory. The detailed public and contribut
| `python/openshell/` | Python SDK | Python bindings and CLI packaging |
| `sdk/typescript/` | TypeScript SDK | Native Connect client, curated sandbox API, and generated protobuf types |
| `proto/` | Protobuf definitions | gRPC service contracts |
| `tests/testkit/` | E2E testkit | Support library and `#[testkit::test]` attribute for SDK-driven E2E suites |
| `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests |
| `docs/` | Published docs | MDX pages, navigation, and content assets |
| `fern/` | Docs site config | Fern site config, components, and theme assets |
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ These are the primary `mise` tasks for day-to-day development:
| `sdk/typescript/` | TypeScript SDK (Connect client and generated protobuf bindings) |
| `proto/` | Protocol buffer definitions and [public API conventions](proto/README.md) |
| `tasks/` | `mise` task definitions and build scripts |
| `tests/testkit/` | Support library for SDK-driven end-to-end suites |
| `deploy/` | Dockerfiles, Helm chart, Kubernetes manifests |
| `docs/` | Published Fern docs source, navigation, and content assets |
| `fern/` | Fern site config, components, and theme assets |
Expand Down
11 changes: 11 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ crates/*/tests/ # Rust integration tests
python/openshell/ # Python unit tests (*_test.py suffix)
e2e/python/ # Python E2E tests (test_*.py prefix)
e2e/rust/ # Rust CLI E2E tests
tests/testkit/ # Support library for SDK-driven E2E suites
```

## Rust Tests
Expand Down Expand Up @@ -129,6 +130,16 @@ requires mTLS.
runtime. It covers the current schema and the Podman in-tree versus external
driver configuration boundary.

### testkit (`tests/testkit/`)

`testkit` is the support library for SDK-driven end-to-end suites. Tests are
`#[testkit::test]` functions with a required doc comment. They find the
gateway the same way the CLI does, through `OPENSHELL_GATEWAY_ENDPOINT`,
`OPENSHELL_GATEWAY`, or the active gateway, and `cx.cli` runs the CLI named
by `OPENSHELL_BIN`. See
[tests/testkit/README.md](tests/testkit/README.md) for how to write and run
tests.

### Python E2E (`e2e/python/`)

`mise run e2e:python` builds `openshell/e2e-python:dev` from
Expand Down
3 changes: 3 additions & 0 deletions tasks/rust.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ run = [
"cargo clippy --manifest-path examples/governance-interceptor/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path examples/isolation-backend-v8/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path tests/testkit/Cargo.toml --all-targets -- -D warnings",
]
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 lint native"
hide = true
Expand All @@ -45,6 +46,7 @@ run = [
"cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all",
"cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all",
"cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all",
"cargo fmt --manifest-path tests/testkit/Cargo.toml --all",
]
hide = true

Expand All @@ -56,6 +58,7 @@ run = [
"cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all -- --check",
"cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all -- --check",
"cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all -- --check",
"cargo fmt --manifest-path tests/testkit/Cargo.toml --all -- --check",
]
hide = true

Expand Down
1 change: 1 addition & 0 deletions tasks/test.toml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ run = [
"cargo test --workspace --exclude openshell-server",
"cargo test -p openshell-server --features test-support",
"cargo nextest run --config-file .config/nextest.toml --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml",
"cargo nextest run --config-file .config/nextest.toml --manifest-path tests/testkit/Cargo.toml",
]
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-precommit native"
hide = true
Expand Down
Loading
Loading