From c11099030a4a3af86d3ae7c7d775dc9644ca7684 Mon Sep 17 00:00:00 2001 From: kerthcet Date: Sat, 17 Oct 2026 12:37:06 +0200 Subject: [PATCH 1/4] docs: bring the docs in line with the code - DEVELOP.md: fix the project layout (protocol/ crate, python.rs, snapshot/), drop references to files that do not exist (test_build.sh, simple_test.py, agent_example.py, protocol.rs, STATUS.md), and document the real make targets, test markers and release flow. - QUICKSTART.md: use make targets, and list_daemons() returning DaemonInfo. - ARCHITECTURE.md: stop claiming daemons authenticate; describe the planned Keycloak-based auth instead. - README.md: file upload is not implemented on the daemon; list snapshots and labels/broadcast; add auth and file upload to the roadmap. - PROTOCOL.md: point at protocol/src/lib.rs. Co-Authored-By: Claude Opus 5.5 Signed-off-by: kerthcet --- README.md | 8 +- docs/ARCHITECTURE.md | 21 +++-- docs/DEVELOP.md | 189 +++++++++++++++++-------------------- docs/QUICKSTART.md | 30 +++--- docs/proposals/PROTOCOL.md | 2 +- 5 files changed, 121 insertions(+), 129 deletions(-) diff --git a/README.md b/README.md index 2b86efd..cef6e54 100644 --- a/README.md +++ b/README.md @@ -18,11 +18,13 @@ Rust-powered WebSocket server with Python API for remote command execution and i - **Command Execution** - Run shell commands on remote machines with timeout control - **Interactive Sessions** - Full PTY sessions with bash for manual work -- **File Transfer** - Upload/download files between controller and workers +- **File Download** - Fetch files from workers (upload is not implemented yet) +- **Snapshots** - Save, tag and restore workspace state on a worker +- **Labels & Broadcast** - Filter workers by label and run a command on many at once - **Tunnel Mode (VPN)** - Secure mesh networking with WireGuard encryption via Tailscale - **High Performance** - Rust async runtime handles high-concurrency workloads - **Auto Reconnection** - Workers reconnect automatically on network failures -- **Cross-Platform** - Linux, macOS, Windows support +- **Cross-Platform** - Linux and macOS release binaries; Windows builds from source ## Architecture @@ -182,6 +184,8 @@ See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. ## Roadmap +- [ ] **Authentication & Authorization** - Keycloak-issued tokens for daemons (today any client that speaks `sandd.v1` is admitted; see [Architecture](./docs/ARCHITECTURE.md#future-authentication)) +- [ ] **File Upload** - Write uploaded files on the daemon - [ ] **Audit Logging** - Track all commands, sessions, and file transfers - [ ] **Metrics** - Prometheus-compatible metrics for monitoring - [ ] **Resource Limits** - CPU/memory/timeout controls per daemon diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index cb91dec..4133d31 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -16,7 +16,7 @@ - ✅ No ports exposed on execution plane (security) - ✅ Works through NAT/firewalls - ✅ Easy to add/remove daemons dynamically -- ✅ Agent controls access (daemon authenticates to agent) +- ✅ Agent decides what runs (daemons only execute what the agent sends) ### Why Rust Server? @@ -56,12 +56,15 @@ Forward raw SSH traffic through WebSocket to daemon's sshd. ### Future: Authentication -```rust -Message::Register { - daemon_id: String, - auth_token: String, // JWT or pre-shared key - metadata: DaemonMetadata, -} -``` +Today the controller admits any client that speaks `sandd.v1`, and a daemon can +register under any id. Run it on a trusted network, or in tunnel mode, where only mesh +members can reach it. + +The plan is to use Keycloak for authentication and authorization: -Agent validates token before accepting connection. +- The daemon gets an access token from Keycloak and sends it as + `Authorization: Bearer ` on the WebSocket upgrade. +- The controller verifies the token against Keycloak's JWKS endpoint before upgrading, + and rejects a bad one with HTTP 401. +- The token's claims decide which daemon id the connection registers as, and what it + is allowed to do. diff --git a/docs/DEVELOP.md b/docs/DEVELOP.md index f6e1d3c..d409949 100644 --- a/docs/DEVELOP.md +++ b/docs/DEVELOP.md @@ -6,98 +6,90 @@ This guide is for developers working on SandD itself. - **Rust**: Install from https://rustup.rs/ - **Python 3.8+**: With pip -- **Maturin**: `pip install maturin` +- **Docker**: Only for the e2e tests and images + +The Makefile creates `.venv/` and installs maturin, pytest and ruff into it as needed. ## Project Structure ``` SandD/ -├── server/ # Rust server with PyO3 bindings +├── protocol/ # Shared message types (sandd-protocol crate) +│ └── src/lib.rs +│ +├── server/ # Controller: Rust WebSocket server + PyO3 bindings │ ├── src/ -│ │ ├── lib.rs # Python API bindings -│ │ ├── server.rs # WebSocket server (axum) +│ │ ├── lib.rs # Crate root +│ │ ├── server.rs # WebSocket server (axum): /ws, /stats, /health │ │ ├── registry.rs # Daemon connection registry -│ │ └── protocol.rs # Message protocol +│ │ └── python.rs # Python API bindings (behind the `python` feature) │ └── Cargo.toml │ -├── sandd/ # Rust daemon binary +├── sandd/ # Daemon binary (runs on workers) │ ├── src/ -│ │ ├── main.rs # Daemon entry point +│ │ ├── main.rs # Entry point, connection loop, tunnel setup │ │ ├── executor.rs # Command execution │ │ ├── session.rs # Interactive sessions (PTY) -│ │ └── protocol.rs # Message protocol +│ │ └── snapshot/ # Workspace snapshots +│ ├── benches/ # Criterion benchmarks │ └── Cargo.toml │ -├── python/sandd/ # Python package wrapper -│ └── __init__.py +├── python/ +│ ├── sandd/ # Python package (Server, models, tunnel keys; AsyncServer is a stub) +│ └── tests/ # Unit, integration and e2e tests │ +├── hack/ # Dockerfiles, compose stacks, install script └── examples/ # Usage examples ``` ## Building -### Quick Build (Development) - ```bash -# Build everything -./test_build.sh - -# Or step by step: -make dev # Build Python package (debug) -make daemon-build # Build daemon (debug) -``` - -### Release Build +make dev # Build and install the Python package (debug, editable) +make daemon-build # Build the daemon (debug): ./target/debug/sandd -```bash make release # Python package (optimized) -make daemon-release # Daemon binary (optimized) +make daemon-release # Daemon binary (optimized): ./target/release/sandd ``` -### Manual Build +Run `make help` for the full list of targets. -```bash -# Python package -maturin develop --release -m server/Cargo.toml - -# Daemon -cargo build --package sandd --release -``` +The `python` feature of `server/` is off by default, so plain `cargo build` and +`cargo test` work without a Python interpreter. Maturin turns it on when building the +wheel (see `[tool.maturin]` in `pyproject.toml`). ## Development Workflow ### 1. Make Changes -Edit files in `server/src/`, `sandd/src/`, or `python/sandd/` +Edit files in `protocol/src/`, `server/src/`, `sandd/src/`, or `python/sandd/`. ### 2. Rebuild ```bash -# If you changed server/ +# If you changed protocol/, server/ or python/sandd/ make dev -# If you changed daemon/ +# If you changed protocol/ or sandd/ make daemon-build - -# If you changed Python wrapper only -# (no rebuild needed, it's just a wrapper) ``` -### 3. Test +### 3. Try It ```bash -# Terminal 1: Start test agent -python3 examples/simple_test.py - -# Terminal 2: Start daemon -./target/debug/sandd --server-url ws://127.0.0.1:8765/ws +# Terminal 1: start a controller +python3 -c " +from sandd import Server +server = Server() +server.wait_for_daemon('dev-1', timeout=60) +print(server.exec('dev-1', 'uname -a').stdout) +" + +# Terminal 2: start a daemon +./target/debug/sandd --server-url ws://127.0.0.1:8765/ws --daemon-id dev-1 ``` -### 4. Run Examples - -```bash -python3 examples/agent_example.py -``` +See [examples/](../examples) for sessions, snapshots and tunnel mode. ## Architecture @@ -130,39 +122,35 @@ file_transfers: Vec> // Chunked buffering ## Testing -### Unit Tests - -```bash -cargo test --workspace -``` - -### Integration Tests - ```bash -pytest tests/ # (when tests are added) +make test # Lint + Rust tests + Python tests (no Docker) +make test-e2e # Direct-mode e2e against daemons in Docker (slow) +make test-e2e-tunnel # Tunnel-mode e2e with a Headscale mesh in Docker (slow) +make lint # ruff check +make format # ruff fix + format +make benchmark # Daemon benchmarks (criterion) ``` -### Manual Testing - -Use `examples/agent_example.py` to test all features. +Python tests are selected by marker: `e2e` for Docker-based tests, `tunnel` for the +tunnel stack. `make test` runs `-m "not e2e"`. ## Common Tasks -### Adding a New Command Type +### Adding a New Message Type -1. Add to `protocol.rs` (both server and daemon): +1. Add the variant to `Message` in `protocol/src/lib.rs` (shared by server and daemon): ```rust - Message::MyNewCommand { field: String } + MyNewCommand { field: String } ``` -2. Handle in `server/src/server.rs`: +2. Handle it in `server/src/server.rs`: ```rust Message::MyNewCommand { field } => { // Forward to daemon or handle } ``` -3. Handle in `sandd/src/main.rs`: +3. Handle it in `sandd/src/main.rs`: ```rust Message::MyNewCommand { field } => { // Execute and respond @@ -171,7 +159,7 @@ Use `examples/agent_example.py` to test all features. ### Adding Python API -1. Add method to `Server` in `server/src/lib.rs`: +1. Add a method to `Server` in `server/src/python.rs`: ```rust #[pymethods] impl Server { @@ -181,7 +169,7 @@ Use `examples/agent_example.py` to test all features. } ``` -2. Add wrapper in `python/sandd/__init__.py`: +2. Add a wrapper in `python/sandd/server.py`: ```python def my_method(self, arg: str) -> None: """User-friendly docstring""" @@ -195,30 +183,21 @@ Use `examples/agent_example.py` to test all features. RUST_LOG=debug ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws ``` -**Python side:** -```python -import logging -logging.basicConfig(level=logging.DEBUG) -``` +**Controller logs:** `Server(verbose=True)` (the default) logs at INFO; set +`RUST_LOG=sandbox_server=debug` for more. -**Check WebSocket traffic:** -```bash -# In server -RUST_LOG=server=debug python3 examples/simple_test.py -``` - -## Known Issues & Limitations (MVP) +## Known Issues & Limitations -### Not Implemented - -1. **File Transfer**: Protocol defined, daemon just logs - - Reason: Deferred for MVP - - Fix: Implement actual file I/O in daemon +1. **File upload is not implemented.** `Server.upload_file` sends the data, but the + daemon only logs the chunks and writes nothing. The call still returns without an + error. `download_file` works. +2. **No daemon authentication.** Any client that speaks `sandd.v1` can register as any + daemon id. Run the controller on a trusted network or in tunnel mode. See + [ARCHITECTURE.md](ARCHITECTURE.md#future-authentication). ### Warnings - PyO3 `non_local_definitions` warning: Safe to ignore (PyO3 macro limitation) -- Unused imports/variables: Run `cargo fix` to clean up ## Performance @@ -232,21 +211,22 @@ At 200 concurrent daemons: ### Before Submitting PR -1. Run tests: `cargo test --workspace` -2. Check formatting: `cargo fmt --all` -3. Check lints: `cargo clippy --all` -4. Test manually with examples -5. Update docs if adding features +1. Run `make test` +2. Check formatting: `cargo fmt --all` and `make format` +3. Check lints: `cargo clippy --workspace --all-targets` +4. Update docs if adding features ### Commit Style ``` -Add feature: brief description +feat: brief description More detailed explanation if needed. Include motivation and context. ``` +Use a `feat:`, `fix:` or `chore:` prefix, as in the existing history. + ### Adding Dependencies - Keep dependencies minimal @@ -255,23 +235,23 @@ Include motivation and context. ## Release Process -1. Update version in `pyproject.toml` and `Cargo.toml` files -2. Update `CHANGELOG.md` (when added) -3. Build release: `make release && make daemon-release` -4. Test release build -5. Tag: `git tag v0.x.0` -6. Build wheel: `maturin build --release -m server/Cargo.toml` -7. Publish: `maturin publish` (when ready) +1. Update the version in `pyproject.toml`, `protocol/Cargo.toml`, `server/Cargo.toml` + and `sandd/Cargo.toml` (and the `sandd-protocol` dependency version in + `sandd/Cargo.toml`) +2. Tag and push: `git tag v0.x.y && git push upstream v0.x.y`. The `Release` workflow + builds the daemon binaries and creates the GitHub release. +3. Publish packages: `make publish` (wheels to PyPI, needs `INFTYAI_PYPI_TOKEN`; crates + to crates.io) ## Troubleshooting ### User Issues **Daemon won't connect:** -- Check agent URL is reachable: `curl -v ws://agent-host:8765/ws` +- Check the controller is up: `curl http://controller-host:8765/health` - Verify firewall allows outbound WebSocket connections -- Check agent server is running: `ps aux | grep python` - Check daemon logs: `RUST_LOG=info ./target/release/sandd ...` +- Check what the controller sees: `curl http://controller-host:8765/stats` **Commands timing out:** - Increase `timeout` parameter in `exec()` (in seconds) @@ -306,6 +286,14 @@ Include motivation and context. **Fix:** Use `-m server/Cargo.toml` flag +### Linker errors about undefined `_Py*` symbols + +**Symptom:** `cargo build`/`cargo test` of `server/` fails to link with missing CPython +symbols + +**Fix:** Don't enable the `python` feature outside maturin. It turns on +`pyo3/extension-module`, which expects an interpreter to supply those symbols. + ## Protocol WebSocket-based JSON protocol for agent-daemon communication. @@ -323,5 +311,4 @@ For complete protocol specification, see [proposals/PROTOCOL.md](proposals/PROTO - Check [ARCHITECTURE.md](ARCHITECTURE.md) for design details - Check [proposals/PROTOCOL.md](proposals/PROTOCOL.md) for protocol specification -- Check [STATUS.md](STATUS.md) for implementation status - Check [QUICKSTART.md](QUICKSTART.md) for usage examples diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index 7b97aa4..4180ac4 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -3,13 +3,13 @@ ## 1. Build Everything ```bash -# Install dependencies -pip3 install maturin - -# Build and install -./test_build.sh +make dev # Python package, installed into .venv/ +make daemon-release # Daemon binary at ./target/release/sandd ``` +Activate the virtualenv (`source .venv/bin/activate`) before running the Python +snippets below. + ## 2. Terminal 1 - Start Agent ```python @@ -25,9 +25,9 @@ while True: count = server.daemon_count() if count > 0: print(f"\n{count} daemon(s) connected:") - for daemon_id in server.list_daemons(): - result = server.exec(daemon_id, "hostname") - print(f" {daemon_id}: {result.stdout.strip()}") + for daemon in server.list_daemons(): + result = server.exec(daemon.id, "hostname") + print(f" {daemon.id}: {result.stdout.strip()}") time.sleep(5) ``` @@ -117,12 +117,10 @@ session.close() ### File Transfer -```python -# Upload file -with open("config.yaml", "rb") as f: - data = f.read() -server.upload_file("worker-1", "/etc/app/config.yaml", data) +> **Note:** `upload_file` is not implemented on the daemon yet. It returns without an +> error but writes nothing. Only download works today. +```python # Download file data = server.download_file("worker-1", "/var/log/app.log") with open("app.log", "wb") as f: @@ -132,9 +130,9 @@ with open("app.log", "wb") as f: ### Managing Daemons ```python -# List connected daemons -daemons = server.list_daemons() -print(f"Connected: {daemons}") +# List connected daemons, optionally filtered by labels +for daemon in server.list_daemons(labels={"env": "prod"}): + print(daemon.id, daemon.version, daemon.labels, daemon.is_busy) # Get statistics stats = server.get_stats() diff --git a/docs/proposals/PROTOCOL.md b/docs/proposals/PROTOCOL.md index 10ecfa7..cb6e62e 100644 --- a/docs/proposals/PROTOCOL.md +++ b/docs/proposals/PROTOCOL.md @@ -443,6 +443,6 @@ All messages are JSON with a `type` field indicating the message type: ## Implementation Details -See `server/src/protocol.rs` for the complete Rust implementation using serde for JSON serialization. +See `protocol/src/lib.rs` for the complete Rust implementation using serde for JSON serialization. Binary data (session I/O, file chunks) is base64-encoded for JSON compatibility. From c3f24e1e641b05b73eb762af6869994bfbc4b020 Mon Sep 17 00:00:00 2001 From: kerthcet Date: Sat, 17 Oct 2026 12:46:53 +0200 Subject: [PATCH 2/4] docs: mark file transfer as not supported yet Co-Authored-By: Claude Opus 5.5 Signed-off-by: kerthcet --- README.md | 3 +-- docs/DEVELOP.md | 6 +++--- docs/QUICKSTART.md | 12 ------------ docs/proposals/PROTOCOL.md | 2 ++ 4 files changed, 6 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index cef6e54..889f67d 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,6 @@ Rust-powered WebSocket server with Python API for remote command execution and i - **Command Execution** - Run shell commands on remote machines with timeout control - **Interactive Sessions** - Full PTY sessions with bash for manual work -- **File Download** - Fetch files from workers (upload is not implemented yet) - **Snapshots** - Save, tag and restore workspace state on a worker - **Labels & Broadcast** - Filter workers by label and run a command on many at once - **Tunnel Mode (VPN)** - Secure mesh networking with WireGuard encryption via Tailscale @@ -185,7 +184,7 @@ See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. ## Roadmap - [ ] **Authentication & Authorization** - Keycloak-issued tokens for daemons (today any client that speaks `sandd.v1` is admitted; see [Architecture](./docs/ARCHITECTURE.md#future-authentication)) -- [ ] **File Upload** - Write uploaded files on the daemon +- [ ] **File Transfer** - Upload and download files between controller and workers - [ ] **Audit Logging** - Track all commands, sessions, and file transfers - [ ] **Metrics** - Prometheus-compatible metrics for monitoring - [ ] **Resource Limits** - CPU/memory/timeout controls per daemon diff --git a/docs/DEVELOP.md b/docs/DEVELOP.md index d409949..85d2dbd 100644 --- a/docs/DEVELOP.md +++ b/docs/DEVELOP.md @@ -188,9 +188,9 @@ RUST_LOG=debug ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws ## Known Issues & Limitations -1. **File upload is not implemented.** `Server.upload_file` sends the data, but the - daemon only logs the chunks and writes nothing. The call still returns without an - error. `download_file` works. +1. **File transfer is not supported yet.** `Server.upload_file` and + `Server.download_file` exist, and the protocol messages are defined, but the + feature is unfinished: don't rely on either call. 2. **No daemon authentication.** Any client that speaks `sandd.v1` can register as any daemon id. Run the controller on a trusted network or in tunnel mode. See [ARCHITECTURE.md](ARCHITECTURE.md#future-authentication). diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index 4180ac4..8173380 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -115,18 +115,6 @@ session.resize(rows=50, cols=120) session.close() ``` -### File Transfer - -> **Note:** `upload_file` is not implemented on the daemon yet. It returns without an -> error but writes nothing. Only download works today. - -```python -# Download file -data = server.download_file("worker-1", "/var/log/app.log") -with open("app.log", "wb") as f: - f.write(data) -``` - ### Managing Daemons ```python diff --git a/docs/proposals/PROTOCOL.md b/docs/proposals/PROTOCOL.md index cb6e62e..403b881 100644 --- a/docs/proposals/PROTOCOL.md +++ b/docs/proposals/PROTOCOL.md @@ -270,6 +270,8 @@ All messages are JSON with a `type` field indicating the message type: ### File Transfer +> **Not supported yet.** These messages are defined, but file transfer is unfinished. + #### FileUploadStart **Direction**: Agent → Daemon **Purpose**: Begin uploading a file to daemon From 09357fb4f7ea1e5edfee5cfeb1129a4876f6bbd9 Mon Sep 17 00:00:00 2001 From: kerthcet Date: Sat, 17 Oct 2026 12:51:49 +0200 Subject: [PATCH 3/4] docs: add a Security section to the README State plainly that daemons are not authenticated yet, what direct and tunnel mode do and do not protect, and that the tunnel authkey is a Headscale key rather than a SandD credential. Co-Authored-By: Claude Opus 5.5 Signed-off-by: kerthcet --- README.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 889f67d..d093883 100644 --- a/README.md +++ b/README.md @@ -172,6 +172,20 @@ server = Server(connect="tunnel", tunnel_config=tunnel_config_from_env()) See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. +## Security + +SandD does not authenticate daemons yet. The controller accepts any client that +speaks the `sandd.v1` protocol, and a daemon can register under any id, so anyone who +can reach the controller's port can join and receive commands meant for that id. + +- **Direct mode:** run the controller only on a trusted network. +- **Tunnel mode:** only nodes admitted to your Headscale mesh can reach the + controller, and traffic is encrypted with WireGuard. The `authkey` is a Headscale + key for joining the mesh, not a SandD credential. + +Daemon authentication and authorization with Keycloak is planned (see +[Architecture](./docs/ARCHITECTURE.md#future-authentication)). + ## Documentation - [Quick Start Guide](./docs/QUICKSTART.md) @@ -183,7 +197,7 @@ See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. ## Roadmap -- [ ] **Authentication & Authorization** - Keycloak-issued tokens for daemons (today any client that speaks `sandd.v1` is admitted; see [Architecture](./docs/ARCHITECTURE.md#future-authentication)) +- [ ] **Authentication & Authorization** - Keycloak-issued tokens for daemons (see [Security](#security)) - [ ] **File Transfer** - Upload and download files between controller and workers - [ ] **Audit Logging** - Track all commands, sessions, and file transfers - [ ] **Metrics** - Prometheus-compatible metrics for monitoring From d4351ee9d5789c33cdb7d4907bf9bda6ba0d8bfb Mon Sep 17 00:00:00 2001 From: kerthcet Date: Sat, 17 Oct 2026 12:54:32 +0200 Subject: [PATCH 4/4] docs: drop the CPython linker troubleshooting entry Co-Authored-By: Claude Opus 5.5 Signed-off-by: kerthcet --- docs/DEVELOP.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/docs/DEVELOP.md b/docs/DEVELOP.md index 85d2dbd..92bd26e 100644 --- a/docs/DEVELOP.md +++ b/docs/DEVELOP.md @@ -286,14 +286,6 @@ Use a `feat:`, `fix:` or `chore:` prefix, as in the existing history. **Fix:** Use `-m server/Cargo.toml` flag -### Linker errors about undefined `_Py*` symbols - -**Symptom:** `cargo build`/`cargo test` of `server/` fails to link with missing CPython -symbols - -**Fix:** Don't enable the `python` feature outside maturin. It turns on -`pyo3/extension-module`, which expects an interpreter to supply those symbols. - ## Protocol WebSocket-based JSON protocol for agent-daemon communication.