diff --git a/README.md b/README.md index 2b86efd..d093883 100644 --- a/README.md +++ b/README.md @@ -18,11 +18,12 @@ Rust-powered WebSocket server with Python API for remote command execution and i - **Command Execution** - Run shell commands on remote machines with timeout control - **Interactive Sessions** - Full PTY sessions with bash for manual work -- **File Transfer** - Upload/download files between controller and workers +- **Snapshots** - Save, tag and restore workspace state on a worker +- **Labels & Broadcast** - Filter workers by label and run a command on many at once - **Tunnel Mode (VPN)** - Secure mesh networking with WireGuard encryption via Tailscale - **High Performance** - Rust async runtime handles high-concurrency workloads - **Auto Reconnection** - Workers reconnect automatically on network failures -- **Cross-Platform** - Linux, macOS, Windows support +- **Cross-Platform** - Linux and macOS release binaries; Windows builds from source ## Architecture @@ -171,6 +172,20 @@ server = Server(connect="tunnel", tunnel_config=tunnel_config_from_env()) See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. +## Security + +SandD does not authenticate daemons yet. The controller accepts any client that +speaks the `sandd.v1` protocol, and a daemon can register under any id, so anyone who +can reach the controller's port can join and receive commands meant for that id. + +- **Direct mode:** run the controller only on a trusted network. +- **Tunnel mode:** only nodes admitted to your Headscale mesh can reach the + controller, and traffic is encrypted with WireGuard. The `authkey` is a Headscale + key for joining the mesh, not a SandD credential. + +Daemon authentication and authorization with Keycloak is planned (see +[Architecture](./docs/ARCHITECTURE.md#future-authentication)). + ## Documentation - [Quick Start Guide](./docs/QUICKSTART.md) @@ -182,6 +197,8 @@ See [Tunnel Mode Guide](./docs/proposals/TUNNEL.md) for setup instructions. ## Roadmap +- [ ] **Authentication & Authorization** - Keycloak-issued tokens for daemons (see [Security](#security)) +- [ ] **File Transfer** - Upload and download files between controller and workers - [ ] **Audit Logging** - Track all commands, sessions, and file transfers - [ ] **Metrics** - Prometheus-compatible metrics for monitoring - [ ] **Resource Limits** - CPU/memory/timeout controls per daemon diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index cb91dec..4133d31 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -16,7 +16,7 @@ - ✅ No ports exposed on execution plane (security) - ✅ Works through NAT/firewalls - ✅ Easy to add/remove daemons dynamically -- ✅ Agent controls access (daemon authenticates to agent) +- ✅ Agent decides what runs (daemons only execute what the agent sends) ### Why Rust Server? @@ -56,12 +56,15 @@ Forward raw SSH traffic through WebSocket to daemon's sshd. ### Future: Authentication -```rust -Message::Register { - daemon_id: String, - auth_token: String, // JWT or pre-shared key - metadata: DaemonMetadata, -} -``` +Today the controller admits any client that speaks `sandd.v1`, and a daemon can +register under any id. Run it on a trusted network, or in tunnel mode, where only mesh +members can reach it. + +The plan is to use Keycloak for authentication and authorization: -Agent validates token before accepting connection. +- The daemon gets an access token from Keycloak and sends it as + `Authorization: Bearer ` on the WebSocket upgrade. +- The controller verifies the token against Keycloak's JWKS endpoint before upgrading, + and rejects a bad one with HTTP 401. +- The token's claims decide which daemon id the connection registers as, and what it + is allowed to do. diff --git a/docs/DEVELOP.md b/docs/DEVELOP.md index f6e1d3c..92bd26e 100644 --- a/docs/DEVELOP.md +++ b/docs/DEVELOP.md @@ -6,98 +6,90 @@ This guide is for developers working on SandD itself. - **Rust**: Install from https://rustup.rs/ - **Python 3.8+**: With pip -- **Maturin**: `pip install maturin` +- **Docker**: Only for the e2e tests and images + +The Makefile creates `.venv/` and installs maturin, pytest and ruff into it as needed. ## Project Structure ``` SandD/ -├── server/ # Rust server with PyO3 bindings +├── protocol/ # Shared message types (sandd-protocol crate) +│ └── src/lib.rs +│ +├── server/ # Controller: Rust WebSocket server + PyO3 bindings │ ├── src/ -│ │ ├── lib.rs # Python API bindings -│ │ ├── server.rs # WebSocket server (axum) +│ │ ├── lib.rs # Crate root +│ │ ├── server.rs # WebSocket server (axum): /ws, /stats, /health │ │ ├── registry.rs # Daemon connection registry -│ │ └── protocol.rs # Message protocol +│ │ └── python.rs # Python API bindings (behind the `python` feature) │ └── Cargo.toml │ -├── sandd/ # Rust daemon binary +├── sandd/ # Daemon binary (runs on workers) │ ├── src/ -│ │ ├── main.rs # Daemon entry point +│ │ ├── main.rs # Entry point, connection loop, tunnel setup │ │ ├── executor.rs # Command execution │ │ ├── session.rs # Interactive sessions (PTY) -│ │ └── protocol.rs # Message protocol +│ │ └── snapshot/ # Workspace snapshots +│ ├── benches/ # Criterion benchmarks │ └── Cargo.toml │ -├── python/sandd/ # Python package wrapper -│ └── __init__.py +├── python/ +│ ├── sandd/ # Python package (Server, models, tunnel keys; AsyncServer is a stub) +│ └── tests/ # Unit, integration and e2e tests │ +├── hack/ # Dockerfiles, compose stacks, install script └── examples/ # Usage examples ``` ## Building -### Quick Build (Development) - ```bash -# Build everything -./test_build.sh +make dev # Build and install the Python package (debug, editable) +make daemon-build # Build the daemon (debug): ./target/debug/sandd -# Or step by step: -make dev # Build Python package (debug) -make daemon-build # Build daemon (debug) -``` - -### Release Build - -```bash make release # Python package (optimized) -make daemon-release # Daemon binary (optimized) +make daemon-release # Daemon binary (optimized): ./target/release/sandd ``` -### Manual Build +Run `make help` for the full list of targets. -```bash -# Python package -maturin develop --release -m server/Cargo.toml - -# Daemon -cargo build --package sandd --release -``` +The `python` feature of `server/` is off by default, so plain `cargo build` and +`cargo test` work without a Python interpreter. Maturin turns it on when building the +wheel (see `[tool.maturin]` in `pyproject.toml`). ## Development Workflow ### 1. Make Changes -Edit files in `server/src/`, `sandd/src/`, or `python/sandd/` +Edit files in `protocol/src/`, `server/src/`, `sandd/src/`, or `python/sandd/`. ### 2. Rebuild ```bash -# If you changed server/ +# If you changed protocol/, server/ or python/sandd/ make dev -# If you changed daemon/ +# If you changed protocol/ or sandd/ make daemon-build - -# If you changed Python wrapper only -# (no rebuild needed, it's just a wrapper) ``` -### 3. Test +### 3. Try It ```bash -# Terminal 1: Start test agent -python3 examples/simple_test.py - -# Terminal 2: Start daemon -./target/debug/sandd --server-url ws://127.0.0.1:8765/ws +# Terminal 1: start a controller +python3 -c " +from sandd import Server +server = Server() +server.wait_for_daemon('dev-1', timeout=60) +print(server.exec('dev-1', 'uname -a').stdout) +" + +# Terminal 2: start a daemon +./target/debug/sandd --server-url ws://127.0.0.1:8765/ws --daemon-id dev-1 ``` -### 4. Run Examples - -```bash -python3 examples/agent_example.py -``` +See [examples/](../examples) for sessions, snapshots and tunnel mode. ## Architecture @@ -130,39 +122,35 @@ file_transfers: Vec> // Chunked buffering ## Testing -### Unit Tests - ```bash -cargo test --workspace +make test # Lint + Rust tests + Python tests (no Docker) +make test-e2e # Direct-mode e2e against daemons in Docker (slow) +make test-e2e-tunnel # Tunnel-mode e2e with a Headscale mesh in Docker (slow) +make lint # ruff check +make format # ruff fix + format +make benchmark # Daemon benchmarks (criterion) ``` -### Integration Tests - -```bash -pytest tests/ # (when tests are added) -``` - -### Manual Testing - -Use `examples/agent_example.py` to test all features. +Python tests are selected by marker: `e2e` for Docker-based tests, `tunnel` for the +tunnel stack. `make test` runs `-m "not e2e"`. ## Common Tasks -### Adding a New Command Type +### Adding a New Message Type -1. Add to `protocol.rs` (both server and daemon): +1. Add the variant to `Message` in `protocol/src/lib.rs` (shared by server and daemon): ```rust - Message::MyNewCommand { field: String } + MyNewCommand { field: String } ``` -2. Handle in `server/src/server.rs`: +2. Handle it in `server/src/server.rs`: ```rust Message::MyNewCommand { field } => { // Forward to daemon or handle } ``` -3. Handle in `sandd/src/main.rs`: +3. Handle it in `sandd/src/main.rs`: ```rust Message::MyNewCommand { field } => { // Execute and respond @@ -171,7 +159,7 @@ Use `examples/agent_example.py` to test all features. ### Adding Python API -1. Add method to `Server` in `server/src/lib.rs`: +1. Add a method to `Server` in `server/src/python.rs`: ```rust #[pymethods] impl Server { @@ -181,7 +169,7 @@ Use `examples/agent_example.py` to test all features. } ``` -2. Add wrapper in `python/sandd/__init__.py`: +2. Add a wrapper in `python/sandd/server.py`: ```python def my_method(self, arg: str) -> None: """User-friendly docstring""" @@ -195,30 +183,21 @@ Use `examples/agent_example.py` to test all features. RUST_LOG=debug ./target/debug/sandd --server-url ws://127.0.0.1:8765/ws ``` -**Python side:** -```python -import logging -logging.basicConfig(level=logging.DEBUG) -``` - -**Check WebSocket traffic:** -```bash -# In server -RUST_LOG=server=debug python3 examples/simple_test.py -``` - -## Known Issues & Limitations (MVP) +**Controller logs:** `Server(verbose=True)` (the default) logs at INFO; set +`RUST_LOG=sandbox_server=debug` for more. -### Not Implemented +## Known Issues & Limitations -1. **File Transfer**: Protocol defined, daemon just logs - - Reason: Deferred for MVP - - Fix: Implement actual file I/O in daemon +1. **File transfer is not supported yet.** `Server.upload_file` and + `Server.download_file` exist, and the protocol messages are defined, but the + feature is unfinished: don't rely on either call. +2. **No daemon authentication.** Any client that speaks `sandd.v1` can register as any + daemon id. Run the controller on a trusted network or in tunnel mode. See + [ARCHITECTURE.md](ARCHITECTURE.md#future-authentication). ### Warnings - PyO3 `non_local_definitions` warning: Safe to ignore (PyO3 macro limitation) -- Unused imports/variables: Run `cargo fix` to clean up ## Performance @@ -232,21 +211,22 @@ At 200 concurrent daemons: ### Before Submitting PR -1. Run tests: `cargo test --workspace` -2. Check formatting: `cargo fmt --all` -3. Check lints: `cargo clippy --all` -4. Test manually with examples -5. Update docs if adding features +1. Run `make test` +2. Check formatting: `cargo fmt --all` and `make format` +3. Check lints: `cargo clippy --workspace --all-targets` +4. Update docs if adding features ### Commit Style ``` -Add feature: brief description +feat: brief description More detailed explanation if needed. Include motivation and context. ``` +Use a `feat:`, `fix:` or `chore:` prefix, as in the existing history. + ### Adding Dependencies - Keep dependencies minimal @@ -255,23 +235,23 @@ Include motivation and context. ## Release Process -1. Update version in `pyproject.toml` and `Cargo.toml` files -2. Update `CHANGELOG.md` (when added) -3. Build release: `make release && make daemon-release` -4. Test release build -5. Tag: `git tag v0.x.0` -6. Build wheel: `maturin build --release -m server/Cargo.toml` -7. Publish: `maturin publish` (when ready) +1. Update the version in `pyproject.toml`, `protocol/Cargo.toml`, `server/Cargo.toml` + and `sandd/Cargo.toml` (and the `sandd-protocol` dependency version in + `sandd/Cargo.toml`) +2. Tag and push: `git tag v0.x.y && git push upstream v0.x.y`. The `Release` workflow + builds the daemon binaries and creates the GitHub release. +3. Publish packages: `make publish` (wheels to PyPI, needs `INFTYAI_PYPI_TOKEN`; crates + to crates.io) ## Troubleshooting ### User Issues **Daemon won't connect:** -- Check agent URL is reachable: `curl -v ws://agent-host:8765/ws` +- Check the controller is up: `curl http://controller-host:8765/health` - Verify firewall allows outbound WebSocket connections -- Check agent server is running: `ps aux | grep python` - Check daemon logs: `RUST_LOG=info ./target/release/sandd ...` +- Check what the controller sees: `curl http://controller-host:8765/stats` **Commands timing out:** - Increase `timeout` parameter in `exec()` (in seconds) @@ -323,5 +303,4 @@ For complete protocol specification, see [proposals/PROTOCOL.md](proposals/PROTO - Check [ARCHITECTURE.md](ARCHITECTURE.md) for design details - Check [proposals/PROTOCOL.md](proposals/PROTOCOL.md) for protocol specification -- Check [STATUS.md](STATUS.md) for implementation status - Check [QUICKSTART.md](QUICKSTART.md) for usage examples diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index 7b97aa4..8173380 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -3,13 +3,13 @@ ## 1. Build Everything ```bash -# Install dependencies -pip3 install maturin - -# Build and install -./test_build.sh +make dev # Python package, installed into .venv/ +make daemon-release # Daemon binary at ./target/release/sandd ``` +Activate the virtualenv (`source .venv/bin/activate`) before running the Python +snippets below. + ## 2. Terminal 1 - Start Agent ```python @@ -25,9 +25,9 @@ while True: count = server.daemon_count() if count > 0: print(f"\n{count} daemon(s) connected:") - for daemon_id in server.list_daemons(): - result = server.exec(daemon_id, "hostname") - print(f" {daemon_id}: {result.stdout.strip()}") + for daemon in server.list_daemons(): + result = server.exec(daemon.id, "hostname") + print(f" {daemon.id}: {result.stdout.strip()}") time.sleep(5) ``` @@ -115,26 +115,12 @@ session.resize(rows=50, cols=120) session.close() ``` -### File Transfer - -```python -# Upload file -with open("config.yaml", "rb") as f: - data = f.read() -server.upload_file("worker-1", "/etc/app/config.yaml", data) - -# Download file -data = server.download_file("worker-1", "/var/log/app.log") -with open("app.log", "wb") as f: - f.write(data) -``` - ### Managing Daemons ```python -# List connected daemons -daemons = server.list_daemons() -print(f"Connected: {daemons}") +# List connected daemons, optionally filtered by labels +for daemon in server.list_daemons(labels={"env": "prod"}): + print(daemon.id, daemon.version, daemon.labels, daemon.is_busy) # Get statistics stats = server.get_stats() diff --git a/docs/proposals/PROTOCOL.md b/docs/proposals/PROTOCOL.md index 10ecfa7..403b881 100644 --- a/docs/proposals/PROTOCOL.md +++ b/docs/proposals/PROTOCOL.md @@ -270,6 +270,8 @@ All messages are JSON with a `type` field indicating the message type: ### File Transfer +> **Not supported yet.** These messages are defined, but file transfer is unfinished. + #### FileUploadStart **Direction**: Agent → Daemon **Purpose**: Begin uploading a file to daemon @@ -443,6 +445,6 @@ All messages are JSON with a `type` field indicating the message type: ## Implementation Details -See `server/src/protocol.rs` for the complete Rust implementation using serde for JSON serialization. +See `protocol/src/lib.rs` for the complete Rust implementation using serde for JSON serialization. Binary data (session I/O, file chunks) is base64-encoded for JSON compatibility.