From e389560166d095b464dfad8339a2f5f23fb79808 Mon Sep 17 00:00:00 2001 From: Wynne Pirini Date: Wed, 28 Oct 2026 17:31:40 +1300 Subject: [PATCH] docs: record the persisted key, and what a restore now needs #173 made the platform secret persist beside the database when it is not supplied in the environment, which fixes a real silent data-loss bug. Two documentation gaps were left: 1. No CHANGELOG entry. This is a user-facing fix plus new behaviour, and the CHANGELOG is what generates release notes. Recorded under Unreleased, describing what actually happens rather than the ideal: a restore missing the key generates one and then refuses to start, so the data is never re-keyed, and the generated file has to be replaced. 2. DEPLOY.md still says recovery is automatic when the volume comes up empty. That is no longer the whole story for deployments without CALNODE_ENCRYPTION_KEY: Litestream replicates the database, not calnode.key, so restoring the replica alone will not start. Added the caveat, the recovery step for a failed restore, and the note that env-key deployments are unaffected. --- CHANGELOG.md | 33 +++++++++++++++++++++++++++++++++ DEPLOY.md | 16 ++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ee2b4fa3..31c54f3e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,23 @@ exact tag (`ghcr.io/calnode/calnode:0.1.0`) if you need stability between upgrad ## [Unreleased] ### Fixed +- **A plain `docker run` no longer loses stored credentials on restart.** The documented + first install starts Calnode with no `CALNODE_ENCRYPTION_KEY`. With `BASE_URL` on + `http://`, the vault treated that as development mode and generated an ephemeral data key, + so every credential stored in the database — the SMTP password, Stripe keys, Zoom and + LiveKit secrets, OAuth client secrets — became unreadable at the next restart, while the + volume made the data look persisted. The only signal was the `BASE_URL` scheme, so a + production deployment behind a TLS-terminating proxy (`BASE_URL http`, TLS at the edge) + took the same silent path. A secret is now resolved before the vault opens: an explicit + `CALNODE_ENCRYPTION_KEY` still wins and writes nothing, an existing key file beside the + database is reused, and only a genuinely fresh install generates one, written `0600` with + `O_EXCL` so two starts cannot overwrite each other. An empty key file is now an error + rather than a cue to generate a replacement. +- **A restore that is missing the encryption key now fails instead of starting fresh over an + encrypted database.** If the database already holds a wrapped data key but no key file is + present, Calnode generates one and then refuses to start, because it cannot unwrap the + existing data key — it never re-keys the data. The generated file is useless: replace it + with the real key, or set `CALNODE_ENCRYPTION_KEY`, before retrying the restore. - **An NS8 module release can no longer mix architectures.** The module build pushes each architecture to a fixed staging tag (`:build-amd64`, `:build-arm64`) and then assembles the multi-arch index from them. The workflow's concurrency group included the git ref, so a @@ -24,6 +41,22 @@ exact tag (`ghcr.io/calnode/calnode:0.1.0`) if you need stability between upgrad cancelled by an unrelated push. The post-publish architecture assertion is what caught it and remains in place. +### Changed +- **The encryption key is persisted beside the database, and must be backed up with it.** + When Calnode generates the platform secret itself it writes `calnode.key` next to the + database (`/data/calnode.key` in the container). Litestream replicates the database, not + that file, so the two must be kept together: restoring the database alone will not start. + Deployments that set `CALNODE_ENCRYPTION_KEY` in the environment are unchanged and write + no key file. + +### Added +- **The published arm64 image is now exercised on native hardware on every PR and every + publish.** A new `arm64-smoke` job runs on an ARM runner, asserts the image and the + container really are `aarch64`, checks the app serves over HTTP, and — the part that + matters for this release — verifies the key file is persisted and that a restart unwraps + the existing data key rather than generating a new one. On a publish it pulls the exact + digest the build job produced, so it cannot pass against a stale tag. + ## [0.11.2] - 2026-10-28 ### Added diff --git a/DEPLOY.md b/DEPLOY.md index 7e298686..6ea0f5b6 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -308,6 +308,22 @@ latest replica from object storage before starting the app. To **prove the round-trip without touching the live DB**, restore to a scratch path (never `/data/calnode.db`) and confirm it's a valid SQLite file. +> **The replica holds the database, not the encryption key.** If you deployed +> without setting `CALNODE_ENCRYPTION_KEY`, the app generated a key at first start +> and wrote it to `calnode.key` next to the database (`/data/calnode.key` in the +> container). Litestream does not replicate that file, so restoring the database +> from the replica on its own will not start: the restored rows are sealed under a +> key that is gone, and the app exits rather than re-keying your data. Back that +> file up separately, and restore it beside the database. +> +> If a restore does fail this way, delete the `calnode.key` that the failed start +> generated before restoring the real one — the process writes a new key on first +> start, and that file will not unlock the restored database. +> +> Deployments that set `CALNODE_ENCRYPTION_KEY` in the environment (the recommended +> production path, and what the NS8 module does) are unaffected: the key comes from +> the environment, survives a volume loss with it, and no key file is written. + **Non-destructive drill, in the container (zero setup — litestream + creds + config are already there):** ```bash railway ssh --service calnode sh -lc '