Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ns8-module.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ jobs:
- name: Validate action scripts and schemas
working-directory: packaging/ns8
run: |
python3 -m py_compile imageroot/actions/configure-module/20configure imageroot/actions/get-configuration/20read
python3 -m py_compile imageroot/actions/configure-module/20configure imageroot/actions/get-configuration/20read imageroot/actions/clone-module/20unclaimhostname
python3 -m py_compile assert-multiarc.py
python3 -m py_compile test_assert_multiarc.py
python3 -m py_compile test_assert_multiarc.py test_clone_module.py
# Offline tests, so a broken invariant fails on every push rather than
# waiting for a release to notice. The registry round-trip tests in the
# same file are opt-in and skipped unless NS8_REGISTRY_TESTS=1.
Expand Down
28 changes: 28 additions & 0 deletions packaging/ns8/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,34 @@ operator of that destination, which may be a third-party storage provider, holds
an encrypted copy of your database and the key that unlocks it. Configure the
backup accordingly.

## Clone and move

Cloning an instance from the cluster admin UI copies the `calnode-data` volume
and the module state, so the copy arrives with all its bookings, users and
settings.

The clone deliberately arrives **unconfigured**. The hostname belongs to one
instance: it is stored in the module state, but the host-based route that answers
for it is registered per instance. A clone that kept the hostname would hold a
name it has no route for, and configuring it with that name would then be
refused, because the source instance still answers for it. So the clone gives the
hostname back and the Settings page asks for a new one. Give it a hostname,
click save, and it is live. That is the same state a freshly added instance is
in, and it is what makes the Settings page and the configure action agree.

Two things the clone keeps, deliberately:

- **The encryption key.** It travels with the copied database and is the only
thing that can unwrap the database key vault. Without it the clone would hold
an unreadable copy of every booking, so it is preserved across the clone even
though the hostname is not.
- **All the data.** The copy is complete, so the clone is a genuine second
instance with its own URL, not a way to share one database.

Moving an instance between nodes works the same way. If the instance is reached
by a public hostname, point your DNS at the new node; the cluster reverse proxy
is reconfigured for you.

## Uninstall

To uninstall the instance, keeping its data for a later reinstall:
Expand Down
130 changes: 130 additions & 0 deletions packaging/ns8/imageroot/actions/clone-module/20unclaimhostname
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
#!/usr/bin/env python3

#
# Copyright (C) 2023 Nethesis S.r.l.
# SPDX-License-Identifier: GPL-3.0-or-later
#

#
# Clone support.
#
# The core clone-module implementation copies the module volumes and the whole
# state/ directory, and that alone is enough to duplicate a working instance.
# Two things in state/ are specific to ONE hostname, though, and carrying them
# to a second instance silently breaks it:
#
# config.json the public hostname and its TLS flags
# calnode.env BASE_URL, which is derived from that hostname
#
# A cloned instance would arrive holding the source's hostname but with no route
# of its own, because the host-based Traefik route is registered by
# configure-module and belongs to the source instance (routes are keyed by
# MODULE_ID, so nothing stale is copied and nothing needs cleaning up here).
# Worse, configuring the clone with that hostname would then be REFUSED, since
# the source still answers for it. A clone would come up quietly broken and
# appear to be a configuration bug.
#
# So the clone gives up the hostname and becomes an unconfigured instance. The
# Settings page then asks for a hostname, configure-module registers a route for
# it, and the clone works. This is the same state a freshly added instance is in.
#
# What must NOT be cleared is CALNODE_ENCRYPTION_KEY. It travels with the copied
# database and is the only thing that can unwrap the database key vault; drop it
# and the clone holds an unreadable copy of every booking. Keep the key, drop the
# hostname.
#
# get-configuration already reports configured:false when config.json has no
# host, so no other change is needed for the UI to prompt.
#
# Steps run in sorted order: the core moves data in 10recvstate and starts the
# service in 90finalize, so this runs after the data lands and before startup.

import json
import os
import sys

CONFIG_FILE = "config.json"
ENV_FILE = "calnode.env"
# Derived from the hostname, so it goes whenever the hostname goes.
HOSTNAME_DERIVED_KEYS = ("BASE_URL",)


def read_env_file(path):
values = {}
try:
with open(path, "r", encoding="utf-8") as handle:
for line in handle.read().splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#") or "=" not in stripped:
continue
key, _, value = stripped.partition("=")
values[key.strip()] = value.strip()
except FileNotFoundError:
pass
return values


def drop_hostname_from_env(path, keep):
"""Remove hostname-derived keys from calnode.env, preserving everything else.

Rewrites the file in the same # comment / KEY=value shape configure-module
writes, so a later configure produces an identical file. Returns the keys
that were actually removed, or [] if there was nothing to do.
"""
values = read_env_file(path)
dropped = [key for key in HOSTNAME_DERIVED_KEYS if key in values]
if not dropped:
return []

# Refuse BEFORE touching the file. Checking afterwards would rewrite the env
# file and then fail, leaving the instance mutated by a step that aborted;
# the clone would then differ from its source in ways nothing recorded.
missing = [key for key in keep if key not in values]
if missing:
raise RuntimeError(
f"refusing to release the hostname: {path} has no "
f"{', '.join(missing)}, so the clone could not decrypt its own "
"database"
)

for key in dropped:
values.pop(key, None)

lines = [
"# Rewritten by clone-module: the hostname was released with the",
"# source instance and is set again by configure-module.",
]
lines += [f"{key}={values[key]}" for key in sorted(values)]
with open(path, "w", encoding="utf-8") as handle:
handle.write("\n".join(lines) + "\n")
os.chmod(path, 0o600)
return dropped


def release_hostname(config_path, env_path, keep=("CALNODE_ENCRYPTION_KEY",)):
"""Return the clone to an unconfigured state, keeping its encryption key."""
dropped = drop_hostname_from_env(env_path, keep)

removed_config = False
if os.path.exists(config_path):
os.remove(config_path)
removed_config = True

if removed_config or dropped:
print(
f"clone is unconfigured; set a hostname to finish (dropped "
f"{', '.join(dropped) or 'nothing'} from {env_path}, "
f"{'removed' if removed_config else 'kept'} {config_path})",
file=sys.stderr,
)
else:
print("clone had no hostname to release", file=sys.stderr)
return removed_config, dropped


if __name__ == "__main__":
state_dir = os.getcwd()
release_hostname(
os.path.join(state_dir, CONFIG_FILE),
os.path.join(state_dir, ENV_FILE),
)
149 changes: 149 additions & 0 deletions packaging/ns8/test_clone_module.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
"""Tests for the clone-module step.

Run offline, so a broken invariant fails on every push:

python3 -m unittest discover -s packaging/ns8 -p 'test_*.py' -v

These cover the one thing that must never go wrong: a cloned instance must give
up the hostname without losing the encryption key that makes its copied database
readable.
"""

import importlib.util
import json
import os
import pathlib
import tempfile
import unittest

_STEP = pathlib.Path(__file__).resolve().parent / (
"imageroot/actions/clone-module/20unclaimhostname"
)
_SPEC = importlib.util.spec_from_loader(
"unclaim", importlib.machinery.SourceFileLoader("unclaim", str(_STEP))
)
unclaim = importlib.util.module_from_spec(_SPEC)
_SPEC.loader.exec_module(unclaim)


def write_env(path, values, comments=True):
lines = ["# Written by configure-module."] if comments else []
lines += [f"{k}={v}" for k, v in values.items()]
path.write_text("\n".join(lines) + "\n", encoding="utf-8")
os.chmod(path, 0o600)


def write_config(path, host):
path.write_text(
json.dumps({"host": host, "lets_encrypt": True, "http2https": True}),
encoding="utf-8",
)


class ReleaseHostname(unittest.TestCase):
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.addCleanup(self.dir.cleanup)
self.config = pathlib.Path(self.dir.name) / "config.json"
self.env = pathlib.Path(self.dir.name) / "calnode.env"

def values(self):
return unclaim.read_env_file(str(self.env))

def test_encryption_key_survives_and_hostname_does_not(self):
write_env(
self.env,
{
"PORT": "3000",
"DATABASE_URL": "sqlite:///data/calnode.db",
"BASE_URL": "https://book.example.com",
"CALNODE_ENCRYPTION_KEY": "abc123",
},
)
write_config(self.config, "book.example.com")

unclaim.release_hostname(str(self.config), str(self.env))

values = self.values()
self.assertNotIn("BASE_URL", values)
self.assertEqual(values["CALNODE_ENCRYPTION_KEY"], "abc123")
self.assertEqual(values["PORT"], "3000")
self.assertEqual(values["DATABASE_URL"], "sqlite:///data/calnode.db")

def test_config_is_removed_so_the_ui_asks_for_a_hostname(self):
write_env(self.env, {"BASE_URL": "https://x.example.com", "CALNODE_ENCRYPTION_KEY": "k"})
write_config(self.config, "x.example.com")

removed, _ = unclaim.release_hostname(str(self.config), str(self.env))

self.assertTrue(removed)
self.assertFalse(self.config.exists())

def test_missing_config_is_not_an_error(self):
"""A clone of an unconfigured instance has nothing to release."""
write_env(self.env, {"PORT": "3000", "CALNODE_ENCRYPTION_KEY": "k"})
removed, dropped = unclaim.release_hostname(str(self.config), str(self.env))
self.assertFalse(removed)
self.assertEqual(dropped, [])
self.assertEqual(self.values()["CALNODE_ENCRYPTION_KEY"], "k")

def test_missing_env_file_is_not_an_error(self):
write_config(self.config, "x.example.com")
removed, dropped = unclaim.release_hostname(str(self.config), str(self.env))
self.assertTrue(removed)
self.assertEqual(dropped, [])
self.assertFalse(self.config.exists())

def test_refuses_to_write_an_env_file_without_the_key(self):
"""Better to fail the clone than to hand back an unreadable database."""
write_env(self.env, {"BASE_URL": "https://x.example.com"})
with self.assertRaises(RuntimeError) as caught:
unclaim.release_hostname(str(self.config), str(self.env))
self.assertIn("CALNODE_ENCRYPTION_KEY", str(caught.exception))

def test_refusal_leaves_the_files_untouched(self):
"""The guard must fire before anything is rewritten.

Checking after the write would abort a step that had already mutated the
clone, so it would differ from its source with nothing recording it.
"""
write_env(self.env, {"BASE_URL": "https://x.example.com", "PORT": "3000"})
write_config(self.config, "x.example.com")
before_env = self.env.read_text()
before_config = self.config.read_text()

with self.assertRaises(RuntimeError):
unclaim.release_hostname(str(self.config), str(self.env))

self.assertEqual(self.env.read_text(), before_env)
self.assertEqual(self.config.read_text(), before_config)

def test_env_file_stays_owner_only(self):
write_env(self.env, {"BASE_URL": "https://x.example.com", "CALNODE_ENCRYPTION_KEY": "k"})
unclaim.release_hostname(str(self.config), str(self.env))
self.assertEqual(os.stat(self.env).st_mode & 0o777, 0o600)

def test_comments_and_blank_lines_are_tolerated(self):
self.env.write_text(
"# Written by configure-module.\n\n"
"PORT=3000\n\n# a note\nBASE_URL=https://x.example.com\n"
"CALNODE_ENCRYPTION_KEY=k\n",
encoding="utf-8",
)
unclaim.release_hostname(str(self.config), str(self.env))
values = self.values()
self.assertNotIn("BASE_URL", values)
self.assertEqual(values["CALNODE_ENCRYPTION_KEY"], "k")
self.assertEqual(values["PORT"], "3000")

def test_is_idempotent(self):
"""clone-module steps can be retried; running twice must not lose more."""
write_env(self.env, {"BASE_URL": "https://x.example.com", "CALNODE_ENCRYPTION_KEY": "k"})
write_config(self.config, "x.example.com")
unclaim.release_hostname(str(self.config), str(self.env))
unclaim.release_hostname(str(self.config), str(self.env))
self.assertEqual(self.values()["CALNODE_ENCRYPTION_KEY"], "k")


if __name__ == "__main__":
unittest.main()
Loading