Skip to content

docs: hotfix protocol - #395

Open
sabinem wants to merge 4 commits into
mainfrom
docs/hotfix-protocol
Open

sabinem wants to merge 4 commits into
mainfrom
docs/hotfix-protocol

Conversation

@sabinem

@sabinem sabinem commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

A protocol for hotfixes, and tooling that enforces it

Whoever has to fix production in a hurry during a hackathon can now follow one page, and three commands stop them before the steps that are easy to get wrong.

Before the first hotfix on a release branch (v0.12.2) worked, but only with a lot of back and forth. The fix's pull request left out the chart bump. Bumping the version first was tempting, because on main that's the right order, but on a release branch it would have published images with no chart pointing at them. A local release branch had silently diverged from GitHub. A one-off CI failure looked like "the images weren't built". And it wasn't obvious that prod needs a chart install and dev needs a restart before anyone sees the fix.

Now: HOTFIX.md walks through it from start to cleanup, with the commands to copy and a table of what can go wrong. just hotfix::start creates the branch with the chart bump already committed. just version::bump refuses to release a release branch that isn't ready, and says why. just hotfix::to-main brings the fix back to main without the release's version numbers.

  • hotfix::check checks:
  • the working tree is clean,
  • the branch is identical to GitHub's,
  • the tag is free,
  • the chart points at the next patch,
  • the chart version is newer than the last release's and not main's,
  • there are release notes.

The step-by-step hotfix section moves from RELEASING.md to HOTFIX.md, so the steps are kept in one place.

Left out: a status command for "are the images and chart published, and is prod on it". It would need registry logins and cluster access. Deploying through cloud-infra also stays a manual step.

Follow-up: the tooling also needs to go into release/0.12, because just reads recipes from the branch you're on. That's a separate PR into release/0.12 with only the tooling commit, and it needs no release.

Verified: every recipe was run against a throwaway clone and remote in good and bad states: not ready, diverged, ready, a second fix in the same release, a conflicting cherry-pick, and a fix already on main. The formatter check passes. Nothing was run against GitHub.

hotfix::start branches from the newest release line and commits the
chart bump first, hotfix::check fails unless a release branch is ready
for a patch release, hotfix::to-main brings a fix back without main
inheriting the release line's numbers. version::bump runs the check
itself on release/* and refuses minor and major there.
Wipe Postgres and Keycloak, sync deps, start, seed, and restart so the
backend loads the seeded roles. Needed after switching between main and
a release branch, whose schemas differ: migrations only ever add.
schema-change now regenerates code and then runs fresh, which gives it
the dependency sync it lacked.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant