Add pnpm verify, a Definition of Done, and CI

.github/workflows/verify.yml

@@ -0,0 +1,36 @@
+# Runs `pnpm verify`, the same checks as on your Mac, on every push to main and every pull request.
+# macOS because the app is built and tested there, and Tauri on Linux needs extra WebKit packages.
+name: verify
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+# A newer push to the same branch cancels the older run. A cancelled run is not a failure.
+concurrency:
+  group: verify-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  verify:
+    runs-on: macos-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
+        with:
+          version: 10.28.1
+      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
+        with:
+          node-version: 22
+          cache: pnpm
+      - run: rustup component add clippy rustfmt
+      - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
+        with:
+          workspaces: src-tauri
+      - run: pnpm install --frozen-lockfile
+      - run: pnpm verify

.gitignore

@@ -11,6 +11,7 @@ node_modules
 dist
 dist-ssr
 *.local
+.claude/settings.local.json
 
 # Editor directories and files
 .vscode/*

CLAUDE.md

@@ -12,13 +12,26 @@ What comes next, and the decisions behind the editor: the "Next" section of
 pnpm install
 pnpm dev                          # the editor in a plain browser: no files, fake AI results
 pnpm tauri dev                    # the desktop app, with live reload
-pnpm test                         # editor, format, Lab, and palette tests (vitest, no DOM)
-pnpm exec tsc --noEmit -p .       # type check
-cd src-tauri && cargo test --lib  # Rust tests
+pnpm verify                       # every check CI runs, in about 10 seconds
+pnpm test                         # only the vitest tests
 pnpm tauri build --bundles app    # src-tauri/target/release/bundle/macos/Undertext.app
 ```
 
-`cargo` must be on `PATH`. With Homebrew's rustup it lives in `/opt/homebrew/opt/rustup/bin`.
+`pnpm tauri` needs `cargo` on `PATH`. With Homebrew's rustup it lives in
+`/opt/homebrew/opt/rustup/bin`. `pnpm verify` finds it there on its own.
+
+## Done
+
+- Done means proven: a test that failed before the change and passes after, or the change seen
+  in the app. Reading the code or a passing compile is not proof.
+- Run `pnpm verify` after each change, and before you call anything done. It records the tree
+  when every check passes on it, or when only docs changed since its last pass.
+- It can't see the app window (WebKit), real AI calls, file dialogs, or the keychain. When a
+  change depends on them, check it in `pnpm tauri dev`, or say it is not verified.
+- Never weaken a check to pass it: no skipped or deleted tests, no loosened assertions. If a
+  check is wrong, stop and say why.
+- Report what ran, what it showed, and what you did not verify. How this works:
+  `docs/agents.md`.
 
 ## How the editor works
 

README.md

@@ -166,11 +166,14 @@ limits. If `claude` isn't in `~/.local/bin`, `/opt/homebrew/bin`, or `/usr/local
 ## Development
 
 ```bash
-pnpm test           # editor and format tests
+pnpm verify         # every check CI runs: types, tests, web build, Rust format, lint, and tests
+pnpm test           # only the vitest tests
 pnpm icons          # rebuild the app icons from brand/ (needs ImageMagick)
-cd src-tauri && cargo test
 ```
 
+`pnpm verify` is also what coding agents run to prove their work. See
+[docs/agents.md](docs/agents.md).
+
 The editor is [CodeMirror](https://codemirror.net) inside a [Tauri](https://tauri.app) app. The page
 text is plain Markdown; alternatives, ghosts, and versions are ranges kept beside it (`src/editor/layers.ts`).
 The icon borrows the mark of [Klipstack](brand/README.md), with a handwritten u in place of its clip.

docs/agents.md

@@ -0,0 +1,90 @@
+# Working with coding agents
+
+Undertext is built with coding agents, mostly Claude Code. This page says what the repository
+checks on its own, what it only asks agents to do, and what is still up to you.
+
+The idea: an agent proves its work before it calls it done, without you asking. A passing check
+is proof; "I read the code" is not.
+
+## The layers so far
+
+1. **`pnpm verify`** runs every check, in about 10 seconds, and records the tree when they all
+   pass.
+2. **CI** runs `pnpm verify` on GitHub for every push to `main` and every pull request.
+3. **`CLAUDE.md`** tells every agent session what done means and to run `pnpm verify`.
+
+## Rules, and what holds them
+
+| Rule | What holds it | How sure |
+| --- | --- | --- |
+| Types, tests, web build, and Rust format, lint, and tests pass | `pnpm verify`, then CI | Checked. CI shows a red run, but nothing blocks a push to `main`. |
+| Run `pnpm verify` before calling work done | `CLAUDE.md` | Asked |
+| Never weaken a check to pass it | `CLAUDE.md` | Asked |
+| Say what was not verified | `CLAUDE.md` | Asked |
+
+"Asked" means an agent can forget it. Later steps turn some of these into checks.
+
+## `pnpm verify`
+
+It runs, in order, and prints one line per check:
+
+| Check | Command |
+| --- | --- |
+| types | `tsc --noEmit -p .` |
+| config types | `tsc` on `vite.config.ts` |
+| tests | `vitest run`, with no `.only` allowed (fails if no test ran) |
+| web build | `vite build`, into a scratch folder, so `dist/` stays as it was |
+| rust format | `cargo fmt --check` |
+| rust lint | `cargo clippy --all-targets --locked -- -D warnings` |
+| rust tests | `cargo test --lib --locked` (fails if no test ran) |
+
+- It never picks a subset of the checks. They are fast, so there is nothing to gain. It runs all
+  of them, or none in the three cases below.
+- On a failure it prints the end of that check's log and the full log's path. Logs live in
+  `.git/undertext/logs/`, or under `.git/worktrees/<name>/` in a linked worktree.
+- **It records a pass** only when every check passed and no file changed during the run. The
+  record is a hash of every file's contents, untracked files included, in
+  `.git/undertext/verified.json`. Each worktree keeps its own. A failed run deletes the record.
+- **Nothing changed** since the last pass: it says so and stops. `pnpm verify --force` runs the
+  checks anyway, for example after `rustup update`.
+- **Only docs changed** since the last pass (`docs/`, `LICENSE`, and `*.md` outside `src/` and
+  `src-tauri/`): nothing to check. It records the new tree.
+- **Only CI files changed** (`.github/`): local checks can't see them. It records nothing, even
+  with `--force`. Push and read the CI run.
+- It finds `cargo` on `PATH`, in `~/.cargo/bin`, or in Homebrew's rustup folder. Without cargo it
+  fails. It never skips a check it can't run.
+
+**What it can't see:** the app window, which is WebKit, while `pnpm dev` runs in Chromium; real AI
+calls; file dialogs; and the keychain. For those, run `pnpm tauri dev` and try the change.
+
+**Known limits:**
+
+- It compares the tree before and after the run. An edit made and undone while it runs goes
+  unseen.
+- It checks the files on disk, not what is staged. When you stage only part of a change, the
+  commit holds a tree that was never checked. CI checks the commit after you push.
+- A stray `.only` fails it, as in CI. A `.skip` does not.
+
+The script is `scripts/verify.mjs`, the checks are in `scripts/verify-steps.mjs`, and its tests are
+in `scripts/verify.test.mjs`. The tests run copies of the scripts in throwaway repositories.
+
+## CI
+
+`.github/workflows/verify.yml` runs `pnpm verify` on a GitHub macOS machine. It is free for public
+repositories.
+
+- **Why macOS:** the app is built and tested there. On Linux, Tauri needs extra WebKit packages.
+- **It is a signal, not a gate.** `main` has no branch protection, so a red run blocks nothing.
+  You push to `main` yourself, and that stays as it is.
+- **Cancelled runs are not failures.** A newer push to the same branch cancels the older run. A
+  run that never got a machine or timed out needs a re-run, not a code change.
+- **Rust is not pinned.** CI uses the latest stable Rust. A new Clippy lint can fail CI before it
+  fails on your Mac. Run `rustup update`, then `pnpm verify --force`.
+- Each action is pinned to one commit, with its version in a comment, so a changed tag can't
+  change what runs.
+
+## Where the ideas come from
+
+Kent Beck, Simon Willison, Mitchell Hashimoto, Anthropic's and Amp's guidance on coding agents,
+OpenAI's harness engineering, Lauren Tan's pstack, Matt Pocock's skills, and Jesse Vincent's
+superpowers.

package.json

@@ -16,6 +16,7 @@
     "preview": "vite preview",
     "tauri": "tauri",
     "test": "vitest run",
+    "verify": "node scripts/verify.mjs",
     "icons": "node scripts/icons.mjs"
   },
   "dependencies": {

scripts/tree.mjs

@@ -0,0 +1,87 @@
+// The checkout's files by content hash, and the record of the last tree that passed `pnpm verify`.
+import { execFileSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import { lstatSync, mkdirSync, readFileSync, readlinkSync, rmSync, writeFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+
+/** The environment without git's own variables, so git acts on `root` even when run from a git hook. */
+let cleanEnv;
+function gitEnv() {
+  if (!cleanEnv) {
+    cleanEnv = { ...process.env };
+    for (const name of execFileSync("git", ["rev-parse", "--local-env-vars"], { encoding: "utf8" }).split("\n")) {
+      if (name) delete cleanEnv[name];
+    }
+  }
+  return cleanEnv;
+}
+
+export const git = (root, ...args) =>
+  execFileSync("git", ["-C", root, ...args], { env: gitEnv(), encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] });
+
+/**
+ * Every file git sees in `root`: tracked, staged and untracked, but not ignored. Hashes contents, so
+ * an edit in the same second counts, and writes nothing to git's object store.
+ */
+export function snapshot(root) {
+  const paths = new Set(git(root, "ls-files", "-z", "--cached", "--others", "--exclude-standard").split("\0"));
+  const files = {};
+  for (const path of [...paths].filter(Boolean).sort()) {
+    const hash = hashFile(join(root, path));
+    if (hash) files[path] = hash;
+  }
+  const fingerprint = createHash("sha256").update(JSON.stringify(files)).digest("hex");
+  return { fingerprint, files };
+}
+
+function hashFile(file) {
+  let stat;
+  try {
+    stat = lstatSync(file);
+  } catch {
+    return null; // deleted
+  }
+  if (stat.isSymbolicLink()) return `link:${readlinkSync(file)}`;
+  if (!stat.isFile()) return null;
+  const mode = stat.mode & 0o111 ? "x" : "-";
+  return createHash("sha256").update(mode).update(readFileSync(file)).digest("hex");
+}
+
+/** Paths that differ between two snapshots' files. */
+export function changedPaths(before, after) {
+  const paths = new Set([...Object.keys(before), ...Object.keys(after)]);
+  return [...paths].filter((path) => before[path] !== after[path]).sort();
+}
+
+/**
+ * Docs: no check reads them, so a change to them alone needs no verify. Markdown under src/ or
+ * src-tauri/ may be a fixture the code reads, so it counts as code.
+ */
+export const isDoc = (path) =>
+  !/^src(-tauri)?\//.test(path) && (path.endsWith(".md") || path.startsWith("docs/") || path === "LICENSE");
+
+/** Only CI reads these. Local checks can't vouch for them. */
+export const isCiOnly = (path) => path.startsWith(".github/");
+
+/** Per checkout: a linked worktree keeps its own record. */
+export const stateDir = (root) => resolve(root, git(root, "rev-parse", "--git-path", "undertext").trim());
+
+const passFile = (root) => join(stateDir(root), "verified.json");
+
+/** The last pass, or null when there is none or the record is unreadable. */
+export function readPass(root) {
+  try {
+    const pass = JSON.parse(readFileSync(passFile(root), "utf8"));
+    return typeof pass?.fingerprint === "string" && pass.files && typeof pass.files === "object" ? pass : null;
+  } catch {
+    return null;
+  }
+}
+
+export function recordPass(root, snap) {
+  const file = passFile(root);
+  mkdirSync(dirname(file), { recursive: true });
+  writeFileSync(file, JSON.stringify({ ...snap, at: new Date().toISOString() }));
+}
+
+export const forgetPass = (root) => rmSync(passFile(root), { force: true });

scripts/verify-steps.mjs

@@ -0,0 +1,43 @@
+// The checks `pnpm verify` runs, in order. CI runs the same list through `pnpm verify`.
+import { existsSync } from "node:fs";
+import { homedir } from "node:os";
+import { delimiter, join } from "node:path";
+import { stateDir } from "./tree.mjs";
+
+/** The folder with cargo: PATH, rustup's default, or Homebrew's rustup, which fish leaves off PATH. */
+function cargoDir() {
+  const dirs = [...(process.env.PATH ?? "").split(delimiter), join(homedir(), ".cargo/bin"), "/opt/homebrew/opt/rustup/bin"];
+  return dirs.find((dir) => dir && existsSync(join(dir, "cargo"))) ?? null;
+}
+
+/** How many tests passed, from the runner's summary. Zero means the step checked nothing. */
+export const vitestPassed = (log) => Number(/Tests\s+.*?(\d+) passed/.exec(log)?.[1] ?? 0);
+export const cargoPassed = (log) =>
+  [...log.matchAll(/test result: \w+\. (\d+) passed/g)].reduce((sum, m) => sum + Number(m[1]), 0);
+
+const ranTests = (count) => (log) => (count(log) > 0 ? null : "No tests ran.");
+
+export function steps(root) {
+  const bin = (name) => join(root, "node_modules/.bin", name);
+  const rust = join(root, "src-tauri");
+  const cargo = cargoDir();
+  const cargoStep = (name, args, check) =>
+    cargo
+      ? { name, cmd: join(cargo, "cargo"), args, cwd: rust, env: { PATH: `${cargo}${delimiter}${process.env.PATH}` }, check }
+      : { name, missing: "cargo not found. Install Rust (see README), or put cargo on PATH." };
+  return [
+    { name: "types", cmd: bin("tsc"), args: ["--noEmit", "-p", "."] },
+    // composite would write tsconfig.node.tsbuildinfo into the tree.
+    { name: "config types", cmd: bin("tsc"), args: ["--noEmit", "-p", "tsconfig.node.json", "--composite", "false"] },
+    // CI rejects a stray `.only`; so does this, or a local pass would fail there.
+    { name: "tests", cmd: bin("vitest"), args: ["run", "--allowOnly=false"], check: ranTests(vitestPassed) },
+    // Outside the tree, so dist/ stays as the last real build left it.
+    { name: "web build", cmd: bin("vite"), args: ["build", "--outDir", join(stateDir(root), "web"), "--emptyOutDir"] },
+    cargoStep("rust format", ["fmt", "--check"]),
+    cargoStep("rust lint", ["clippy", "--all-targets", "--locked", "--", "-D", "warnings"]),
+    cargoStep("rust tests", ["test", "--lib", "--locked"], ranTests(cargoPassed)),
+  ];
+}
+
+/** What these checks can't see. */
+export const unseen = "the app window (WebKit), real AI calls, file dialogs, and the keychain";