feat(verify-run): judge any executor's write from the system of record, aligned with RFC-0002

packages/flow/docs/VERIFY_RUN.md

@@ -0,0 +1,141 @@
+# Check a write that something else made
+
+Preview, aligned with RFC-0002 (Open Effect Receipt). The receipt format may
+still change before the RFC is final.
+
+An agent, a script, or a person says it saved a record. `verify-run` reads
+the system of record and decides whether that is true. It uses the same
+record checks a production `run` uses (`rest`, `sql`, `fhir`, `graphql`, and
+the other kinds in [`EFFECT_KIT.md`](EFFECT_KIT.md)). It makes no AI model
+calls.
+
+```bash
+# Before the agent starts: read the store so new rows can be told from old ones.
+openadapt-flow verify-run --verify verify.yaml --params task.json \
+  --capture-baseline before.json
+
+# ... the agent does its work ...
+
+openadapt-flow verify-run --verify verify.yaml --params task.json \
+  --run-record agent-run.json --authorization-ref TICKET-1234 \
+  --baseline before.json --out share/
+```
+
+`verify.yaml` has two parts, each in a shape Flow already reads:
+
+```yaml
+schema_version: openadapt.verify/v1
+verifier:        # the same keys as the effects: section of deployment.yaml
+  kind: rest
+  base_url: http://localhost:8080
+  records_path: /api/db
+effects:         # the same list as effect_contract.yaml
+  - kind: record_written
+    match:
+      patient_id: {param: patient_id}
+      note: {param: note}
+    expected_count: 1
+    count_new_only: true
+```
+
+It is the same file `openadapt-flow qualify --verify` reads (see
+[`QUALIFY_A_WORKFLOW.md`](QUALIFY_A_WORKFLOW.md)). verify-run ignores the
+qualify-only sections (`app`, `save_step`, `inputs`, `campaign`, and
+`verifier.session`), expands `record` to the same effect list qualify uses,
+and expands `${NAME}` environment references the same way. So the receipt's
+`workflow_digest` equals the `verify_config_sha256` qualify reports for the
+same file. `max_execution_window_s` (default 3600) is the one key only
+verify-run reads.
+
+The three inputs come from different people on purpose:
+
+- `--params` holds the values the task named, such as which patient. They
+  come from whoever assigned the task, so the agent can't pick what it is
+  checked against.
+- `--run-record` is what the agent reports. It must name `executor_kind`
+  (`agent`, `script`, `human`, `flow_replay`, or `unknown`). The receipt
+  keeps a salted digest of the file and reads nothing else from it except
+  `executor_id` and `executor_build`, which it also keeps only as digests.
+- `--baseline` is the read taken before the agent started. Without it, a
+  contract that counts new rows can't be judged, and the verdict is
+  `INDETERMINATE`. A baseline works once: a second run with the same file
+  is refused. One older than `max_execution_window_s` (default 3600) counts
+  as no baseline.
+
+Optional flags:
+
+- `--approver approver.json` names who approved the write, as
+  `{kind, id, policy_ref}` with an opaque `id`. If the approver is the
+  executor itself, the outcome is `rejected_policy` even when the write
+  landed.
+- `--remediates RECEIPT_ID` links this receipt to an earlier one it fixes.
+- `--correlation-id ID` carries your own ticket or trace id.
+
+Both link ids must be opaque (8 to 128 of `A-Za-z0-9._:-`). They are absent
+unless given, and when present they are signed with the rest of the receipt.
+
+## Verdicts
+
+| What the store shows | `effect_verdict` | `outcome` |
+|---|---|---|
+| The declared record, exactly once, through a system-of-record read | `CONFIRMED` | `verified` |
+| The same, but the executor approved its own write | `CONFIRMED` | `rejected_policy` |
+| Nothing that matches | `REFUTED` | `halted_before_effect` |
+| A duplicate, a wrong value, or only some of the effects | `REFUTED` | `reconciliation_required` |
+| The store could not be read, no baseline, or a separate-session read | `INDETERMINATE` or `CONFIRMED` | `reconciliation_required` |
+
+Only a read through the system of record produces `verified`. A screen
+read-back (`onscreen`) is refused before anything runs.
+
+`verification_tier` on the receipt says how the effects were checked
+(RFC-0002, weakest effect wins):
+
+- `independent_readback`: every effect was read from the system of record
+  without the UI (Flow evidence tier 1).
+- `screen_readback`: at least one effect was read through a UI session,
+  including a separate signed-in session (Flow tier 2).
+- `unchecked`: at least one effect was not read.
+
+Only `independent_readback` can yield `verified`.
+
+## The receipt
+
+`share/effect-receipt.json` holds `receipt` and `evidence`. `receipt` has
+every field of the Execute receipt (`openadapt.execute-evidence-receipt/v1`)
+plus `executor_kind`, `executor_ref_sha256`, `effect_verdict`, and
+`verification_tier` (and `remediates` and `correlation_id` when given),
+under the schema `openadapt.effect-receipt/v1`. `evidence.readback` lists
+each effect's contract hash, verdict, and counts. `receipt.evidence_digest`
+is the SHA-256 of `evidence`. Neither part holds record values, URLs, or the
+agent's own text.
+
+Every digest whose input could hold customer data (the run record, the
+authorization reference, the baseline, and each bound parameter inside an
+effect's contract hash) is an HMAC-SHA256 keyed with a 256-bit salt drawn
+for this run (`openadapt.run-salt/v1`). `evidence.salting.salt_commitment`
+is the SHA-256 of the salt. The salt itself stays in a private file: the
+baseline, or `share/effect-run.private.json` when there is no baseline. Each
+REST read carries `X-OpenAdapt-Execution-Id` and a `traceparent` header so
+the store's own logs can be matched to the run.
+
+`evidence.authorization.approver` keeps a placeholder shape until OpenAdapt
+types defines approver identity.
+
+The file is unsigned. An issuer signs it with
+`openadapt.effect-seal-signature/v1`, over its own domain, so a verifier
+built for Execute Seals refuses an effect receipt instead of passing it.
+Check a signed file offline against keys you pinned:
+
+```bash
+openadapt-flow verify-run --check signed.json --keys seal-keys.json
+```
+
+The check reports the signature (`verified`) and the outcome
+(`verified_outcome`) separately. It exits 0 only when both hold. A genuine
+signature over `halted_before_effect` still exits 1 unless you pass
+`--accept-outcome halted_before_effect` (repeatable). An unsigned file is
+never a pass.
+
+The baseline and run files hold raw records or the salt. They are written
+with mode 0600 and never go into the receipt. Delete them when you no longer
+need them.

packages/flow/openadapt_flow/__main__.py

@@ -2415,6 +2415,80 @@ def _cmd_attach_seal(args: argparse.Namespace) -> int:
     return 1
 
 
+def _cmd_verify_run(args: argparse.Namespace) -> int:
+    """``verify-run``: judge an effect another executor claims, from the store."""
+    from openadapt_flow.verify_run import (
+        VerifyRunError,
+        capture_baseline,
+        check_effect_receipt,
+        load_verify_config,
+        verify_run,
+        write_result,
+    )
+
+    def _json(path: str) -> Any:
+        return json.loads(Path(path).read_text(encoding="utf-8"))
+
+    if args.check:
+        if not args.keys:
+            print(
+                "verify-run --check needs --keys (a pinned Seal key registry)",
+                file=sys.stderr,
+            )
+            return 2
+        registry = _json(args.keys)
+        if isinstance(registry, dict):
+            registry = registry.get("keys", [])
+        result = check_effect_receipt(
+            _json(args.check), registry, accept_outcomes=args.accept_outcome or ()
+        )
+        print(json.dumps(result, indent=2, sort_keys=True))
+        if result.get("verified") and not result.get("verified_outcome"):
+            print(
+                "The signature is genuine, but the outcome is not verified.",
+                file=sys.stderr,
+            )
+        return 0 if result.get("ok") else 1
+    if not args.verify:
+        print("verify-run needs --verify verify.yaml", file=sys.stderr)
+        return 2
+    try:
+        cfg = load_verify_config(args.verify)
+        params = _json(args.params) if args.params else {}
+        if not isinstance(params, dict):
+            raise VerifyRunError("--params must be a JSON object")
+        if args.capture_baseline:
+            capture_baseline(cfg, params, Path(args.capture_baseline))
+            print(
+                f"baseline written: {args.capture_baseline} (private; holds raw records)"
+            )
+            return 0
+        if not args.run_record or not args.authorization_ref or not args.out:
+            raise VerifyRunError(
+                "verify-run needs --run-record, --authorization-ref, and --out"
+            )
+        result = verify_run(
+            cfg,
+            params=params,
+            run_record=_json(args.run_record),
+            authorization_ref=args.authorization_ref,
+            baseline_path=Path(args.baseline) if args.baseline else None,
+            private_dir=Path(args.out),
+            approver=_json(args.approver) if args.approver else None,
+            remediates=args.remediates,
+            correlation_id=args.correlation_id,
+        )
+    except VerifyRunError as exc:
+        print(f"verify-run REFUSED: {exc}", file=sys.stderr)
+        return 2
+    out = write_result(result, Path(args.out))
+    receipt = result["receipt"]
+    print(f"effect verdict: {receipt['effect_verdict']}")
+    print(f"outcome: {receipt['outcome']}  executor: {receipt['executor_kind']}")
+    print(f"receipt (unsigned): {out}")
+    return 0 if receipt["outcome"] == "verified" else 1
+
+
 def _cmd_run(args: argparse.Namespace) -> int:
     """Execute a bundle under a named deployment profile -- FAIL-CLOSED.
 
@@ -7448,6 +7522,100 @@ def build_parser() -> argparse.ArgumentParser:
     )
     sa.set_defaults(func=_cmd_attach_seal)
 
+    p = sub.add_parser(
+        "verify-run",
+        help=(
+            "Read the system of record and rule on an effect that an agent, "
+            "script, or person claims it made; writes an unsigned "
+            "openadapt.effect-receipt/v1. The executor's own record is "
+            "hashed, never trusted"
+        ),
+    )
+    p.add_argument(
+        "--verify",
+        default=None,
+        metavar="YAML",
+        help="verify.yaml: verifier wiring and effect contract",
+    )
+    p.add_argument(
+        "--params",
+        default=None,
+        metavar="JSON",
+        help="The authorized task's values the contract binds",
+    )
+    p.add_argument(
+        "--run-record",
+        default=None,
+        metavar="JSON",
+        help="The executor's run record (needs executor_kind)",
+    )
+    p.add_argument(
+        "--authorization-ref",
+        default=None,
+        help="Opaque id of the authorization in your own system",
+    )
+    p.add_argument(
+        "--baseline",
+        default=None,
+        metavar="JSON",
+        help="Baseline captured before the executor ran",
+    )
+    p.add_argument(
+        "--capture-baseline",
+        default=None,
+        metavar="JSON",
+        help="Read the store now and write a private baseline, then exit",
+    )
+    p.add_argument(
+        "--out", default=None, metavar="DIR", help="Directory for effect-receipt.json"
+    )
+    p.add_argument(
+        "--check",
+        default=None,
+        metavar="JSON",
+        help="Check a signed effect receipt offline instead",
+    )
+    p.add_argument(
+        "--keys",
+        default=None,
+        metavar="JSON",
+        help="Pinned Seal key registry for --check",
+    )
+    p.add_argument(
+        "--accept-outcome",
+        action="append",
+        default=None,
+        choices=[
+            "halted_before_effect",
+            "reconciliation_required",
+            "rejected_policy",
+            "failed_platform",
+        ],
+        help=(
+            "With --check: also exit 0 for this outcome when the signature is "
+            "genuine (repeatable; no value accepts every outcome)"
+        ),
+    )
+    p.add_argument(
+        "--approver",
+        default=None,
+        metavar="JSON",
+        help="Who approved the write: {kind, id, policy_ref}; id is opaque",
+    )
+    p.add_argument(
+        "--remediates",
+        default=None,
+        metavar="RECEIPT_ID",
+        help="receipt_id of an earlier effect receipt this run remediates",
+    )
+    p.add_argument(
+        "--correlation-id",
+        default=None,
+        metavar="ID",
+        help="Your own opaque id linking this receipt to a ticket or trace",
+    )
+    p.set_defaults(func=_cmd_verify_run)
+
     p = sub.add_parser(
         "resume",
         help=(