Skip to content

verify-run: a changed non-target record is collateral, not only a vanished one (#1373) - #1375

Merged
abrichr merged 4 commits into
mainfrom
claude/verify-run-collateral-change
Oct 20, 2026
Merged

abrichr merged 4 commits into
mainfrom
claude/verify-run-collateral-change

Conversation

@abrichr

@abrichr abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member

Fixes #1373.

Problem

verify-run caught a record outside the target that vanished, but not one that changed. On the scoreboard's OpenEMR claims run (#1365), the agent saved its own chart correctly while another chart's home phone was changed in place. verify-run returned verified on 5 of 5 of those items.

Change

  • Effect.forbid_collateral_change (record_written only). With a baseline, it refutes when a pre-state record that is not the target, and is still there by stable_id, has different fields. The reason names the fields. Records without an id are already covered, because their identity is their content and a change reads as a vanished record.
    • Off by default in a run. A run that reads a shared table other people edit would otherwise refute every save.
    • Bound into the contract hash only when set, like count_new_only, so every existing digest, authorization and ledger entry is unchanged.
  • verify-run applies it when it judges every record_written effect, unless the file sets forbid_collateral_change: false on that effect.
    • The contract as written is unchanged, so verify_config_sha256 still equals qualify's for the same file (test_the_qualify_example_loads_and_hashes_like_qualify passes).
    • evidence.readback.effects[].collateral_change_checked records that the check ran.
  • Docs: VERIFY_RUN.md (a new section and the verdict table) and EFFECT_KIT.md.

Tests

tests/test_verify_run_collateral_change.py:

  • The OpenEMR collateral_edit case is refuted, and names phone_home.
  • The old behaviour holds without the flag.
  • Untouched neighbours confirm, and a vanished neighbour is still collateral loss.
  • The target itself may change.
  • The flag is refused on field_equals.
  • Existing contract digests are unchanged.
  • verify-run judges with the check on and leaves the parsed contract alone, and an effect can opt out.
  • End to end on the MockMed store: an in-place edit of another record is refuted.

Ran locally, all passing: the effect, qualify, record-check, verify-run, run-gate, transaction-outcome and silent-wrong-action suites (1151 passed, 1 skipped); mypy on the three changed modules; scripts/check_release_consistency.py.

Safety review notes

  • This only adds refusals; it never turns a refusal into a confirmation.
  • Run-time contracts are unchanged unless they opt in.
  • The read-only audit path (verify-run) becomes stricter. A read set that includes records other actors edit during the window will now come back reconciliation_required. That is the intended fail-closed direction, and the docs say how to opt out per effect.

🤖 Generated with Claude Code

abrichr and others added 2 commits October 19, 2026 14:53
…ot only one that vanished (#1373)

Effect.forbid_collateral_change (record_written only, off by default in a
run, bound into the contract hash only when set) refutes when a pre-state
record that is not the target is still there but its fields differ.
verify-run applies it to every record_written effect when it judges, unless
an effect opts out, and leaves the file's contract and digest unchanged so
they still equal qualify's; the receipt's evidence records that the check
ran. Regression: the OpenEMR collateral_edit case from the scoreboard's
claims run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eline's nested keys come back sorted)

A rebuilt OpenEMR clean save was refuted: the baseline file stores records
as canonical JSON, so the existing chart's nested addresses came back with
their keys in another order and str() saw a change. Values are now compared
as sorted-key JSON. Test: nested key order is not a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr marked this pull request as draft October 19, 2026 19:35
@abrichr

abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

Draft until the review fixes are pushed: normalised SQL value types (datetime, Decimal, bytes) with a SQL round-trip test, and the refusal reason naming a possible concurrent edit. Mark ready once they're in.

… JSON form; say it may be another user's edit

Review of #1375: SQL rows come back as raw driver values (Decimal,
datetime, bytes) while verify-run's baseline went through JSON, so an
unchanged row could read as changed. Field values now compare by type:
numbers numerically when either side is a number, dates and times as
parsed values (either ISO separator), bytes against their hex or UTF-8
text, nested values element by element; two strings still compare exactly
('02139' to '2139' is a change). The refutation now says the change may be
another user's concurrent edit and that a person should reconcile it.
Tests: an unchanged SQL row (Decimal, TIMESTAMP, BLOB) read against its own
round-tripped baseline confirms, and a real edit to it refutes; pairwise
driver-vs-JSON cases; numeric-looking text compares exactly; the reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr

abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

Review fixes pushed: values compare by type against the baseline's JSON form (Decimal, datetime, bytes; nested values element by element; two strings exactly), with an SQL round-trip test, and the refutation says the change may be another user's concurrent edit that a person should reconcile. Local: 1160 passed, 1 skipped across the effect, qualify, record-check, verify-run, run-gate, transaction-outcome and silent-wrong-action suites; mypy clean. Known limit, unchanged by this PR: verify-run cannot write a baseline that holds non-UTF-8 bytes (EffectState JSON mode), so a binary column needs records_drop_fields.

@abrichr
abrichr marked this pull request as ready for review October 19, 2026 19:38
… content

The new Effect field entered every serialized workflow.json, so bundles
sealed before it no longer matched their content digest (registry pull,
loop showcase, and the pinned pre-field digest test failed in CI). It is
now a reviewed sealed-canonical omission when False, like new_records and
identity_field; a contract that turns the check on stays sealed content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr

abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

Included in batch #1384, which runs CI once for several reviewed PRs. This PR is marked merged when #1384 merges. Please don't push here unless asked; a new commit leaves it out of the batch.

@abrichr
abrichr merged commit d20023e into main Oct 20, 2026
7 of 35 checks passed
abrichr added a commit that referenced this pull request Oct 20, 2026
@abrichr
abrichr deleted the claude/verify-run-collateral-change branch October 20, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verify-run: collateral check catches a vanished non-target record but not a changed one

1 participant