Skip to content

docs: record checks that cover their own blind spots (design) - #1434

Merged
abrichr merged 1 commit into
mainfrom
verification-coverage/design
Oct 20, 2026
Merged

abrichr merged 1 commit into
mainfrom
verification-coverage/design

Conversation

@abrichr

@abrichr abrichr commented Oct 20, 2026

Copy link
Copy Markdown
Member

Design only. No code changes.

packages/flow/docs/VERIFICATION_COVERAGE.md proposes one rule for what verified means, and three checks behind it:

The declared delta happened, nothing else in the declared slice changed, and the inputs match their source.

  • Frame conditions: the contract declares the slice and the target fields. A single judge compares the slice before and after the save, through the same read-only read. It attributes each change outside the declared records by using the app's audit log or a modified-by/version field. A change by another named user counts as concurrent. A change by the bot, or one nobody can explain, is collateral. This generalizes verify-run: a changed non-target record is collateral, not only a vanished one (#1373) #1375's forbid_collateral_change.
  • Mutation-qualified checks: before a check is admitted, seven fault classes are applied to a test copy's store, and the check reads them through its own read: wrong value per field, look-alike record, unrelated edit, partial, duplicate, missing, delayed. Coverage (killed / applicable) goes into a coverage record bound to the check's digest and into the receipt. scaffold-verifier's fault test becomes this gate.
  • Provenance: intake keeps each value's source line and the document digest. A second extraction by a different method must agree before the save; a disagreement goes to a person. After the save, a round trip compares the record as read back with the source document.
  • Receipt: one optional signed field, checked, says for each part held, refuted, not_checked (or instruction for the source part).

The section "What changes for existing runs" lists the behavior changes that need the safety review before any implementation PR merges. The first implementation applies the rule to verify-run and the guarded save. Flow replay follows in a separate PR.

🤖 Generated with Claude Code

Frame conditions (nothing else in the slice changed, with attribution of
concurrent edits), mutation-qualified checks (a fault suite applied to a
store copy, coverage signed into the receipt), and parameter provenance with
a second extraction and a round trip. Proposes what verified means and what
changes for existing runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr enabled auto-merge October 20, 2026 16:52
@abrichr
abrichr added this pull request to the merge queue Oct 20, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Oct 20, 2026
@abrichr
abrichr added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 68b9480 Oct 20, 2026
39 checks passed
@abrichr
abrichr deleted the verification-coverage/design branch October 20, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant