Skip to content

Fault gate admits a one-table frame at 18/18 while independent faults show 9/15 missed #1506

Open

Description

@abrichr

Fail-open through admission. The fault gate (#1445) admits the OpenEMR referral contract with 18 of 18 faults killed, while an independently written fault suite shows that contract missing 9 of 15 faults.

Measured

Independent suite: benchmark/verification_coverage/independent/ (draft, branch verification-coverage/independent). The spec and the 17 cases were written and committed before reading the gate's classes or any generated check. They ran on a SQLite copy of the pinned OpenEMR fixture's real baseline (283 tables).

The task's shipped contract (scoreboard/claims_openemr.py VERIFY_YAML, reading patient_data, default frame):

fault gate independent suite
claims delta, frame
result 18/18 killed, clean passes 6/15 faults caught; 2 of 3 legitimate runs flagged

Missed: edits to another chart's history, a note and an allergy on another patient, an extra insurance row on the new chart, a missing history_data row, and three misread inputs saved faithfully (DOB, swapped names, provider).

Why

The gate applies its faults to the one table the check reads (SqliteCopy(table=...), and unrelated_edit edits "another record in the slice"). So frame coverage means "nothing else in this table moved". The coverage record and the receipt's checked.coverage don't say that the slice is one table out of 283.

What would close it

  • Record the slice in the coverage record (frame_slice: [patient_data], tables_in_store: 283), and let an admission policy require the slice to cover the app's write set from a witnessed save. The fixture already knows the write set: benchmark/openemr_local/fixture.py EXPECTED_TABLE_DELTAS.
  • Add a gate fault class that writes outside the read set (another table, the same or another subject). A check that can't see it reports frame as not_checked for those tables instead of held.

Activity

  1. added 2 commits that reference this issue on Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions