Skip to content

Self-repair can't take openIMIS's relabelled Save: RECONCILIATION_REQUIRED from unreviewed-risk steps, and the halt is on the scroll #1331

Open

Description

@abrichr

What happened

The scoreboard ran the openIMIS claim batch through SelfRepair (#1265; no signed repair policy, so #1294's standard changes don't apply). It used the program from the launch demo, on the demo profile with the SQL record check. Results: claims 45 to 49 were correct (not_needed, 0 model calls). At claim 50 the icon-only Save becomes "Save claim", and self-repair answered not_repairable on 3 of 3 attempts. Nothing was written, the repair agent was never called, and claims 51 to 100 are paused for a person.

Rows: packages/flow/benchmark/scoreboard/raw/openimis/program_repair/2026-10-19-a/items.csv (#1303). Runner: scoreboard/repair_arm.py.

Two reasons, both fail-closed

  1. The run ends RECONCILIATION_REQUIRED, not HALTED_BEFORE_EFFECT. step_002 (press Tab after the insurance number) and step_012 (a click on an unlabelled service-picker target) carry risk_review_required. transaction._is_consequential_result counts them as possibly consequential, and no verifier proves they had no effect. So classify_transaction_outcome falls back to RECONCILIATION_REQUIRED, and SelfRepair.run stops there. This is the same unreviewed-risk gap as JIT admission: openIMIS claim program refused under clinical-write (risk override, idempotency key, low confidence) #1329 (require_reviewed_action_risk). Until a reviewed risk classification can be bound through the JIT path, no openIMIS halt can be repaired. A program compiled from a Sonnet 5.5 recording of the same task (sb-sonnet-1) ends the same way.
  2. The halt is on a scroll, not on the Save click. This program scrolls back up before Save (step_014, closed loop, looking for step_015's anchor). After the relabel, the scroll exhausts its budget without finding the Save, so the run halts at step_014. _repairable accepts only a click halt (step_014 is not a click on an on-screen target). Here the missing control is the next step's click target, and the scroll only failed because it couldn't find it.

Possible fixes (for a safety review)

Without (1), (2) changes nothing on openIMIS.

Reproduce

cd packages/flow/benchmark
python -m scoreboard.repair_arm --program-run launch-4 --run-id <id> --tag <2-3 letters> --start 45 --n 100

This needs the openIMIS stack from demo/cua_to_program/run.sh and the compiled launch-4 program.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions