[abrichr]

A launch demo, built on real runs. Claude enters one openIMIS insurance claim, OpenAdapt compiles that run into a program, and the program enters the next claims with no model calls, reading each save back from the database. This PR also starts the scoreboard: one benchmark for the agent alone, its first run under OpenAdapt, and the compiled program.

Status: demo profile, not admitted for production. The standard run gate refuses these openIMIS forms today. The reasons are in demo/cua_to_program/results/launch-4/admission.json and the README.

Measured (openIMIS, run launch-4, synthetic data)

	Agent (once)	Compiled program
Model	claude-opus-5-5, computer_toolset_20260801	none
Time	44.75 s	median 14.38 s per saved claim (n=49)
Model calls / cost	8 / $0.1176	0 / $0
Read back from the database	no (correct when read afterwards)	49 of 50 saved, 4/4 checks each, all first attempt
Claim 50 (Save button relabelled)	—	stopped before saving, 3 of 3 attempts, nothing written; batch paused, 51–100 wait for a person

• Load: all times were taken on a shared host at load 59–620 on 12 cores, so they are labelled as such everywhere. Timed numbers will come from the scoreboard at load 12 or under.
• Engine: the batch ran on main plus fix(flow): re-observe once before halting on an identity mismatch #1293 (local merge a8cca993, recorded in the results).
• Correctness batch: 100 claims with no UI change, in progress (results/launch-4-c100/).
• Spend: Claude API spend is tracked in packages/flow/benchmark/scoreboard/raw/spend.json ($3.17 when this was written) against the $30 cap.

Videos

• cua-to-program-full.mp4 (83.9 s)
• cua-to-program-social.mp4 (19.5 s)

Engine changes (packages/flow)

Strict improvements only; no gate or lease is relaxed here.

• jit.WebApp.setup(page): unmeasured sign-in and blank-form setup, never recorded.
• jit.WebApp.open(): matches feat(flow): compile_agent runs on Windows, macOS, Linux, RDP and Citrix apps #1283.
• jit.WebApp.record_video_dir and capture_settle_timeout_s: tests in tests/test_jit_webapp_setup.py.
• Effect mining: once parameters are confirmed, they join the record selector. Without this, the mined check matched every earlier record with the same constant fields and refuted every item after the first. Tests in tests/test_effect_mining.py.

Scoreboard (packages/flow/benchmark/scoreboard)

PROTOCOL.md, the row schema (outcome decided by read-back), tasks (MockMed in-process with injected faults and a rename at item 50; openIMIS; OpenEMR imported as its published set), the runner, importers, and the builder for scoreboard.json and SCOREBOARD.md. Arms 1–3 are running now, and the raw data is committed.

Follow-ups (separate PRs)

• fix(flow): re-observe once before halting on an identity mismatch #1293: re-observe once on a re-render (reviewed; CI queued).
• fix(flow): run gate counts a submit binding as an icon-only Save's identity #1274: the run gate counts a submit binding as identity. Draft; needs safety review.
• fix(flow): guarded focusing click on new-record form fields #1275: focusing-click lease on new-record forms. Draft; needs safety review.
• Open, no PR:
  • checks compiled from transient screens;
  • parameter hygiene misses OCR-confused values;
  • a fetch cut off by navigation never finishes in Playwright;
  • the openIMIS fixture re-fetches its compose files on every process.

🤖 Generated with Claude Code

Review

An independent review found no fail-open regression in the engine. Fixed since then:

• the generated scoreboard files are rebuilt from the committed raw data;
• the README and PR claims now match the data;
• the model is shown per arm;
• openIMIS counts a claim saved under another claim number as wrong;
• the UI state the app actually served is recorded per row;
• the scoreboard has a "done correctly, all items" column;
• protocol notes (Amendment 1);
• the effect-mining comments, probe text and a negative test.

Left as follow-ups:

• WebApp.open() doesn't check that setup leaves the page at url;
• the OpenEMR importer infers "reported" from the chart count;
• count_new_only for mined records.