Repository navigation
JIT lift: a run input typed in two steps (agent detour) is never lifted; distill keeps the detour #1328
Description
What happened
In the scoreboard's OpenEMR referral task (arm 2, run 2026-10-18-a), Claude Sonnet 5.5 registered the patient correctly under the JIT wrapper: the SQL read-back found 12 of 12 fields. The candidate could not be admitted, because the run input sex was never proposed as a parameter:
AdmissionError: --accept-params: no flagged proposal named 'sex' (available: ['DOB', 'city', 'country_code', 'email', 'fname', 'form_sex', 'form_sex_identified', 'lname', 'phone_home', ...])
Evidence: packages/flow/benchmark/scoreboard/raw/openemr_referral/first_run/2026-10-18-a/items.csv (on the claude/launch-demo-cua-to-program-8fa902 branch, #1303).
Why
The agent typed the value Female twice:
- Step 10: into
#form_sex_identified(Gender Identity), the wrong field. - Step 34: into
#form_sex(Birth Sex), after OpenEMR showed "Birth Sex is not valid".
openadapt_flow/jit/lift.py lift_params lifts an input only when its value is typed as a whole in exactly one step. Here it reported typed in 2 steps and left sex unlifted. The two typed steps kept their field-label proposals (form_sex_identified, form_sex), which carry no source: task_input.
Distill kept the detour as well (distill.json: 39 steps in, 39 out, nothing dropped). So the compiled program would also write the value into Gender Identity, a field the task never asked for. The read-back doesn't check that field, so the run still counted as correct.
Two gaps
- Lifting. When an input's value is typed in several steps, the lifter could prefer the step whose field the final record shows the value in (or the last write to a field that stuck), instead of giving up. If that's not safe, it should name the candidate steps so the reviewer can pick one in
--accept-params. - Distill. A type into one field, a validation error, then the same value typed into another field is a correction. The first write is not part of the task, and distill should propose dropping it. Even if the parameter question is resolved, replaying the detour writes an extra field.
Reproduce
cd packages/flow/benchmark
python -m scoreboard.run first_run openemr_referral --run-id <id>This needs the OpenEMR fixture; see scoreboard/tasks/openemr.py. The agent may or may not take the same detour on another run.
Activity
abrichr commentedon Oct 19, 2026
Fixed in #1336.
also_typed_in. It fails closed, naming the steps, when there's no record check, or when none or several of the steps match.save_retried). The misdirected entry is dropped with its focus click and Enter (entry_corrected), only when the saved record holds the value under the second field's name and has no field named like the first.Re-run of arm 2 admission
From the recorded
2026-10-18-arun, rebuilt offline on #1303 + #1336 + #1339. No model calls.sexat Birth Sex.confirm_paramsandreview: both succeed. No placeholder effects.clinical-writecertification still refuses, so the canary's gate refuses before anything runs. None of these come from this issue:require_reviewed_action_risk: Enter on steps 002, 019, 022, 031.prohibit_vacuous_postconditions: those Enters, plus step_033.prohibit_unarmed_clicksandrequire_identity_for: step_010 (the Contact tab) and step_033.require_idempotency_key_foron step_034 (Confirm Create): a new patient has no input that identifies one record, so feat(jit): a person names the record check's idempotency key at confirm_params #1339's person-named key doesn't apply.