Skip to content

feat(flow): search repair bridges on a staging copy, scored by read-back - #1302

Closed
abrichr wants to merge 7 commits into
mainfrom
claude/harness2-staging-repair
Closed

abrichr wants to merge 7 commits into
mainfrom
claude/harness2-staging-repair

Conversation

@abrichr

@abrichr abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member

Draft. It is stacked on three things:

The diff against this base therefore also shows #1299 and #1300. Once those land, I'll retarget this PR to main. This PR's own files are repair/staging_search.py, tests/test_staging_search.py, the menu2 drift in mockmed/static/app.js, and a section in docs/HARNESS2.md.

Why

Self-repair probes a proposed fix up to the last point before the save, then stops. That shows the bridge reaches the save on the right record. It can't show what the save writes. Two bridges can look the same up to the save and still write different records.

What this adds

openadapt_flow.repair.staging_search does the following for each candidate bridge, on a resettable staging copy of the app:

  1. Runs SelfRepair with that one bridge (halt, patch, probe, campaigns) and stops at the approval request, which nobody answers.
  2. Runs the patched bundle end to end on a fresh copy, through the save.
  3. Reads the copy's store with the reward worker and scores it, using the same search reward as feat(flow): Harness² benchmark adapter scored by read-back #1299.

StagingSearchAgent is a RepairAgent. It hands production self-repair one proposal: the best candidate that verified on staging, or none. After that the pipeline is unchanged: production probe, campaigns, one person's approval, staged, canary, active. The search never opens a production session. It uses only SelfRepair's public API and adds nothing to self-repair itself.

  • Candidate proposers:
    • ScriptedBridges for tests.
    • ModelBridges: reads the halted step, the controls OCR finds on the live screen, and the full traces of the halted run and the last N verified runs (traces.from_flow_run). It answers with control labels, which are mapped to screen points, so it never invents coordinates.
  • MockMed ?drift=menu2: the save moved into a menu with two save paths, Billing > Submit encounter (Triage) and Clinical > Submit as Consult. Both pass self-repair's probe.

Tests

tests/test_staging_search.py uses self-repair's own MockMed fixtures, with a separate staging server:

  • End to end: on staging, the Consult path writes a Consult record (wrong value, -0.5) and the Billing path verifies (1.0). Production self-repair then receives only the Billing bridge, a person approves it on the phone, the resumed run is VERIFIED, and production holds exactly one Triage record.
  • No verified candidate: the agent proposes nothing, and it proposes at most once.
  • ModelBridges: maps labels to points, and the halted trace reaches the prompt.

All three pass locally (about 3 minutes, because of the tutorial recording).

Coordination: self-repair owns the bridge mechanics. Pre-approved repairs owns auto-promotion. This PR changes neither.

🤖 Generated with Claude Code

abrichr and others added 7 commits October 18, 2026 15:51
Self-repair's probe stops before the save, so it cannot tell two bridges
apart that reach the save the same way but write different records.
openadapt_flow.repair.staging_search runs each candidate through self-repair
on a resettable staging copy, then runs the patched bundle through the save
there and reads the copy's store with the reward worker. The best verified
candidate is the one proposal production self-repair receives; its probe,
campaigns, approval, staging and canary are unchanged. The search never
opens a production session.

- StagingCopy, StagingBridgeSearch, StagingSearchAgent (a RepairAgent).
- ScriptedBridges for tests; ModelBridges reads the halted trace and the
  last verified traces and answers with control labels mapped to points.
- MockMed ?drift=menu2: a second save path in the menu (Submit as Consult).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Each search runs in its own directory; run paths, stores and episodes are
  single-use, so a second search no longer fails.
- Every staging candidate's approval request is withdrawn and the staging
  candidate rejected, so nothing on staging can be approved.
- The rollout's reward worker key and token are deleted after scoring.
- The reward contract checks every field of StagingCopy.record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr

abrichr commented Oct 19, 2026

Copy link
Copy Markdown
Member Author

The self-repair bridge branch is now #1325. This PR stays a draft on top of it: the staging search was deferred by the founder's decision on 2026-10-18. The names it uses (RepairProposal(actions=), SelfRepairResult.patched_bundle, the tests.test_self_repair helpers) are unchanged there.

Base automatically changed from claude/self-repair-bridge to main October 19, 2026 16:32
@abrichr

abrichr commented Oct 20, 2026

Copy link
Copy Markdown
Member Author

Closing as stale during PR cleanup; superseded by later work. Reopen if still needed.

@abrichr abrichr closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant