Skip to content

feat(flow): Meta-Harness-style trace export for GUI rollouts - #1300

Merged
abrichr merged 7 commits into
mainfrom
claude/harness2-trace-export
Oct 19, 2026
Merged

abrichr merged 7 commits into
mainfrom
claude/harness2-trace-export

Conversation

@abrichr

@abrichr abrichr commented Oct 18, 2026

Copy link
Copy Markdown
Member

Stacked on #1299. Merge that first.

What this adds

openadapt_flow.integrations.harness2.traces writes the filesystem that Meta-Harness gives its proposer: every candidate's harness, plus each rollout's full execution trace and its reward. Meta-Harness reports that full traces beat summaries, 56.7 to 38.7.

  • Full trace: every step is kept in full: the screen, the action, the agent's stated reason, and any error.
  • Identity at each write: was the patient on screen the requested one (verified / mismatch / unreadable)?
  • Read-back: the records added, whose they are, the outcome, the reward, and the receipt id.
  • Two sources:
  • Layout: INDEX.md, index.jsonl (best reward first), candidates/<label>/harness/, and traces/<id>/trace.{md,json}.
  • Privacy: screen text goes through privacy.scrub_text, the same as REPORT.md. Identity verdicts are computed before scrubbing, so they survive it.
  • Wiring: run_task writes the export for every task. In readback mode, the identity verdict also appears in the trajectory that Harness² proposers read.

Tests

tests/test_harness2_traces.py covers:

  • a wrong-patient write produces a mismatch, with a read-back showing one stray record;
  • a right-patient write is verified;
  • the export layout and its ordering;
  • scrubbed text keeps the verdict;
  • report.json keeps the runtime identity.

The adapter test now also checks the export and the identity line in the proposer's context. Both files pass locally (27 tests).

🤖 Generated with Claude Code

abrichr and others added 3 commits October 18, 2026 15:31
Adds openadapt_flow.integrations.harness2.traces. It writes a filesystem an
outer-loop proposer can read: every candidate's harness, and for each rollout
the full trace (every screen, action, reason and error), the identity verdict
at each write, and the read-back from the system of record with the reward.

- from_rollout reads an agent rollout; the identity at a write comes from the
  patient banner the agent was looking at.
- from_flow_run reads a compiled program's report.json, keeping the runtime's
  own identity-gate verdicts and effect checks.
- Screen text is scrubbed like REPORT.md; verdicts are computed first.
- run_task writes the export for every task. In readback mode the identity
  verdict also appears in the trajectory Harness² proposers read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from claude/harness2-adapter to main October 19, 2026 04:02
@abrichr
abrichr merged commit b07ba6c into main Oct 19, 2026
34 checks passed
@abrichr
abrichr deleted the claude/harness2-trace-export branch October 19, 2026 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant