Repository navigation
feat(flow): Meta-Harness-style trace export for GUI rollouts - #1300
Merged
Merged
Conversation
Adds openadapt_flow.integrations.harness2.traces. It writes a filesystem an outer-loop proposer can read: every candidate's harness, and for each rollout the full trace (every screen, action, reason and error), the identity verdict at each write, and the read-back from the system of record with the reward. - from_rollout reads an agent rollout; the identity at a write comes from the patient banner the agent was looking at. - from_flow_run reads a compiled program's report.json, keeping the runtime's own identity-gate verdicts and effect checks. - Screen text is scrubbed like REPORT.md; verdicts are computed first. - run_task writes the export for every task. In readback mode the identity verdict also appears in the trajectory Harness² proposers read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1299. Merge that first.
What this adds
openadapt_flow.integrations.harness2.traceswrites the filesystem that Meta-Harness gives its proposer: every candidate's harness, plus each rollout's full execution trace and its reward. Meta-Harness reports that full traces beat summaries, 56.7 to 38.7.verified/mismatch/unreadable)?from_rolloutreads an agent rollout from feat(flow): Harness² benchmark adapter scored by read-back #1299.from_flow_runreads a compiled program'sreport.jsonand keeps the runtime's own identity-gate verdicts and effect checks. The repair search in the next PR reads halted runs this way.INDEX.md,index.jsonl(best reward first),candidates/<label>/harness/, andtraces/<id>/trace.{md,json}.privacy.scrub_text, the same asREPORT.md. Identity verdicts are computed before scrubbing, so they survive it.run_taskwrites the export for every task. Inreadbackmode, the identity verdict also appears in the trajectory that Harness² proposers read.Tests
tests/test_harness2_traces.pycovers:report.jsonkeeps the runtime identity.The adapter test now also checks the export and the identity line in the proposer's context. Both files pass locally (27 tests).
🤖 Generated with Claude Code