Repository navigation
fix(experiment): preserve complete agent output - #644
Open
bingdongni wants to merge 1 commit into
Open
bingdongni wants to merge 1 commit into
bingdongni wants to merge 1 commit into
Conversation
|
Sentinel 已审核此变更,未发现需要报告的问题。 开发 agent 查询详情: 调用 MCP 工具 由 Sentinel 审核 · 目标提交 078408daf9f7 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Experiment comparisons can corrupt multibyte agent output when a UTF-8 character spans stream chunks, or parse an incomplete response when the process exits before its output streams close. Decode the streams incrementally and parse after they close.
What changed
How to test
node --test test/experimentAgentRunner.test.js test/experimentComparison.test.js(32 passed) andnpm test(3,308 passed), CI job.node --check index.jspassed. I did not run barenode index.jsbecause it starts an evolution run.proxyTracePlatformInstall.test.jsandspawnReplacementProcess.test.js, both excluded bynpm test. The same seven failures occur on the upstream base: platform templates are missing and some source assertions target obfuscated files.Risk
Medium: an agent that leaves its output pipes open after exiting will now wait for the pipes to close or the existing per-arm timeout. The result shape and command invocation are unchanged.
Harness/evaluator governance
Upstream governance surface:
src/experiment/agentRunner.jssubprocess output handling for comparison arms.Downstream EvoX impact: Experiment responses preserve complete UTF-8 output; the AgentResult shape and comparison contract are unchanged.
Rollout-local scope: Applies only when an experiment arm invokes its agent CLI; no live rollout is initiated by this change.
Promotion boundary: The experiment runner behavior changes by default only after maintainer review and merge.
Evaluator mismatch sets: Observation and evidence can now include previously corrupted or late output; action, repair, verification, and belief policies are unchanged.
Non-regression evidence: Real subprocess tests cover split stdout, split stderr, and delayed pipe closure; 3,308 project tests passed on Linux. The full template test selection has seven independently reproduced baseline failures as noted above.
Fix-severity review: medium
Owner approval: Experiment runner maintainer review is requested and remains pending.
Security boundary: The existing command, arguments, environment, host, network, and credential access are unchanged.
Rollback: Revert this commit to restore the previous output handling.
Live promotion: no
Autonomous evaluator self-editing: no
Self-check
node --test test/*.test.js). The seven baseline failures and CI results are described above.Related
None.