Skip to content

fix: preserve UTF-8 across agent output chunks - #140

Merged
zzxwill merged 1 commit into
masterfrom
fix/agent-utf8-stream
Oct 21, 2026
Merged

zzxwill merged 1 commit into
masterfrom
fix/agent-utf8-stream

Conversation

@zzxwill

@zzxwill zzxwill commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

When a UTF-8 character crossed two process-output chunks, Jury decoded each fragment separately: a reviewer writing café produced caf�� in the live stream and saved finding. Decode stdout and stderr incrementally so Unicode text survives pipe boundaries, structured result parsing, and failure diagnostics.

Five real-child-process regressions failed before the fix and pass afterward. Acknowledgments force the split instead of assuming OS chunk sizes or relying on sleeps. Coverage includes two-, three-, and four-byte characters, structured results, and stderr diagnostics. Usage documentation and the changelog describe the correction.

Validation on a9059df0d0dfae5bb0eaf8637efd90e8ea198714:

  • node --test test/stream.test.js: 10 passed.
  • npm test: 277 passed, 0 failed.
  • npm run test:package: passed.
  • git diff --check origin/master...HEAD: passed.
  • Actual Jury: Codex reviewer and judge configured, exit 0, NO NEW FINDINGS, one round on the unchanged final commit. The reviewer sandbox blocked temp-directory creation, so its review inspected the diff and assertions; independent runtime checks passed on the same commit.

Portable equivalent of the executed local command (the recorded invocation used the absolute worktree path and redirected output to an evidence log):

GIT_AUTHOR_NAME=zzxwill GIT_COMMITTER_NAME=zzxwill node bin/jury.js review --dir "$PWD" --trunk master --reviewer codex --judge codex --rounds 2 --summary 'Fix UTF-8 corruption when agent stdout or stderr splits a multibyte character across pipe reads. Five regressions fail before the fix and pass after, using a real local child process with an acknowledgment between writes. Verify live output, raw text, parsed findings, structured results, and stderr diagnostics. All 277 tests and installed-package checks pass. Keep review fixes limited to this decoding bug and its regressions.' --web=false --push=false

Final-head remote CI is pending.

@zzxwill
zzxwill marked this pull request as ready for review October 21, 2026 14:10
@zzxwill
zzxwill merged commit 3445f38 into master Oct 21, 2026
8 checks passed
@zzxwill
zzxwill deleted the fix/agent-utf8-stream branch October 21, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant