Repository navigation
fix(qoder): 转发前拒绝上游空流 / reject empty upstream streams before relaying - #286
Merged
caigee-cmd merged 1 commit intoOct 9, 2026
Merged
Conversation
The in-process COSY client relayed Qoder's heartbeat-only 200 streams verbatim and always appended [DONE], so strict OpenAI-compatible clients (pi, DSH) failed the turn with 'Stream ended without finish_reason' while the request log recorded success. The Node worker path already guarded against this (upstream_empty); the v0.6.17 in-process rewrite dropped those guards. - ChatStream peeks the upstream body before the first client byte and fails over via providers.Error when the stream carries no content, finish_reason, usage, or error frame - writeOpenAIChunks synthesizes the terminal finish_reason (stop / tool_calls) when a relayed stream was cut before the terminal chunk, and emits a classified error frame when an empty stream slips past the pre-relay guard - ChatNonStream rejects empty upstream bodies instead of fabricating stop
This was referenced Oct 9, 2026
Closed
caigee-cmd
pushed a commit
that referenced
this pull request
Oct 9, 2026
A heartbeat-only upstream stream (upstream_empty, introduced as a pre-relay guard in #286) says nothing about the account's health: credential, quota, and rate-limit state are all fine, and the same account may succeed on the very next request. Classifying it as a generic unavailable failure applies the default 15s cooldown with exponential backoff on repeats, so whenever the upstream empty-stream rate spikes the pool starves: every account ends up cooled down and requests queue behind unrelated timeouts. Observed live while the upstream empty-stream rate was around 40-60%. Special-case Code=upstream_empty in ClassifyError: keep Failover=true (another account serves the turn) but set Cooldown=0 so the account stays in the rotation. Generic unavailable errors keep their cooldown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
中文
问题现象
严格的 OpenAI 兼容客户端(pi coding agent、DeepSeek Harness 及所有基于 pi-ai 的客户端)使用 Qoder 模型时会随机报错:
同时 cli2api 的请求日志把这几轮记录为成功。
根因
本地对 v0.6.19 实测复现:Qoder 上游网关偶发返回
200+ 仅含心跳帧 / 空 delta 帧的 SSE 流(2~3 帧,无content、无tool_calls、无usage、无终态finish_reason、无 error 帧),随后干净关闭连接。进程内 COSY 客户端(
internal/providers/qoder/direct.go)原样转发该响应体并无条件追加data: [DONE],产出一个"看起来正常但实际为空"且违反 OpenAI SSE 规范的补全(规范要求最后一个 choice chunk 必须携带非空finish_reason)。线上抓包的失败形态:
Node worker 老路径(
worker/src/sse.mjs)本来已有对应防线(upstream_empty结构化错误、upstream_stream_incomplete、终态 reason 合成),但 v0.6.17 的进程内重构(#276)把 Go 直连路径上的这些守卫丢掉了,而生产环境app.go无条件SetDirect,默认走的就是这条无守卫路径。修复内容
internal/providers/qoder/direct.go:ChatStream在向客户端写出第一个字节之前窥探上游 SSE 流。一旦出现有效帧(content / reasoning / tool_calls / 终态finish_reason/ usage / error),缓冲字节按序回放、后续照常实时转发,不损失流式体验。若上游流只发心跳就关闭,返回providers.Error{Kind: unavailable, Code: "upstream_empty", Status: 502}—— 此时还没有任何字节发给客户端,executor 可以正常分类、冷却该账号并切换到其他账号重试。writeOpenAIChunks现在跟踪已转发流是否出现过终态finish_reason。若上游已输出真实内容但在终态 chunk 前被切断(字节已上线、无法重试),合成终态 chunk(有 tool_calls 则tool_calls,否则stop),严格客户端可以挽救部分答案而不是硬失败。若有空流漏过前置守卫,则发出 error 帧(data: {"error": ...}),网关已有的错误分类链路会把它记录为失败而不是成功。outcomeFromUpstreamSSE对空上游响应体报upstream_empty,不再凭空合成finish_reason: "stop"。测试
internal/providers/qoder/direct_test.go新增回归测试:TestDirectChatStreamRejectsEmptyUpstreamBeforeRelay— 仅心跳流 → 转发前报upstream_emptyTestDirectChatStreamReplaysPeekedFrames— 窥探到的帧完整回放,不丢失TestDirectChatStreamSalvagesTruncatedStream/...TruncatedToolCalls— 终态前被切断的流合成stop/tool_calls终态帧TestDirectChatNonStreamRejectsEmptyUpstreamgo test ./...、go test -race(qoder/gateway/executor)、go vet ./...、gofmt、git diff --check全部通过。其他
changelog/unreleased/约定添加双语文案qoder-empty-stream-guard.mdEnglish
Problem
Strict OpenAI-compatible clients (pi coding agent, DeepSeek Harness, anything on pi-ai) fail Qoder-model turns with
Error: Stream ended without finish_reason, while the cli2api request log records those turns as success.Root cause
Reproduced locally against v0.6.19: the Qoder upstream gateway intermittently answers
200with an SSE stream carrying only heartbeat / empty-delta frames (nocontent, notool_calls, nousage, no terminalfinish_reason, no error frame), then closes cleanly. The in-process COSY client relayed that body verbatim and unconditionally appendeddata: [DONE], producing a valid-looking but empty, spec-invalid completion.The Node worker path (
worker/src/sse.mjs) already guarded against this (upstream_empty,upstream_stream_incomplete, terminal-reason synthesis), but the v0.6.17 in-process rewrite (#276) dropped those guards on the Go path — which is the default path in production (app.goalways callsSetDirect).Fix
ChatStreampeeks the upstream body before the first client byte. Meaningful frames are replayed in order (real-time streaming preserved); a heartbeat-only stream fails withupstream_emptybefore any byte is relayed, so the executor can classify, cool the account down, and fail over.finish_reason(bytes already on the wire) get a synthesizedstop/tool_callsterminal chunk; an empty stream that slips past the pre-relay guard emits an error frame that the gateway classifies and logs as a failure.upstream_emptyinstead of fabricatingfinish_reason: "stop".Tests
Five new regression tests;
go test ./...,-race,go vet,gofmt,git diff --checkall pass. Bilingual changelog fragment included per convention.