Skip to content

fix(qoder): 转发前拒绝上游空流 / reject empty upstream streams before relaying - #286

Merged
caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
BrianBoyCN:fix/qoder-empty-stream-guard
Oct 9, 2026
Merged

caigee-cmd merged 1 commit into
caigee-cmd:mainfrom
BrianBoyCN:fix/qoder-empty-stream-guard

Conversation

@BrianBoyCN

Copy link
Copy Markdown
Contributor

中文

问题现象

严格的 OpenAI 兼容客户端(pi coding agent、DeepSeek Harness 及所有基于 pi-ai 的客户端)使用 Qoder 模型时会随机报错:

Error: Stream ended without finish_reason

同时 cli2api 的请求日志把这几轮记录为成功。

根因

本地对 v0.6.19 实测复现:Qoder 上游网关偶发返回 200 + 仅含心跳帧 / 空 delta 帧的 SSE 流(2~3 帧,无 content、无 tool_calls、无 usage、无终态 finish_reason、无 error 帧),随后干净关闭连接。

进程内 COSY 客户端(internal/providers/qoder/direct.go)原样转发该响应体并无条件追加 data: [DONE],产出一个"看起来正常但实际为空"且违反 OpenAI SSE 规范的补全(规范要求最后一个 choice chunk 必须携带非空 finish_reason)。

线上抓包的失败形态:

FRAME 1: {"choices":[{"delta":{},"finish_reason":null,...}]}
FRAME 2: {"choices":[{"delta":{},"finish_reason":null,...}]}
FRAME 3: data: [DONE]          <- 全程未出现 finish_reason

Node worker 老路径(worker/src/sse.mjs)本来已有对应防线(upstream_empty 结构化错误、upstream_stream_incomplete、终态 reason 合成),但 v0.6.17 的进程内重构(#276)把 Go 直连路径上的这些守卫丢掉了,而生产环境 app.go 无条件 SetDirect,默认走的就是这条无守卫路径。

修复内容

internal/providers/qoder/direct.go:

  1. 转发前守卫 — ChatStream 在向客户端写出第一个字节之前窥探上游 SSE 流。一旦出现有效帧(content / reasoning / tool_calls / 终态 finish_reason / usage / error),缓冲字节按序回放、后续照常实时转发,不损失流式体验。若上游流只发心跳就关闭,返回 providers.Error{Kind: unavailable, Code: "upstream_empty", Status: 502} —— 此时还没有任何字节发给客户端,executor 可以正常分类、冷却该账号并切换到其他账号重试。
  2. 终态守卫 — writeOpenAIChunks 现在跟踪已转发流是否出现过终态 finish_reason。若上游已输出真实内容但在终态 chunk 前被切断(字节已上线、无法重试),合成终态 chunk(有 tool_calls 则 tool_calls,否则 stop),严格客户端可以挽救部分答案而不是硬失败。若有空流漏过前置守卫,则发出 error 帧(data: {"error": ...}),网关已有的错误分类链路会把它记录为失败而不是成功。
  3. 非流式守卫 — outcomeFromUpstreamSSE 对空上游响应体报 upstream_empty,不再凭空合成 finish_reason: "stop"。

测试

internal/providers/qoder/direct_test.go 新增回归测试:

  • TestDirectChatStreamRejectsEmptyUpstreamBeforeRelay — 仅心跳流 → 转发前报 upstream_empty
  • TestDirectChatStreamReplaysPeekedFrames — 窥探到的帧完整回放,不丢失
  • TestDirectChatStreamSalvagesTruncatedStream / ...TruncatedToolCalls — 终态前被切断的流合成 stop / tool_calls 终态帧
  • TestDirectChatNonStreamRejectsEmptyUpstream

go test ./...、go test -race(qoder/gateway/executor)、go vet ./...、gofmt、git diff --check 全部通过。

其他

  • 按 changelog/unreleased/ 约定添加双语文案 qoder-empty-stream-guard.md

English

Problem

Strict OpenAI-compatible clients (pi coding agent, DeepSeek Harness, anything on pi-ai) fail Qoder-model turns with Error: Stream ended without finish_reason, while the cli2api request log records those turns as success.

Root cause

Reproduced locally against v0.6.19: the Qoder upstream gateway intermittently answers 200 with an SSE stream carrying only heartbeat / empty-delta frames (no content, no tool_calls, no usage, no terminal finish_reason, no error frame), then closes cleanly. The in-process COSY client relayed that body verbatim and unconditionally appended data: [DONE], producing a valid-looking but empty, spec-invalid completion.

The Node worker path (worker/src/sse.mjs) already guarded against this (upstream_empty, upstream_stream_incomplete, terminal-reason synthesis), but the v0.6.17 in-process rewrite (#276) dropped those guards on the Go path — which is the default path in production (app.go always calls SetDirect).

Fix

  1. Pre-relay guard — ChatStream peeks the upstream body before the first client byte. Meaningful frames are replayed in order (real-time streaming preserved); a heartbeat-only stream fails with upstream_empty before any byte is relayed, so the executor can classify, cool the account down, and fail over.
  2. Terminal-chunk guard — streams cut before the terminal finish_reason (bytes already on the wire) get a synthesized stop / tool_calls terminal chunk; an empty stream that slips past the pre-relay guard emits an error frame that the gateway classifies and logs as a failure.
  3. Non-stream guard — empty non-stream bodies return upstream_empty instead of fabricating finish_reason: "stop".

Tests

Five new regression tests; go test ./..., -race, go vet, gofmt, git diff --check all pass. Bilingual changelog fragment included per convention.

The in-process COSY client relayed Qoder's heartbeat-only 200 streams
verbatim and always appended [DONE], so strict OpenAI-compatible
clients (pi, DSH) failed the turn with 'Stream ended without
finish_reason' while the request log recorded success. The Node worker
path already guarded against this (upstream_empty); the v0.6.17
in-process rewrite dropped those guards.

- ChatStream peeks the upstream body before the first client byte and
  fails over via providers.Error when the stream carries no content,
  finish_reason, usage, or error frame
- writeOpenAIChunks synthesizes the terminal finish_reason (stop /
  tool_calls) when a relayed stream was cut before the terminal chunk,
  and emits a classified error frame when an empty stream slips past
  the pre-relay guard
- ChatNonStream rejects empty upstream bodies instead of fabricating
  stop
@caigee-cmd
caigee-cmd merged commit 10c7515 into caigee-cmd:main Oct 9, 2026
6 checks passed
caigee-cmd pushed a commit that referenced this pull request Oct 9, 2026
A heartbeat-only upstream stream (upstream_empty, introduced as a
pre-relay guard in #286) says nothing about the account's health:
credential, quota, and rate-limit state are all fine, and the same
account may succeed on the very next request. Classifying it as a
generic unavailable failure applies the default 15s cooldown with
exponential backoff on repeats, so whenever the upstream empty-stream
rate spikes the pool starves: every account ends up cooled down and
requests queue behind unrelated timeouts. Observed live while the
upstream empty-stream rate was around 40-60%.

Special-case Code=upstream_empty in ClassifyError: keep Failover=true
(another account serves the turn) but set Cooldown=0 so the account
stays in the rotation. Generic unavailable errors keep their cooldown.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants