Repository navigation
[STALE: REBASE FROM NEW FORK] fix(anthropic): keep pooled sessions on one account across processes - #3
[STALE: REBASE FROM NEW FORK] fix(anthropic): keep pooled sessions on one account across processes#3vadymhimself wants to merge 3 commits into
Conversation
Under round-robin, an unbound keyed session now takes the rendezvous-hash winner (highest sha256(key \0 account)) over the eligible accounts, so a fresh process routes it to the same account and its prompt cache survives. A 429 unbinds only the session that hit it, and that session fails over to its hash runner-up instead of unbinding every session on the account. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
✅ Deterministic PR hygiene checks passed. |
⏳ DRAFT
What to do
Its title has been prefixed with |
A hash-seat pick reported reason "affinity", the same as a live in-process binding, so the [ar-acct] measurement could not tell a fresh-process hash placement from a held binding. It now reports "hash"; like "affinity" it does not advance the round-robin ring at commit. The header no longer implies a session returns to a recovered seat inside a warm process: it stays bound to its runner-up there. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
…ites (lidge-jun#8590) * devin: opt-in Cognition blocklist rewrite on history data surfaces The blocklist sanitizer only touched instruction surfaces (tool descriptions, the #2 system prompt); data fields stayed byte-exact, so one quoted trigger phrase anywhere in a long thread history made every later request fail with upstream permission_denied. New OPENCODEX_DEVIN_SANITIZE_DATA_SURFACES=1 opt-in extends the same rewrites to message text (#3), tool-call arguments (#6.3), image captions (#10.3) and replayed thinking (#11). Identifiers (tool name, tool_call_id) and opaque signature blobs stay byte-exact. Off by default: rewriting literal history changes what the model sees, so refusing stays the default failure. The permission_denied hint now names the opt-in. Tests pin both modes: data literal by default, rewritten when enabled. * devin: add Codex CLI boilerplate phrases to Cognition blocklist rewrites Two more live-isolated triggers in the Codex CLI developer boilerplate: the identity sentence "Codex refers to the open-source agentic coding interface" and the "Don not output ANSI escape codes directly — the CLI renderer applies them" bullet. Both are rewritten meaning-preservingly like the existing entries. Also adds the dump/bisect probe scripts used to isolate the triggers. * fix(devin): drop stale signatures after thinking sanitization Keep reasoning signatures only when the serialized thinking is unchanged. Preserve signature-only replay and default-off sanitization, and add wire, authentication and continuity regressions with matching adapter docs. * chore(devin): keep live credential probes out of the runtime PR
|
This PR is based on the superseded 2.65 gateway line. Rebuild the change from |
|
Trial evidence: before the merge the build ran 30.5 h on the laptop gateway and 24 h on the personal Mac Mini gateway, both with For new work:
Before opening a PR, use pinned Bun 1.4.4 with an isolated
Hosted-only behavior needs For this PR: carried as deterministic rendezvous-hash session placement ( 🤖 Generated with Claude Code |
Base branch
fix/free-downgrade-fork: its tip is97c47773c chore: release 2.65.0-vadymhimself.13(tagv2.65.0-vadymhimself.13), the build the live and hosted gateways run.main/devare at 2.43.0 and do not contain the .13 release, which is why #1/#2 were marked [WRONG BRANCH].Two prompt-cache holes in
src/oauth/anthropic-routing.tsHole 1: affinity is process-local.
sessionAffinityis an in-memory Map. A fresh process (a new Lambda environment) has an empty Map, so a running session is round-robined to a possibly different account and rewrites its whole prompt cache (50k+ tokens).Fix: under
strategy: "round-robin", a keyed session with no live affinity entry takes the account with the highestsha256(sessionKey + "\0" + accountId)amonggetEligibleAnthropicAccounts(now)(healthy, not cooled, usable),reason: "affinity". Every process computes the same answer; the Map remains the in-process cache. A hash placement reportsreason: "hash"(not"affinity"), so the[ar-acct]line tells a fresh-process hash seat from a held binding; likeaffinityit does not advance the round-robin ring. A cooled account moves only its own sessions, each to its runner-up. A fresh process returns them to the recovered seat; a warm process keeps them bound to the runner-up. Keyless sessions,quotaandfill-firstare unchanged (their picks are policy, not arbitrary; scoping to round-robin kept all their existing tests green).Hole 2: a 429 unbinds every session on the account.
rotateAnthropicAccountOn429calledclearAnthropicSessionAffinityForAccount(failedAccountId), so bystander sessions lost their binding and were re-placed like new sessions.Fix: delete only the hitting session's entry. Its failover target is its hash runner-up (round-robin pool only), falling back to the stock
pickAlternateAnthropicAccount. Bystanders move on their own next call because the account is cooled. Account removal and credential denial still clear the whole account.Ported from the closed AgentRadar PR lidge-jun#4562 (
pin-patch.mjsstring replacements), now in source.Test
New case in
tests/adapters/anthropic/anthropic-account-pool.test.ts: 60 sessions spread over 3 accounts; the same sessions map to the same accounts after a full routing-state reset (fresh process stand-in); a 429 fors0leaves the bystander on the same account bound (1 entry, stock leaves 0); the cooled hash seat fails over tos0's runner-up and the bystander moves off it. Fails on stock source.Removed/adjusted 3 round-robin tests that asserted ring/
stickyLimitplacement of keyed sessions, which the hash replaces by design (stickyLimitnow only affects keyless picks and the keyless 429 fallback).At
b60e43491, Bun 1.4.4: all 39 test files that touch Anthropic routing (tests/adapters/anthropic/*plus every file importing the router): 759 pass, 0 fail.bun x tsc --noEmitexits 0. Earlier run:bun scripts/test.tsover the 17 routing/pool/transport test files: 395 pass, 0 fail.CI failures not caused by this diff
The diff touches only
src/oauth/anthropic-routing.tsand its pool test.fix/free-downgrade-forknever ran CI before this PR, and these three fail on its .13 tip too:enforce-target: the fork gate requiresdev, which is at 2.43.0 and lacks the .13 release (see Base branch).src/adapters/anthropic.ts(2035 lines) andtests/codex-integration/codex-auth-api.test.ts(6544) are over the committed baseline. This PR does not change either file.package.jsonstill says2.65.0-vadymhimself.13, the already-tagged version. The.14bump belongs to the release commit, not to this fix.Known cost
Cooldowns are still per process (
upstreamHealthMap). On a fresh process, a session whose hash seat is spent pays one extra 429 before moving to its runner-up.Rollout
Merging this does not move hosted gateways. That needs a new build plus a separate image pin in the AgentRadar repo (
infra/environments/hosted-ocx/image.auto.tfvars) and the releaseocx-roll.🤖 Generated with Claude Code