Repository navigation
feat(chat): requests waiting in the queue survive an isolate reset - #2922
mattzcarey wants to merge 8 commits into
Conversation
… request's message only when it is the leaf
… and are not held by a run recovery already settled
🦋 Changeset detectedLatest commit: 0c3797b The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Replaces #2918, which GitHub closed as merged when a branch above it in the stack was rebased to contain its commits. Nothing was merged to |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
🟡 agents import sizes: 1 entry point grew
Changed exports (7)
How this worksEach runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement. Compared |
A client request that was waiting in the turn queue when the isolate reset now gets its own reply, after the turn the reset cut off. Each accepted request writes a small admission record before it stores its user message. The record goes when the request's turn has a fiber, or when the request ends without one. After a reset, once no interrupted turn waits for recovery, the records left are queued again in order, each under its own request id. Pending
Thinksubmissions wait for that recovery the same way.The turn queue lives in memory and only the running turn has a fiber, so a deploy, eviction or crash used to lose every request that was waiting. Its user message stayed in the transcript with no reply of its own (
U1, U2, A1where it should endU1, U2, A1, A2), and its tab never got a terminal frame.Thinkhad the same bug for one running and one pending submission: both endedcompletedwith one reply between them.What an app can observe:
abortedwithout calling the model. A clear deletes every record. Underlatest,mergeanddebounce, a request a later one superseded stays superseded.onStart.skippedwhen its message is gone, or when a user message that nothing is waiting on follows it (for example after a rollback). A record older than an hour is dropped.Think: a submission pending at the reset runs after the recovery of the running one. One accepted after the reset runs as before.cf_agents_chat_admissions, costs three statements (two rows written) per client request. An older release never reads it.The rule was model-checked first (
ChatAdmission.tla, with a reset at every step and a broken variant for each guard). The ai-chat and Think tests reset the isolate for real withctx.abort()and failed before this change. The core has unit tests on SQLite, and the agents, ai-chat and Think suites ran. On a deployed Worker (tc-queue), 100 of 100 rooms endedU1, U2, A1, A2with the second tab getting its reply: 40 reset withctx.abort()and 60 with a realwrangler deployundercode_update_strategy: immediate, halfAIChatAgentand halfThink. 16 of 16 submission pairs ended in order. Upgrade from the base release and rollback to it both behave like the base, with no errors.