Skip to content

feat(chat): requests waiting in the queue survive an isolate reset - #2922

Open
mattzcarey wants to merge 8 commits into
refactor/turn-core-repairfrom
refactor/turn-core-durable-queue
Open

mattzcarey wants to merge 8 commits into
refactor/turn-core-repairfrom
refactor/turn-core-durable-queue

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

A client request that was waiting in the turn queue when the isolate reset now gets its own reply, after the turn the reset cut off. Each accepted request writes a small admission record before it stores its user message. The record goes when the request's turn has a fiber, or when the request ends without one. After a reset, once no interrupted turn waits for recovery, the records left are queued again in order, each under its own request id. Pending Think submissions wait for that recovery the same way.

The turn queue lives in memory and only the running turn has a fiber, so a deploy, eviction or crash used to lose every request that was waiting. Its user message stayed in the transcript with no reply of its own (U1, U2, A1 where it should end U1, U2, A1, A2), and its tab never got a terminal frame. Think had the same bug for one running and one pending submission: both ended completed with one reply between them.

What an app can observe:

  • After a reset, a request that was waiting runs as its own turn under its own request id. A tab that reconnects gets its stream, or its terminal frame.
  • A stop by the request id, before or after the reset, ends it aborted without calling the model. A clear deletes every record. Under latest, merge and debounce, a request a later one superseded stays superseded.
  • A waiting request's user message no longer counts as the conversation moving on when a recovery checks the leaf. So the recovered turn still continues or retries, where it used to skip.
  • A recovered turn that is retried (it had streamed nothing) still builds its request from the stored transcript. So it sees the questions waiting behind it, as a live turn does when they arrive before its model call.
  • Replays start after the app's onStart.
  • A replay ends skipped when its message is gone, or when a user message that nothing is waiting on follows it (for example after a rollback). A record older than an hour is dropped.
  • Think: a submission pending at the reset runs after the recovery of the running one. One accepted after the reset runs as before.
  • A new table, cf_agents_chat_admissions, costs three statements (two rows written) per client request. An older release never reads it.

The rule was model-checked first (ChatAdmission.tla, with a reset at every step and a broken variant for each guard). The ai-chat and Think tests reset the isolate for real with ctx.abort() and failed before this change. The core has unit tests on SQLite, and the agents, ai-chat and Think suites ran. On a deployed Worker (tc-queue), 100 of 100 rooms ended U1, U2, A1, A2 with the second tab getting its reply: 40 reset with ctx.abort() and 60 with a real wrangler deploy under code_update_strategy: immediate, half AIChatAgent and half Think. 16 of 16 submission pairs ended in order. Upgrade from the base release and rollback to it both behave like the base, with no errors.


Devin Review

@changeset-bot

changeset-bot Bot commented Nov 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 0c3797b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
agents Patch
@cloudflare/ai-chat Patch
@cloudflare/think Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@mattzcarey

Copy link
Copy Markdown
Member Author

Replaces #2918, which GitHub closed as merged when a branch above it in the stack was rebased to contain its commits. Nothing was merged to main; this is the same branch and the same commits.

@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
@pkg-pr-new

pkg-pr-new Bot commented Nov 20, 2026

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2922

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2922

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2922

hono-agents

npm i https://pkg.pr.new/hono-agents@2922

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2922

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2922

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2922

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2922

commit: 0c3797b

@agent-think

agent-think Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

🟡 agents import sizes: 1 entry point grew

Entry point Exports Largest gzip change Size now
🟡 agents/chat 2 resized, 5 new up to +967 B (+9.42%) 11 KiB
Changed exports (7)
Import Gzip change Size now
🟡 agents/chat#ChatTurns +967 B (+9.42%) 11 KiB
🟡 agents/chat#ChatRecoveryEngine +137 B (+1.41%) 9.7 KiB
🆕 agents/chat#ChatAdmissions — 3.2 KiB
🆕 agents/chat#createChatAdmissionStore — 3.1 KiB
🆕 agents/chat#chatRecoveryNames — 3 KiB
🆕 agents/chat#CHAT_ADMISSIONS_TABLE — 2.6 KiB
🆕 agents/chat#CHAT_ADMISSION_TTL_MS — 2.6 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared 7314bb0d → 0c3797b6 · workflow run · reported by agent-think[bot]

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 potential issues.

Devin Review

Comment thread packages/ai-chat/src/index.ts Resolved
Comment thread packages/ai-chat/src/index.ts Resolved
Comment thread packages/ai-chat/src/index.ts Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant