Skip to content

refactor(chat): one recovery attempt sequence for AIChatAgent and Think - #2873

Open
mattzcarey wants to merge 18 commits into
refactor/chat-turn-corefrom
refactor/turn-core-recovery
Open

mattzcarey wants to merge 18 commits into
refactor/chat-turn-corefrom
refactor/turn-core-recovery

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

AIChatAgent and Think now run chat recovery through the shared ChatRecoveryEngine. It owns three sequences both classes used to write out themselves: a recovery attempt (recovery-attempt.ts), the routing of a live stall or transient error into recovery (recovery-route.ts), and the dispatch of a recovery callback. Each host binds its turn machinery once in the adapter and passes hooks for what differs, which is its leaf check, how the turn runs, how a partial reply is stored, and Think's submission and messenger steps.

The two copies had drifted, and an order bug in one of them (the stop check sat after the timeout branch) was only found in review. Moving them into one place changes what an app can observe in these cases:

  • A client answers a tool call while a recovery is scheduled to continue the same reply. The answer's turn continues it and the incident ends skipped with continued_by_interaction. Before, both turns called the model.
  • A recovery attempt throws in AIChatAgent. It now ends with onExhausted (recovery_error) and the terminal message, as Think did. Before, the incident stayed scheduled and the tab showed "recovering".
  • The user stops an attempt and it then throws, in Think. The incident records the stop. Before, it gave up with recovery_error.
  • A stop by the continuation's own request id ends the incident skipped with user_cancelled. Before, it was failed.
  • A recovered turn gives up mid-turn in AIChatAgent. The incident stays exhausted. Before, it was then marked failed.
  • onChatRecovery declines a live interruption (continue: false) in Think. The turn ends as the stream error it hit, so the client, onChatResponse and a chat() caller's onError all get that error. Before, Think sent a generic "declined" message and reported a clean interruption.
  • An error routed into recovery waits as long as the provider's retry-after or retry-after-ms asks, up to 60 seconds. Think now does this for any error classified transient. For AIChatAgent it is nominal, because a provider 429 arrives as an SSE error chunk and never reaches recovery.
  • A turn is interrupted while another turn's recovery waits for the chat to settle. It opens its own incident. Before, it was filed as a later attempt of the waiting recovery.
  • A recovery attempt hits a platform transient. Its incident stays scheduled, with deferredError, until the same attempt runs again, so a stop or a client's answer in between still reaches it. Before, it read failed for that window and emitted chat:recovery:failed.
  • A stop by the original request id, the incident's current id or the recovered turn's own id ends the incident skipped with user_cancelled. A recovered turn that returns aborted ends the same way.
  • A turn declined inside a recovery attempt emits one chat:recovery:skipped event. Before, a failed followed it.
  • onChatResponse, the turns it starts and a continuation timer armed during a recovered turn no longer run as part of the recovery attempt.
  • An isolate reset cuts off a recovery attempt's turn. The reply is continued once. Before, the callback that was delivered again and the interrupted turn's own recovery both continued it, and the same text was stored twice. Incidents gain scheduleCount and startedDelivery, and recovery payloads gain delivery.
  • A tab reconnects after a reset before recovery has run, in Think. The turn is still continued. Before, the tab's resume closed the stream, recovery read it as finished, the incident ended skipped with stream_terminal, and the turn's submission was marked completed.
  • Retry-After is also read from inside the AI SDK's RetryError, so it survives the SDK's own retries.
  • Recovery gives up during a live stall in AIChatAgent. A tab that connects afterwards still gets the terminal message. Before, the stalled turn ending cleared it.

To review, start with the file comment in recovery-attempt.ts, which lists the attempt's steps in order, then runRecoveredTurn below it. recovery-route.ts is the second sequence. In the hosts, _chatRecoveryContinueDetached and _routeStreamInterruption show what each one still supplies. The persisted incident gains one optional field, scheduled.

The order of the checks was modelled in TLA+ before the code changed, with a variant per check that must break a named rule. Tests ran in agents, ai-chat and think. recovery-attempt.test.ts and recovery-route.test.ts cover the two sequences, and each change in the list has a host test that fails without it, except the AIChatAgent half of retry-after, which is covered only for a platform-transient error.

@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f2bbc4b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@cloudflare/ai-chat Patch
@cloudflare/think Patch
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

@pkg-pr-new

pkg-pr-new Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2873

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2873

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2873

hono-agents

npm i https://pkg.pr.new/hono-agents@2873

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2873

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2873

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2873

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2873

commit: f2bbc4b

@agent-think

agent-think Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🔴 agents import sizes: 1 entry point over threshold

Entry point Exports Largest gzip change Size now
🔴 agents/chat 2 resized, 9 new up to +3.7 KiB (+65.53%) 9.4 KiB
Changed exports (11)
Import Gzip change Size now
🔴 agents/chat#ChatRecoveryEngine +3.7 KiB (+65.53%) 9.4 KiB
🟡 agents/chat#retryAfterSeconds +124 B (+4.23%) 3 KiB
🆕 agents/chat#runRecoveredTurn — 4 KiB
🆕 agents/chat#routeStreamInterruption — 3.9 KiB
🆕 agents/chat#handleRecoveryAttemptError — 3.1 KiB
🆕 agents/chat#chatRecoveryStorageAdapter — 2.9 KiB
🆕 agents/chat#scheduleRecoveredTurn — 2.8 KiB
🆕 agents/chat#enqueueChatRecoveryTask — 2.8 KiB
🆕 agents/chat#chatRecoveryProgressReader — 2.7 KiB
🆕 agents/chat#cutOffReplyToSettle — 2.7 KiB
🆕 agents/chat#CHAT_RECOVERY_CONTINUED_REASON — 2.6 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared da59cf74 → f2bbc4bc · workflow run · reported by agent-think[bot]

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 1 new potential issue.

Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

…ontinued; a stop by any id ends the incident as stopped
… attempt plumbing and incident bookkeeping for both hosts
… its reply, and knows its own turn by async context
…try-After is honored in both hosts, and the engine dispatches recovery callbacks
…, and a continuation supersedes only the recovery of the reply it continues
…t tracks reaches the attempt, and only the attempt's own turn runs in its scope
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-recovery branch from a1d92e0 to f2bbc4b Compare November 19, 2026 14:44
@mattzcarey
mattzcarey removed this pull request from stack #2869 November 20, 2026 12:09
@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant