Repository navigation
refactor(chat): one path for turns the server starts - #2870
Open
mattzcarey wants to merge 4 commits into
Open
mattzcarey wants to merge 4 commits into
mattzcarey wants to merge 4 commits into
Conversation
🦋 Changeset detectedLatest commit: d60ba14 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
Contributor
🟡 agents import sizes: 1 entry point grew
Changed exports (7)
How this worksEach runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement. Compared |
mattzcarey
force-pushed
the
refactor/turn-core-programmatic
branch
from
November 17, 2026 15:38
1405e4c to
5d18758
Compare
mattzcarey
changed the base branch from
refactor/chat-turn-core
to
refactor/turn-core-recovery
November 17, 2026 15:38
This was referenced Nov 17, 2026
mattzcarey
force-pushed
the
refactor/turn-core-programmatic
branch
from
November 17, 2026 19:35
5d18758 to
4618db1
Compare
mattzcarey
force-pushed
the
refactor/turn-core-programmatic
branch
from
November 18, 2026 17:18
4618db1 to
fd4eab1
Compare
saveMessages, continueLastTurn and the retry turn in AIChatAgent and Think go through ChatTurns.runServerTurn. Each host has one function that builds such a turn, Think has one for a turn's model call and reply, and both run a turn's durable fiber through ChatTurnFibers.
A continuation takes a reply over from its scheduled recovery once its model call returned a reply, not at fiber start. A reply that finished and was stored is completed, also when the chat is cleared while onChatResponse runs. A context pinned on a turn is never written to the stored request context: the recovery engine hands the recovered context to the turn, and a live interruption reads the turn's own.
…onChatResponse, and tests for surviving mutants An aborted result carries reason stopped or interrupted (cut off and handed to recovery). Think._runModelTurn fires onChatResponse once for an overflow with no retry left and for a turn stopped at one. server-turn and turn-fiber get tests for the rules mutation testing found unpinned.
…e, and one onChatResponse in the ai-chat test agent
mattzcarey
force-pushed
the
refactor/turn-core-programmatic
branch
from
November 19, 2026 15:18
fd4eab1 to
d60ba14
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second PR of the turn core stack, on top of #2873. Turns the server starts itself (
saveMessages,continueLastTurn, the recovery retry, an agent-tool run's turn) now run through one function in the core,ChatTurns.runServerTurn(packages/agents/src/chat/server-turn.ts), the counterpart ofrunClientRequest. Each host builds such a turn in one place (_runServerTurn),Thinkhas one function for a turn's model call and reply (_runModelTurn), and both hosts run a turn's durable fiber throughChatTurnFibers(turn-fiber.ts).AIChatAgentandThinkeach repeated the same steps around these turns and had the same gaps. What an app can observe:signal, orabortAllRequests()while it waits in the queue) stores nothing, never calls the model and returnsaborted.onChatResponsedoes not fire, and the record of an earlier failure stays. Before, the stop was lost in the queue or the model was called with an aborted signal. This also closes the gap refactor(chat): one recovery attempt sequence for AIChatAgent and Think #2873 left open: a recovery turn stopped or superseded while it waits in the queue no longer calls the model.onChatResponse.skipped, also when it cuts a reply off (it gaveaborted). A stop that lands after a reply finished leaves the resultcompleted.abortAllRequests()duringonChatMessagestops the reply inAIChatAgentfor every kind of turn, and stops a queued reaction to a detached notification.completed, also when the chat is cleared whileonChatResponseruns. It was reportedskipped.onChatMessagethrows or returns nothing) leaves the recovery in place. refactor(chat): one recovery attempt sequence for AIChatAgent and Think #2873 took the reply when the turn's fiber started.abortedresult ofsaveMessagesorcontinueLastTurncarries areason:"stopped"for a stop,"interrupted"for a reply that was cut off and handed to chat recovery, which may still finish it. Before, both read as a stop.Think, a turn whose context overflow cannot be recovered firesonChatResponseonce witherror, on every path, and a turn stopped at an overflow fires it once withaborted. Continuations,continueLastTurnand recovered turns fired it onmainand stopped doing so when they gained overflow recovery; client requests andsaveMessagesnever did.AIChatAgent.continueLastTurnalways handsonErrorthe erroronChatMessagethrew. WhenonErroritself throws, the second call used to get that thrown error.Thinkrecovers a context overflow (contextOverflow.reactive) on continued and retried turns too. An overflow with no retry left always ends its stream with an error frame and a terminal record. A turn stopped at the overflow or during compaction reportsaborted, with noonChatErrorand no second model call.Think'schat:turn:startandchat:turn:finishcarry the queue generation and the status the caller gets for these turns, which also openpersist_chat_resultandcompact_chat_historyspans.Tests ran:
agentschat,ai-chatworkers, the fullthinksuite,pnpm run check. Each change above has a test in each host it applies to that fails without it. The TLA+ models (kept outside the repo) hold, #2873's included, with a broken variant for each guard.