Repository navigation
refactor(chat): one agent-tool child run for AIChatAgent and Think, and Think.chat() on the turn core - #2874
Open
mattzcarey wants to merge 6 commits into
Conversation
🦋 Changeset detectedLatest commit: 8117abc The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
Contributor
🟡 agents import sizes: 1 entry point grew
Changed exports (9)
How this worksEach runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement. Compared |
mattzcarey
added a commit
that referenced
this pull request
Nov 12, 2026
… of an open run Review round 1 on #2874. - A cancel of a run chat recovery is re-running cancels the request ids on the run row (the recovered turn, a recovery attempt registered under the first id, a scheduled recovery) and nothing else. The sweeps are gone: AIChatAgent's abortAllRequests, Think's recovered submissions, ChatTurns.cancelRecoveryAttempts. - The recovery entry points skip a turn whose run was cancelled, right after their own cancelled check, and record the incident as user_cancelled. continueLastTurn and the retry turn no longer do it. - A run is open while its status is starting or running. AgentToolRuns decides whether a row may be written; the stores write. - AIChatAgent's store no longer turns a failed query into 'no run'. - Docs: what a chat() callback gets for a cancel and for a clear.
mattzcarey
added a commit
that referenced
this pull request
Nov 17, 2026
…as one, no dedicated-agent assumption Cross-review of #2874. - A run whose turn hands back aborted while chat recovery works on it was interrupted, not cancelled. It stays open and is settled when recovery ends. It used to be written aborted, and recovery was then skipped as a user cancel. - A cancel writes its own marker (cancelled_at). Recovery reads that, not the status. The parent's signal goes through cancel(), so it is recorded at once with its reason, whatever the stream said. - rebind binds only the run whose row names the recovered turn's root request id. reconcileStale looks only at incidents and the active stream of the run's own request ids, and after a restart counts only what follows the run's start as the run's (start_message_id). - AIChatAgent hands the run's input to its turn as the body. It no longer writes the stored request context when the run starts or restores it when the run ends. - chat(): a clear after the caller's own cancel is not reported, and a request id the registry cannot key is rejected. - A tail that is closed while it reads its backlog finishes the drain first.
mattzcarey
force-pushed
the
refactor/turn-core-rpc-agent-tool
branch
from
November 17, 2026 17:36
315faf0 to
5432e26
Compare
mattzcarey
changed the base branch from
refactor/chat-turn-core
to
refactor/turn-core-continuation-stop
November 17, 2026 17:36
mattzcarey
force-pushed
the
refactor/turn-core-rpc-agent-tool
branch
from
November 17, 2026 22:36
5432e26 to
668afdf
Compare
mattzcarey
force-pushed
the
refactor/turn-core-rpc-agent-tool
branch
from
November 18, 2026 19:10
668afdf to
0bd307f
Compare
…ink.chat() on runServerTurn AgentToolRuns (agents/chat) is the one owner of what an agent does when a parent runs it as a tool, for AIChatAgent and Think: the run table (one SQL store, the hosts name the tables), the run's life, inspect, chunks, tail, and progress. The hosts supply how the run's turn is queued, what its output is, and how a frame is broadcast. Think.chat() is a server-started turn: it runs on ChatTurns.runServerTurn and Think._runModelTurn with the RPC callback as its sink. Its own admission, overflow loop and model call are gone. What changes for a child run: - its terminal status is written once - a cancel is recorded as one, with its reason, and stops only that run's turn: through the run's controller, or by the request ids on the run row when chat recovery is re-running the turn - a run cancelled while it waits in the queue stores nothing and never calls the model, and chat recovery does not re-run its turn - a run whose reply is cut off while recovery can finish it stays open and takes the recovered turn's status - only the turn the run's row names binds to it, holds it open or settles it - a chat reset ends it as error What changes for chat(): - a request cancelled before its message is stored stores nothing and never calls the model - a chat cleared under it calls onError and stores nothing; a caller that had cancelled is told nothing more - it gets the overflow handling every other turn has
…nd runAgentTool waits for a child whose isolate reset A stop aborts the turn's signal, and whatever waits on that signal then fails with the abort's reason: a provider call rejects with the caller's string or Error, not with an AbortError. The reply decided 'stopped or failed' from the thrown value, so a stop that carried a reason ended as an error whenever the failure beat the next chunk: onError to an RPC caller, an error frame to every tab, the error hook. The signal decides now (isTurnStopped, turn-stop.ts), in runServerTurn, runClientRequest, the turn fiber, both Think reply loops and AIChatAgent's error hook wrapper. The stall watchdog aborts with a ChatStreamStalledError so its abort is not taken for a stop. An awaited runAgentTool tails the child over an RPC stream. A reset of the child's isolate breaks that stream, and the parent returned error with the stream's message while the child's chat recovery went on to complete the run. The parent now re-resolves the child and, when the run's row is still open, waits for it the way a recovered parent does, within the same budgets. Giving up is interrupted, not error.
…ry stop path, and the parent's wait on a child is bounded A run left open for chat recovery had nothing to settle it when that recovery was cancelled: its tails stayed open, the parent's runAgentTool never returned, and a later inspect reconciled the stopped run as completed from the cut-off partial. Every stop now goes through ChatTurns.cancel / stopAll, which tell the host (ChatTurnsHost.stopped) unless the abort is the runtime's own. The host cancels the agent-tool run the request names (AgentToolRuns.stopped) and a recovery that is scheduled for it. So a client's cancel, cancelChat, abortRequest, abortAllRequests and a caller's signal do the same. abortRequest used to leave a scheduled recovery in place. Also: - a stop that lands after the stall watchdog aborted the turn counts as a stop (ChatTurns.isStopped) - a run whose recovery was cancelled reconciles as aborted, never as completed - an awaited runAgentTool inspects the child once per no-progress window, so a run with nothing left to finish it is settled and the call returns - chunks forwarded after a re-attach are numbered past the ones already sent; the count survives a broken tail - a row from before root_request_id takes the root recovery names the turn by, not the recovered turn's id - a cancel that lands while the lifecycle reads recovery state leaves nothing in memory, and a tail of a finished run leaves no closer - a run is not anchored on an empty reply its own turn removes
…ned, and remove what it found dead Stryker on agent-tool-runs.ts and agent-tool-run-store.ts left 193 mutants alive at the stack tip. Now 3 survive in agent-tool-runs.ts (a per-run timestamp that is only ever freed, and a log line) and none in the store. Tests added for: the state of a turn's recovery (attempting, exhausted, failed, root id only), an error turn with no text, the tail's error paths, terminal-only runs, the request cache, what is kept in memory, the anchor of a run's part of the transcript, and each store method as the first call on an older table. Removed as dead or duplicate: the idle guard in intercept (the function it calls has it), closeAfterDrain and the tail's closed flag, the seeded counter cleanup, the stale re-check in the reconcile, the controller check in reconcileOwn, null guards on values that cannot be null. Fixes found on the way: - a run's part of the transcript starts when its turn starts, not when it was queued; a run whose turn was still queued when the isolate went away is an error that says so, whatever another turn stored since (turn_started_at) - an awaited runAgentTool whose start call fails asks the child: it waits for a run the child did start, and returns an error result, with its row settled, when the child has none
… and the tests follow the layers below - AgentToolRuns leaves a run open when its turn hands back aborted with reason interrupted (runServerTurn decides it), not by reading recovery state: no storage read, and no window for a cancel to land in - chat()'s reply path stores through _storeLiveReply and keeps its reset guard; an overflow retry hands the partial on - tests: a stopped server turn carries reason stopped; a run's input is never the stored request context; the follow-up test uses the agent's existing onChatResponse switch
mattzcarey
force-pushed
the
refactor/turn-core-rpc-agent-tool
branch
from
November 19, 2026 20:45
0bd307f to
df6d226
Compare
…cks the reason alone
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AIChatAgentandThinknow share one implementation of an agent-tool child run:AgentToolRuns(packages/agents/src/chat/agent-tool-runs.ts) with one SQL store (agent-tool-run-store.ts).Think.chat()becomes a server-started turn likesaveMessages: it runs onChatTurns.runServerTurnandThink._runModelTurnfrom the PRs below, with the RPC callback as the reply's sink.AIChatAgentis about 850 lines shorter andThinkabout 750.The two classes wrote the child run twice and disagreed at the edges, and
chat()was the last turn with its own admission and overflow loop. What an app can observe, for a child run:AIChatAgenta later outcome could replaceaborted.AIChatAgentaborted every request on the agent,Thinkevery recovered submission.skipped, reasonuser_cancelled).runningand takes the recovered turn's status. Both classes reported itaborted.error.AIChatAgentrecordedabortedfor a running one.Thinkrun's stored chunks are readable while it runs, and a tail of a run an earlier isolate left open settles it and closes. A tail closed while it reads its backlog delivers the backlog first.inspectAgentToolRunreportsoutputandsummaryfor a completed run only.ThinkreportedgetAgentToolOutput()for any status.completed, stored as its string.AIChatAgentfailed the run.saveMessages, so a subclass that overridessaveMessagesdoes not see it.Think.startAgentToolRuntakes an optionalsignal, asAIChatAgentdoes (additive). It works for a caller in the same isolate.root_request_id,cancelled_at,start_message_id,turn_started_at, plusinput_jsonand an index forThink,stream_idforAIChatAgent. A run row the previous release wrote keeps the request id recovery names the turn by as its root on its first recovery after the upgrade.AIChatAgent's run table throws. It used to read as "no such run".For
Think.chat():onErrorand stores nothing. It used to resolve without a terminal call. A caller that had already cancelled is told nothing more.chat:turn:start,chat:turn:finishand thechat_turnspan carrygenerationfortrigger: "rpc"turns.Two fixes for bugs that are on
maintoo, found by an end-to-end probe of the stack:abortedwhatever reason the abort carried. Withabort("reason")orabort(new Error(...))the provider call fails with that reason, not with anAbortError, and the reply decided "stopped or failed" from the thrown value.Think.chat()ended such a cancel as an error about half the time:onError(reason), an error frame to every tab, noonChatResponse. InAIChatAgent, anonChatMessagethat failed on itsabortSignalmadesaveMessagesreject, calledonError, and left a client's cancelled request without a terminal frame. The turn's signal decides now, in one function (isTurnStopped,packages/agents/src/chat/turn-stop.ts), forchat(),saveMessages,continueLastTurn, a client request,abortRequest(id, reason)and an agent-tool cancel, in both classes. The stall watchdog's abort now carries aChatStreamStalledErroras its reason (same message), which is how it is told apart from a stop.runAgentToolno longer returnserrorwhen the child's isolate resets mid-run. The error was the parent's tail of the child, an RPC stream, rejecting with "ReadableStream received over RPC disconnected prematurely."; the child's chat recovery then completed the run. The parent now reconnects to the child and, while the run is still open, waits for it within the existing re-attach budgets and returns the run's own status. If the budget runs out it returnsinterrupted(reason: "no-progress"or"window-exceeded"). A child that cannot recover the turn, or cannot be reached, is stillerror.docs/agents/agent-tools.mdlists the cases.Fixes from review, on top of the above:
aborted, with the reason, and closes the parent's tail. Before, a run left open for chat recovery whose recovery was then cancelled stayedrunning, the parent'srunAgentToolnever returned, and a later inspect reported the stopped runcompletedfrom its cut-off reply.cancelChat,abortRequest,abortAllRequestsand a caller'ssignalall go throughChatTurns.cancelorstopAll, which tell the host once (ChatTurnsHost.stopped).abortRequestand a caller'ssignalnow also cancel a recovery of the request that is scheduled and not yet running. They used to leave it to run.Think, a stop that lands right after the stall watchdog aborted the turn is a stop. The stalled reply was recovered anyway.runAgentToolinspects the child once peragentToolReattachNoProgressTimeoutMs, so a run with nothing left to finish it is settled and the call returns.runAgentToolno longer throws when the call that starts the child fails. If the child did start the run (its isolate went away under the call), the parent waits for it as above. If the child has no such run, the result iserrorwith that failure, and the parent's row is settled. It used to staystarting.errorwith a text that says so, whatever another turn stored since. New columnturn_started_aton both run tables.abortedwithreason: "interrupted"(refactor(chat): one path for turns the server starts #2870), not by reading recovery state.Tests ran:
agentschat,ai-chatworkers, the fullthinksuite,pnpm run check. Two TLA+ models for this PR (AgentToolRun,ChatRpc, kept outside the repo, each guard with a broken variant) pass together with the models of the PRs below. Mutation testing (Stryker) ofagent-tool-runs.tsandagent-tool-run-store.tsagainst the unit tests leaves 3 of 698 mutants alive, none of which changes what a run reports. Each change to what a run or a request stores, calls or reports has a test that fails without it, except the eventgeneration, thesaveMessagesoverride, andchat()'s stop during compaction (tested for the other turns in #2870).