Skip to content

refactor(chat): one agent-tool child run for AIChatAgent and Think, and Think.chat() on the turn core - #2874

Open
mattzcarey wants to merge 6 commits into
refactor/turn-core-continuation-stopfrom
refactor/turn-core-rpc-agent-tool
Open

mattzcarey wants to merge 6 commits into
refactor/turn-core-continuation-stopfrom
refactor/turn-core-rpc-agent-tool

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

AIChatAgent and Think now share one implementation of an agent-tool child run: AgentToolRuns (packages/agents/src/chat/agent-tool-runs.ts) with one SQL store (agent-tool-run-store.ts). Think.chat() becomes a server-started turn like saveMessages: it runs on ChatTurns.runServerTurn and Think._runModelTurn from the PRs below, with the RPC callback as the reply's sink. AIChatAgent is about 850 lines shorter and Think about 750.

The two classes wrote the child run twice and disagreed at the edges, and chat() was the last turn with its own admission and overflow loop. What an app can observe, for a child run:

  • Its terminal status is written once. In AIChatAgent a later outcome could replace aborted.
  • A cancel is recorded as one, with its reason, whatever the run's stream reported. It stops only that run's turn, also while chat recovery is re-running it. AIChatAgent aborted every request on the agent, Think every recovered submission.
  • A cancel of an unknown or finished run does nothing. A run cancelled while it waits in the queue stores nothing and never calls the model, and chat recovery does not re-run its turn (the incident ends skipped, reason user_cancelled).
  • A run whose reply is cut off while chat recovery can finish it (a dropped connection, a stall) stays running and takes the recovered turn's status. Both classes reported it aborted.
  • A chat reset ends a run as error. AIChatAgent recorded aborted for a running one.
  • Recovery of another turn on the same agent no longer binds to, holds open, or settles a run. Only the turn the run's row names does.
  • A Think run's stored chunks are readable while it runs, and a tail of a run an earlier isolate left open settles it and closes. A tail closed while it reads its backlog delivers the backlog first.
  • inspectAgentToolRun reports output and summary for a completed run only. Think reported getAgentToolOutput() for any status.
  • A run whose output JSON cannot hold stays completed, stored as its string. AIChatAgent failed the run.
  • The run's turn no longer goes through the public saveMessages, so a subclass that overrides saveMessages does not see it.
  • Think.startAgentToolRun takes an optional signal, as AIChatAgent does (additive). It works for a caller in the same isolate.
  • Both run tables hold the same columns: new are root_request_id, cancelled_at, start_message_id, turn_started_at, plus input_json and an index for Think, stream_id for AIChatAgent. A run row the previous release wrote keeps the request id recovery names the turn by as its root on its first recovery after the upgrade.
  • A failing query on AIChatAgent's run table throws. It used to read as "no such run".

For Think.chat():

  • A request cancelled before its message is stored (in the queue, or while its message function runs) stores nothing and never calls the model.
  • A chat cleared under the request calls onError and stores nothing. It used to resolve without a terminal call. A caller that had already cancelled is told nothing more.
  • An overflow that cannot be recovered also sends the error frame and records the terminal status, and a turn stopped while its history is compacted is not run again, as for every other turn.
  • chat:turn:start, chat:turn:finish and the chat_turn span carry generation for trigger: "rpc" turns.

Two fixes for bugs that are on main too, found by an end-to-end probe of the stack:

  • A stopped turn ends aborted whatever reason the abort carried. With abort("reason") or abort(new Error(...)) the provider call fails with that reason, not with an AbortError, and the reply decided "stopped or failed" from the thrown value. Think.chat() ended such a cancel as an error about half the time: onError(reason), an error frame to every tab, no onChatResponse. In AIChatAgent, an onChatMessage that failed on its abortSignal made saveMessages reject, called onError, and left a client's cancelled request without a terminal frame. The turn's signal decides now, in one function (isTurnStopped, packages/agents/src/chat/turn-stop.ts), for chat(), saveMessages, continueLastTurn, a client request, abortRequest(id, reason) and an agent-tool cancel, in both classes. The stall watchdog's abort now carries a ChatStreamStalledError as its reason (same message), which is how it is told apart from a stop.
  • An awaited runAgentTool no longer returns error when the child's isolate resets mid-run. The error was the parent's tail of the child, an RPC stream, rejecting with "ReadableStream received over RPC disconnected prematurely."; the child's chat recovery then completed the run. The parent now reconnects to the child and, while the run is still open, waits for it within the existing re-attach budgets and returns the run's own status. If the budget runs out it returns interrupted (reason: "no-progress" or "window-exceeded"). A child that cannot recover the turn, or cannot be reached, is still error. docs/agents/agent-tools.md lists the cases.

Fixes from review, on top of the above:

  • A stop of an agent-tool run's turn on the child settles the run aborted, with the reason, and closes the parent's tail. Before, a run left open for chat recovery whose recovery was then cancelled stayed running, the parent's runAgentTool never returned, and a later inspect reported the stopped run completed from its cut-off reply.
  • Every stop does the same: a client's cancel, cancelChat, abortRequest, abortAllRequests and a caller's signal all go through ChatTurns.cancel or stopAll, which tell the host once (ChatTurnsHost.stopped). abortRequest and a caller's signal now also cancel a recovery of the request that is scheduled and not yet running. They used to leave it to run.
  • In Think, a stop that lands right after the stall watchdog aborted the turn is a stop. The stalled reply was recovered anyway.
  • An awaited runAgentTool inspects the child once per agentToolReattachNoProgressTimeoutMs, so a run with nothing left to finish it is settled and the call returns.
  • Chunks the parent forwards after it re-attached to a child are numbered after the ones it had already sent. A recovered turn's stream is shorter than the first turn's, so the numbers were reused and clients dropped those chunks as repeats.
  • A run is not anchored on an empty assistant reply, which its own turn removes.
  • An awaited runAgentTool no longer throws when the call that starts the child fails. If the child did start the run (its isolate went away under the call), the parent waits for it as above. If the child has no such run, the result is error with that failure, and the parent's row is settled. It used to stay starting.
  • A run's own part of the transcript starts when its turn starts, not when the run was queued: a reply another turn stored while the run waited is not the run's. A run whose turn was still waiting when the isolate went away ends error with a text that says so, whatever another turn stored since. New column turn_started_at on both run tables.
  • A run is left open for recovery when its turn ends aborted with reason: "interrupted" (refactor(chat): one path for turns the server starts #2870), not by reading recovery state.

Tests ran: agents chat, ai-chat workers, the full think suite, pnpm run check. Two TLA+ models for this PR (AgentToolRun, ChatRpc, kept outside the repo, each guard with a broken variant) pass together with the models of the PRs below. Mutation testing (Stryker) of agent-tool-runs.ts and agent-tool-run-store.ts against the unit tests leaves 3 of 698 mutants alive, none of which changes what a run reports. Each change to what a run or a request stores, calls or reports has a test that fails without it, except the event generation, the saveMessages override, and chat()'s stop during compaction (tested for the other turns in #2870).

@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8117abc

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
agents Patch
@cloudflare/ai-chat Patch
@cloudflare/think Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2874

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2874

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2874

hono-agents

npm i https://pkg.pr.new/hono-agents@2874

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2874

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2874

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2874

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2874

commit: 8117abc

devin-ai-integration[bot]

This comment was marked as resolved.

@agent-think

agent-think Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🟡 agents import sizes: 1 entry point grew

Entry point Exports Largest gzip change Size now
🟡 agents/chat 3 resized, 6 new up to +190 B (+2.23%) 8.5 KiB
Changed exports (9)
Import Gzip change Size now
🟡 agents/chat#ChatTurns +190 B (+2.23%) 8.5 KiB
🟡 agents/chat#ChatTurnFibers +155 B (+4.77%) 3.3 KiB
🟡 agents/chat#runServerTurn +101 B (+3.22%) 3.2 KiB
🆕 agents/chat#AgentToolRuns — 6.3 KiB
🆕 agents/chat#createAgentToolRunStore — 4 KiB
🆕 agents/chat#agentToolRunOutcome — 2.7 KiB
🆕 agents/chat#isTurnStopped — 2.7 KiB
🆕 agents/chat#abortReasonText — 2.6 KiB
🆕 agents/chat#isAgentToolRunOpen — 2.6 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared 9dfa9c21 → 8117abc5 · workflow run · reported by agent-think[bot]

mattzcarey added a commit that referenced this pull request Nov 12, 2026
… of an open run

Review round 1 on #2874.

- A cancel of a run chat recovery is re-running cancels the request ids
  on the run row (the recovered turn, a recovery attempt registered
  under the first id, a scheduled recovery) and nothing else. The sweeps
  are gone: AIChatAgent's abortAllRequests, Think's recovered
  submissions, ChatTurns.cancelRecoveryAttempts.
- The recovery entry points skip a turn whose run was cancelled, right
  after their own cancelled check, and record the incident as
  user_cancelled. continueLastTurn and the retry turn no longer do it.
- A run is open while its status is starting or running. AgentToolRuns
  decides whether a row may be written; the stores write.
- AIChatAgent's store no longer turns a failed query into 'no run'.
- Docs: what a chat() callback gets for a cancel and for a clear.
devin-ai-integration[bot]

This comment was marked as resolved.

mattzcarey added a commit that referenced this pull request Nov 17, 2026
…as one, no dedicated-agent assumption

Cross-review of #2874.

- A run whose turn hands back aborted while chat recovery works on it
  was interrupted, not cancelled. It stays open and is settled when
  recovery ends. It used to be written aborted, and recovery was then
  skipped as a user cancel.
- A cancel writes its own marker (cancelled_at). Recovery reads that,
  not the status. The parent's signal goes through cancel(), so it is
  recorded at once with its reason, whatever the stream said.
- rebind binds only the run whose row names the recovered turn's root
  request id. reconcileStale looks only at incidents and the active
  stream of the run's own request ids, and after a restart counts only
  what follows the run's start as the run's (start_message_id).
- AIChatAgent hands the run's input to its turn as the body. It no
  longer writes the stored request context when the run starts or
  restores it when the run ends.
- chat(): a clear after the caller's own cancel is not reported, and a
  request id the registry cannot key is rejected.
- A tail that is closed while it reads its backlog finishes the drain
  first.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Devin Review

@mattzcarey
mattzcarey force-pushed the refactor/turn-core-rpc-agent-tool branch from 315faf0 to 5432e26 Compare November 17, 2026 17:36
@mattzcarey mattzcarey changed the title refactor(chat): agent-tool runs and Think.chat() go through the turn core refactor(chat): one agent-tool child run for AIChatAgent and Think, and Think.chat() on the turn core Nov 17, 2026
@mattzcarey
mattzcarey changed the base branch from refactor/chat-turn-core to refactor/turn-core-continuation-stop November 17, 2026 17:36
@mattzcarey
mattzcarey added this pull request to stack #2869 November 17, 2026 18:50
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-rpc-agent-tool branch from 5432e26 to 668afdf Compare November 17, 2026 22:36
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-rpc-agent-tool branch from 668afdf to 0bd307f Compare November 18, 2026 19:10
…ink.chat() on runServerTurn

AgentToolRuns (agents/chat) is the one owner of what an agent does when a
parent runs it as a tool, for AIChatAgent and Think: the run table (one
SQL store, the hosts name the tables), the run's life, inspect, chunks,
tail, and progress. The hosts supply how the run's turn is queued, what
its output is, and how a frame is broadcast.

Think.chat() is a server-started turn: it runs on ChatTurns.runServerTurn
and Think._runModelTurn with the RPC callback as its sink. Its own
admission, overflow loop and model call are gone.

What changes for a child run:
- its terminal status is written once
- a cancel is recorded as one, with its reason, and stops only that
  run's turn: through the run's controller, or by the request ids on the
  run row when chat recovery is re-running the turn
- a run cancelled while it waits in the queue stores nothing and never
  calls the model, and chat recovery does not re-run its turn
- a run whose reply is cut off while recovery can finish it stays open
  and takes the recovered turn's status
- only the turn the run's row names binds to it, holds it open or
  settles it
- a chat reset ends it as error

What changes for chat():
- a request cancelled before its message is stored stores nothing and
  never calls the model
- a chat cleared under it calls onError and stores nothing; a caller
  that had cancelled is told nothing more
- it gets the overflow handling every other turn has
…nd runAgentTool waits for a child whose isolate reset

A stop aborts the turn's signal, and whatever waits on that signal then
fails with the abort's reason: a provider call rejects with the caller's
string or Error, not with an AbortError. The reply decided 'stopped or
failed' from the thrown value, so a stop that carried a reason ended as
an error whenever the failure beat the next chunk: onError to an RPC
caller, an error frame to every tab, the error hook. The signal decides
now (isTurnStopped, turn-stop.ts), in runServerTurn, runClientRequest,
the turn fiber, both Think reply loops and AIChatAgent's error hook
wrapper. The stall watchdog aborts with a ChatStreamStalledError so its
abort is not taken for a stop.

An awaited runAgentTool tails the child over an RPC stream. A reset of
the child's isolate breaks that stream, and the parent returned error
with the stream's message while the child's chat recovery went on to
complete the run. The parent now re-resolves the child and, when the
run's row is still open, waits for it the way a recovered parent does,
within the same budgets. Giving up is interrupted, not error.
…ry stop path, and the parent's wait on a child is bounded

A run left open for chat recovery had nothing to settle it when that
recovery was cancelled: its tails stayed open, the parent's runAgentTool
never returned, and a later inspect reconciled the stopped run as
completed from the cut-off partial.

Every stop now goes through ChatTurns.cancel / stopAll, which tell the
host (ChatTurnsHost.stopped) unless the abort is the runtime's own. The
host cancels the agent-tool run the request names (AgentToolRuns.stopped)
and a recovery that is scheduled for it. So a client's cancel, cancelChat,
abortRequest, abortAllRequests and a caller's signal do the same.
abortRequest used to leave a scheduled recovery in place.

Also:
- a stop that lands after the stall watchdog aborted the turn counts as
  a stop (ChatTurns.isStopped)
- a run whose recovery was cancelled reconciles as aborted, never as
  completed
- an awaited runAgentTool inspects the child once per no-progress
  window, so a run with nothing left to finish it is settled and the
  call returns
- chunks forwarded after a re-attach are numbered past the ones already
  sent; the count survives a broken tail
- a row from before root_request_id takes the root recovery names the
  turn by, not the recovered turn's id
- a cancel that lands while the lifecycle reads recovery state leaves
  nothing in memory, and a tail of a finished run leaves no closer
- a run is not anchored on an empty reply its own turn removes
…ned, and remove what it found dead

Stryker on agent-tool-runs.ts and agent-tool-run-store.ts left 193
mutants alive at the stack tip. Now 3 survive in agent-tool-runs.ts (a
per-run timestamp that is only ever freed, and a log line) and none in
the store.

Tests added for: the state of a turn's recovery (attempting, exhausted,
failed, root id only), an error turn with no text, the tail's error
paths, terminal-only runs, the request cache, what is kept in memory,
the anchor of a run's part of the transcript, and each store method as
the first call on an older table.

Removed as dead or duplicate: the idle guard in intercept (the function
it calls has it), closeAfterDrain and the tail's closed flag, the seeded
counter cleanup, the stale re-check in the reconcile, the controller
check in reconcileOwn, null guards on values that cannot be null.

Fixes found on the way:
- a run's part of the transcript starts when its turn starts, not when
  it was queued; a run whose turn was still queued when the isolate went
  away is an error that says so, whatever another turn stored since
  (turn_started_at)
- an awaited runAgentTool whose start call fails asks the child: it
  waits for a run the child did start, and returns an error result,
  with its row settled, when the child has none
… and the tests follow the layers below

- AgentToolRuns leaves a run open when its turn hands back aborted with
  reason interrupted (runServerTurn decides it), not by reading recovery
  state: no storage read, and no window for a cancel to land in
- chat()'s reply path stores through _storeLiveReply and keeps its reset
  guard; an overflow retry hands the partial on
- tests: a stopped server turn carries reason stopped; a run's input is
  never the stored request context; the follow-up test uses the agent's
  existing onChatResponse switch
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-rpc-agent-tool branch from 0bd307f to df6d226 Compare November 19, 2026 20:45
@mattzcarey
mattzcarey removed this pull request from stack #2869 November 20, 2026 12:09
@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant