Skip to content

refactor(chat): one path for turns the server starts - #2870

Open
mattzcarey wants to merge 4 commits into
refactor/turn-core-recoveryfrom
refactor/turn-core-programmatic
Open

mattzcarey wants to merge 4 commits into
refactor/turn-core-recoveryfrom
refactor/turn-core-programmatic

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Second PR of the turn core stack, on top of #2873. Turns the server starts itself (saveMessages, continueLastTurn, the recovery retry, an agent-tool run's turn) now run through one function in the core, ChatTurns.runServerTurn (packages/agents/src/chat/server-turn.ts), the counterpart of runClientRequest. Each host builds such a turn in one place (_runServerTurn), Think has one function for a turn's model call and reply (_runModelTurn), and both hosts run a turn's durable fiber through ChatTurnFibers (turn-fiber.ts).

AIChatAgent and Think each repeated the same steps around these turns and had the same gaps. What an app can observe:

  • A turn stopped before it starts (an already-aborted signal, or abortAllRequests() while it waits in the queue) stores nothing, never calls the model and returns aborted. onChatResponse does not fire, and the record of an earlier failure stays. Before, the stop was lost in the queue or the model was called with an aborted signal. This also closes the gap refactor(chat): one recovery attempt sequence for AIChatAgent and Think #2873 left open: a recovery turn stopped or superseded while it waits in the queue no longer calls the model.
  • A turn stopped after its messages are stored and before the model call keeps the messages, calls no model and fires no onChatResponse.
  • A chat clear gives skipped, also when it cuts a reply off (it gave aborted). A stop that lands after a reply finished leaves the result completed.
  • abortAllRequests() during onChatMessage stops the reply in AIChatAgent for every kind of turn, and stops a queued reaction to a detached notification.
  • A server-started turn runs with the tools and body stored when it starts, not when it was queued. A caller can pass its own: an agent-tool run always runs with its input.
  • A turn whose reply finished and was stored is completed, also when the chat is cleared while onChatResponse runs. It was reported skipped.
  • A continuation takes a reply over from a scheduled recovery once its model call returned a reply. One that ends earlier (a repair or a hook throws, onChatMessage throws or returns nothing) leaves the recovery in place. refactor(chat): one recovery attempt sequence for AIChatAgent and Think #2873 took the reply when the turn's fiber started.
  • Tools and body passed with a turn (an agent-tool run's input, a recovered turn's context) go to that turn only and are never written to the chat's stored request context. A recovered turn no longer replaces the stored tools and body, and an agent-tool run no longer writes its input there and puts the old context back afterwards, which could undo a clear.
  • An aborted result of saveMessages or continueLastTurn carries a reason: "stopped" for a stop, "interrupted" for a reply that was cut off and handed to chat recovery, which may still finish it. Before, both read as a stop.
  • In Think, a turn whose context overflow cannot be recovered fires onChatResponse once with error, on every path, and a turn stopped at an overflow fires it once with aborted. Continuations, continueLastTurn and recovered turns fired it on main and stopped doing so when they gained overflow recovery; client requests and saveMessages never did.
  • AIChatAgent.continueLastTurn always hands onError the error onChatMessage threw. When onError itself throws, the second call used to get that thrown error.
  • Think recovers a context overflow (contextOverflow.reactive) on continued and retried turns too. An overflow with no retry left always ends its stream with an error frame and a terminal record. A turn stopped at the overflow or during compaction reports aborted, with no onChatError and no second model call.
  • Think's chat:turn:start and chat:turn:finish carry the queue generation and the status the caller gets for these turns, which also open persist_chat_result and compact_chat_history spans.

Tests ran: agents chat, ai-chat workers, the full think suite, pnpm run check. Each change above has a test in each host it applies to that fails without it. The TLA+ models (kept outside the repo) hold, #2873's included, with a broken variant for each guard.

@changeset-bot

changeset-bot Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d60ba14

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@cloudflare/ai-chat Patch
@cloudflare/think Patch
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

@pkg-pr-new

pkg-pr-new Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2870

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2870

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2870

hono-agents

npm i https://pkg.pr.new/hono-agents@2870

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2870

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2870

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2870

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2870

commit: d60ba14

@agent-think

agent-think Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🟡 agents import sizes: 1 entry point grew

Entry point Exports Largest gzip change Size now
🟡 agents/chat 1 resized, 6 new +424 B (+6.89%) 6.4 KiB
Changed exports (7)
Import Gzip change Size now
🟡 agents/chat#ChatTurns +424 B (+6.89%) 6.4 KiB
🆕 agents/chat#ChatTurnFibers — 3.2 KiB
🆕 agents/chat#runServerTurn — 3.1 KiB
🆕 agents/chat#prepareTurnMessages — 2.7 KiB
🆕 agents/chat#mapServerTurnOutcome — 2.7 KiB
🆕 agents/chat#toSaveMessagesResult — 2.6 KiB
🆕 agents/chat#pinnedOr — 2.6 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared f2bbc4bc → d60ba149 · workflow run · reported by agent-think[bot]

devin-ai-integration[bot]

This comment was marked as resolved.

saveMessages, continueLastTurn and the retry turn in AIChatAgent and Think
go through ChatTurns.runServerTurn. Each host has one function that builds
such a turn, Think has one for a turn's model call and reply, and both run
a turn's durable fiber through ChatTurnFibers.
A continuation takes a reply over from its scheduled recovery once its model
call returned a reply, not at fiber start. A reply that finished and was
stored is completed, also when the chat is cleared while onChatResponse
runs. A context pinned on a turn is never written to the stored request
context: the recovery engine hands the recovered context to the turn, and a
live interruption reads the turn's own.
…onChatResponse, and tests for surviving mutants

An aborted result carries reason stopped or interrupted (cut off and handed
to recovery). Think._runModelTurn fires onChatResponse once for an overflow
with no retry left and for a turn stopped at one. server-turn and turn-fiber
get tests for the rules mutation testing found unpinned.
…e, and one onChatResponse in the ai-chat test agent
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-programmatic branch from fd4eab1 to d60ba14 Compare November 19, 2026 15:18

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

@mattzcarey
mattzcarey removed this pull request from stack #2869 November 20, 2026 12:09
@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant