Skip to content

fix(chat): one before-turn repair and one request context for AIChatAgent and Think - #2872

Open
mattzcarey wants to merge 1 commit into
refactor/turn-core-rpc-agent-toolfrom
refactor/turn-core-repair
Open

mattzcarey wants to merge 1 commit into
refactor/turn-core-rpc-agent-toolfrom
refactor/turn-core-repair

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Top of the turn-core stack, on #2874. It adds the last two shared pieces: what a turn settles in the transcript before it calls the model (ChatTurns.settleBeforeTurn, agents/chat/before-turn.ts), and one owner for the stored client tools and body (ChatRequestContext, agents/chat/request-context.ts). AIChatAgent and Think supply only where they store messages and those two values. These are D11 and D12 in design/rfc-chat-turn-core.md.

The two classes disagreed here, and each was wrong somewhere. Apps can observe all of this:

  • Every turn fails the calls earlier replies left unanswered: a client tool call, an approval request, a call the user approved that never ran. AIChatAgent kept them, and the provider rejected the next request with AI_MissingToolResultsError. A result that arrives later is ignored.
  • repairInterruptedToolPart is called for those calls too. In AIChatAgent it only ever saw a dead server call. The default error text now says what is known: a client's tool "may still have run", an unanswered approval "did not run".
  • A skipped turn says why. SaveMessagesResult has an optional reason: "cleared", "nothing-to-continue" or "awaiting-client". continueLastTurn() returns the last one, with an empty requestId, while the reply it continues still waits on a client. It calls no model and changes nothing. An auto-continuation in that state is dropped with the new chat:continuation:dropped reason awaiting_client.
  • A reply never reaches the model ending in reasoning. Anthropic rejects an assistant message whose final block is a thinking block, so one reply cut off after reasoning failed every later request. The trailing reasoning is removed when a cut-off reply is stored and before every turn.
  • A reply that holds nothing is not stored, and a stored one is removed. "Nothing" is no part a client can show and none a model request can carry, once the reasoning the reply ends in is gone.
    • Not stored: a reply cut off while the model was still reasoning, or before any content. Think stored it and a client drew an empty bubble. Tabs that drew the reasoning get the message list after the turn's terminal frame, so they drop it.
    • Removed: a stored reply that is only reasoning. This includes a reply that finished that way (the model ran out of output tokens while thinking), not only a cut-off one. The user saw it, and the next turn deletes it. I kept that on purpose: an AIChatAgent app builds its request from the stored messages, and Anthropic rejects that shape. chat:transcript:repaired now carries removedMessages, each as it was stored, so an app can keep a copy.
    • What removes it: the next new reply, continueLastTurn(), or the recovery that would continue it. The last two then answer the question again. continueLastTurn() over such a reply used to be continued on main.
    • The recovery's decision to retry is written to the incident before the reply is removed, so a second delivery of the same payload (after a transient, an eviction or a stable-timeout reschedule) still retries.
    • In Think, a turn cut off after start-step and before any content is now retried, where it used to store that marker and continue it.
  • A tool call is stored before the tool runs. Both classes flushed only settled results at once, so a reset while a tool ran could lose the call. The reply is now rebuilt with it, and the call becomes an error. The default error text for a server tool cut off mid-run gained a sentence: "The tool may still have run."
  • A cut-off text or reasoning part is stored closed (state: "done"), in both classes. It stayed streaming after a stop.
  • A skipped regenerate leaves the stored tools and body alone (AIChatAgent). It stored them before it decided to skip.
  • An AIChatAgent continuation no longer writes into a cut-off text part. It used to resume a text part that was still streaming: the continuation's text-start was withheld from clients and its text was appended to the old part. When the model reasoned first, that reasoning landed after the merged text, so a recovered reply read text, then reasoning, and ended in reasoning. Now the cut-off part is closed when the continuation starts (settleInterruptedPart, which Think already did and which now lives in the core), and the continuation's reasoning and text are new parts after it. A tab sees text-start and a second text block where it used to see the first block grow. A plain-text continuation also opens its own part.
  • A tool result replaces the stored clientTools only when it settles its call, and not across a clear. A late duplicate from another tab used to swap them. A continuation runs with the stored tools and body, read when its turn starts.
  • AIChatAgent emits chat:transcript:repaired, before a turn and when recovery settles the reply it continues. The event has a new optional removedReasoningParts.
  • Think: a request with no clientTools key clears the stored tools, and a continuation whose repair throws is reported and settled.

Validation:

  • TLA+ models first (local, not in the repo): ChatPending and ChatTurns gained thirteen rules, each guard has a broken variant. The models are now checked with everystate, and I compared every case against TLC.
  • Each change has a test that fails without it, in chat-turn-races.test.ts, continue-last-turn.test.ts and Think's client-tools.test.ts, empty-reply.test.ts and terminal-frame-order.test.ts. The reasoning rule is tested against what convertToModelMessages produces.
  • Mutation testing found rules in repair-transcript.ts, before-turn.ts and request-context.ts that no test pinned. 26 mutants are now killed, seven guards that could not fire are deleted, and three survivors are equivalent.
  • agents chat tests, ai-chat workers, the full think suite, pnpm run build and pnpm run check ran on the rebased branch.
  • Test fixtures changed with the empty-reply rule: Think's stall and error fixtures count chunks from the first content chunk, two abort tests stop after the first text chunk, and an AIChatAgent SSE fixture sent textDelta where the AI SDK reads delta. Each had been storing a reply with no content.

@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7314bb0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@cloudflare/ai-chat Patch
@cloudflare/think Patch
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

@pkg-pr-new

pkg-pr-new Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2872

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2872

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2872

hono-agents

npm i https://pkg.pr.new/hono-agents@2872

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2872

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2872

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2872

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2872

commit: 7314bb0

@agent-think

agent-think Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🔴 agents import sizes: 1 entry point over threshold

Entry point Exports Largest gzip change Size now
🔴 agents/chat 5 resized, 15 new up to +1.5 KiB (+17.61%) 10 KiB
Changed exports (20)
Import Gzip change Size now
🔴 agents/chat#ChatTurns +1.5 KiB (+17.61%) 10 KiB
🔴 agents/chat#settleInterruptedReply +439 B (+13.84%) 3.5 KiB
🔴 agents/chat#repairInterruptedToolParts +326 B (+10.66%) 3.3 KiB
🟡 agents/chat#runContinuationTurn +131 B (+3.21%) 4.1 KiB
🟡 agents/chat#ChatRecoveryEngine +116 B (+1.2%) 9.5 KiB
🆕 agents/chat#settleReplyBeforeRecovery — 3.9 KiB
🆕 agents/chat#repairTranscriptBeforeTurn — 3.5 KiB
🆕 agents/chat#ChatRequestContext — 3.2 KiB
🆕 agents/chat#removeEmptyReplyToContinue — 3.2 KiB
🆕 agents/chat#replyToContinueIsEmpty — 2.9 KiB
🆕 agents/chat#interruptedToolPartAsError — 2.8 KiB
🆕 agents/chat#stripTrailingReasoning — 2.8 KiB
🆕 agents/chat#persistRepairedTranscript — 2.8 KiB
🆕 agents/chat#INTERRUPTED_TOOL_ERROR_TEXT — 2.7 KiB
🆕 agents/chat#continuationWaitsOnClient — 2.7 KiB
🆕 agents/chat#unsettledToolCallIds — 2.7 KiB
🆕 agents/chat#replyIsEmpty — 2.6 KiB
🆕 agents/chat#clientRequestContext — 2.6 KiB
🆕 agents/chat#settleInterruptedPart — 2.6 KiB
🆕 agents/chat#chunkMustBeStoredNow — 2.6 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared 8117abc5 → 7314bb0d · workflow run · reported by agent-think[bot]

devin-ai-integration[bot]

This comment was marked as resolved.

@mattzcarey
mattzcarey force-pushed the refactor/turn-core-repair branch from 2e778da to eded472 Compare November 17, 2026 18:36
@mattzcarey
mattzcarey changed the base branch from refactor/chat-turn-core to refactor/turn-core-rpc-agent-tool November 17, 2026 18:36
@mattzcarey mattzcarey changed the title fix(chat): one before-turn repair and request-context rule for AIChatAgent and Think fix(chat): one before-turn repair and one request context for AIChatAgent and Think Nov 17, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

Comment thread packages/think/src/think.ts Resolved
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-repair branch from 1787434 to 7314bb0 Compare November 19, 2026 21:48
@mattzcarey
mattzcarey removed this pull request from stack #2869 November 20, 2026 12:09
@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant