Skip to content

fix(chat): a stop drops the continuations that were waiting when it landed - #2881

Open
mattzcarey wants to merge 6 commits into
refactor/turn-core-continuationfrom
refactor/turn-core-continuation-stop
Open

mattzcarey wants to merge 6 commits into
refactor/turn-core-continuationfrom
refactor/turn-core-continuation-stop

Conversation

@mattzcarey

@mattzcarey mattzcarey commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

On top of #2876: stopping the turn that is running also drops the auto-continuations that were waiting when the stop landed. A tool result answered while the reply streamed, or one that arrived while a continuation ran before it was stopped, no longer starts the model again. The drop is one place in the turn core (ChatTurns.cancel, stopAll, linkExternal), for AIChatAgent and Think.

Both classes did the opposite, and agreed on it:

  • a stopped stream re-armed the continuation a mid-stream tool result had scheduled, so the model ran again about 50 ms after the stop
  • a stopped continuation turn ran the result deferred while it ran as the next continuation

Apps can observe this:

  • It holds however the turn is stopped: the client's cancel, abortRequest, abortAllRequests, Think.cancelAllChats, or the signal passed to saveMessages, continueLastTurn or chat() (through refactor(chat): one path for turns the server starts #2870's runServerTurn).
  • The dropped result stays in the transcript and the model reads it on the next turn. A tab waiting on the dropped continuation gets STREAM_RESUME_NONE, and chat:continuation:dropped is emitted with reason stopped.
  • A result that arrives after the stop still continues, also while the stopped turn winds down. A second stop for the same turn drops nothing.
  • The stall watchdog is the one abort that is not a stop: recovery continues that turn. It still goes through abortRequest.
  • Think's WebSocket cancel now goes through ChatTurns.cancel, like cancelChat.
  • abortAllRequests and cancelAllChats drop every waiting continuation whether a turn is running or not: one in its coalesce window, one parked on an unanswered tool call, and Think's queued connection-less one.
  • abortRequest(id) with no turn running drops the waiting continuation when id is the request whose reply just ended.
  • The stop button reaches the server while a tab waits for a tool continuation to start. The tab has no request id then, so the client sends cf_agent_chat_request_cancel with continuation: true, and the server stops the continuation that connection waits on. Before, the stop only ended the tab's local stream and the model ran anyway.
  • abortAllRequests also stops a recovery attempt that is waiting to run, which a test now covers.
  • In chat:continuation:dropped, requestId is the continuation's own id (null for a result dropped before it had one), and the new stoppedRequestId names the turn whose stop dropped it.

Validation:

  • agents chat tests, ai-chat workers and the think suite ran, with pnpm run check. Each case has a host test that failed before the change, except "a result that arrives after a continuation was stopped still continues", which pins a regression an earlier version of this PR had.
  • The ChatTurns TLA+ model was changed first: rule StopEndsContinuations, with a broken variant for each of the four ways a stop lands and for the second-stop and drop-at-settle mistakes. The stack's merged models pass all their cases.
  • The stop button was checked in a browser against a deployed Worker: one model call with the fix, two without.

Read ChatTurns.cancel and #stop in packages/agents/src/chat/chat-turns.ts first. Everything else follows from them.

@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 9dfa9c2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@cloudflare/ai-chat Patch
@cloudflare/think Patch
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2881

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2881

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2881

hono-agents

npm i https://pkg.pr.new/hono-agents@2881

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2881

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2881

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2881

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2881

commit: 9dfa9c2

@agent-think

agent-think Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🟡 agents import sizes: 1 entry point grew

Entry point Exports Largest gzip change Size now
🟡 agents/chat 1 resized, 1 new +589 B (+7.41%) 8.3 KiB
Changed exports (2)
Import Gzip change Size now
🟡 agents/chat#ChatTurns +589 B (+7.41%) 8.3 KiB
🆕 agents/chat#dropWaitingContinuations — 2.7 KiB
How this works

Each runtime export is bundled on its own, minified, and gzipped. Changes smaller than 100 B, or smaller than 1% and 1 KiB, are ignored. Growth over 10% or 5 KiB is marked 🔴. This report is informational and does not fail CI. The workflow artifact contains every measurement.

Compared a68db392 → 9dfa9c21 · workflow run · reported by agent-think[bot]

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@mattzcarey
mattzcarey force-pushed the refactor/turn-core-continuation-stop branch from e77e94b to 3051352 Compare November 17, 2026 16:25
@mattzcarey
mattzcarey added this pull request to stack #2869 November 17, 2026 18:50
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-continuation-stop branch from 3051352 to 79e5960 Compare November 17, 2026 22:05
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-continuation-stop branch from 79e5960 to 5040ea4 Compare November 18, 2026 18:12
@mattzcarey
mattzcarey force-pushed the refactor/turn-core-continuation-stop branch from 5040ea4 to 9dfa9c2 Compare November 19, 2026 17:20

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

Comment thread packages/think/src/think.ts Resolved
@mattzcarey
mattzcarey removed this pull request from stack #2869 November 20, 2026 12:09
@mattzcarey
mattzcarey added this pull request to stack #2923 November 20, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant