Skip to content

fix(acp-client): retry Claude's model confirmation once before prompting - #980

Open
matt2e wants to merge 1 commit into
mainfrom
why-is-fable-flakey
Open

matt2e wants to merge 1 commit into
mainfrom
why-is-fable-flakey

Conversation

@matt2e

@matt2e matt2e commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Claude's SDK confirms every setModel with a short API probe that has a fixed five-second deadline. When the probe misses that deadline, the bridge returns an internal error saying Couldn't confirm model "<id>" with the API…. Staged's pre-prompt setup then failed straight away, even though the same model was accepted a few seconds later on the next run. Session records show this happening over and over for claude-fable-5-1[1m].

  • Retry once on Claude's confirmation error (crates/acp-client/src/driver.rs): a new send_config_selection wraps session/set_config_option. For the model option only, and only on a Claude bridge (identified by the name the agent gives on initialize), it waits 1s after this exact error and sends the identical request one more time. That makes two attempts total, and a retry is never sent while an attempt is still unanswered. All other errors (auth, invalid value, transport, generic), other options and other adapters still get a single attempt, with error text unchanged.
  • Stop and timeout still work: the existing setup timeout and Stop handling drop the setup future, which also covers the delay and the retry. A Stop during either one sends nothing more and no prompt is released.
  • Saved selection is kept when both attempts fail: the error is now a plain sentence (MODEL_CONFIRMATION_EXHAUSTED_MESSAGE). It deliberately doesn't match is_config_selection_unavailable_error, so the session keeps its saved model/effort selection for the user's retry.
  • Logging: each attempt logs a sanitized record (phase, adapter, requested/resolved value, attempt, duration, outcome). Prompts, credentials and raw responses are not logged.
  • Runner refactor (session_runner.rs): the post-run clearing of a stale selection moves into clear_unavailable_acp_config_selection so it can be tested directly. Behaviour is unchanged.

Tests

New driver/tests/model_confirmation.rs runs run_acp_session end to end against a scripted in-process agent under paused time. It covers:

  • fail then succeed, and both attempts failing
  • Stop during the delay and during the second attempt
  • non-matching errors, and a non-Claude adapter
  • a model that's already current
  • effort options changed by the retried model
  • a retried fallback value
  • the setup deadline expiring

It also covers the matcher and the adapter key. A runner test checks that a confirmation failure keeps the saved selection while a stale one is still cleared.

🤖 Generated with Claude Code

Claude's SDK confirms every `setModel` with a short API probe on a fixed
five-second deadline. When the probe misses it, the bridge returns an
internal error whose `details` read `Couldn't confirm model "<id>" with
the API…`, Staged's pre-prompt setup failed on the spot, and the same
selection was accepted seconds later on the next run. Session records
show this exact failure recurring for `claude-fable-5-1[1m]`.

The driver now treats that one error as confirmation uncertainty rather
than an unavailable model: on a Claude bridge, the identical
`session/set_config_option` request is sent once more after a one-second
delay on the same connection — two attempts total, never while an attempt
is still unanswered, and only for the model option. The existing setup
timeout and cancellation boundary still bound the whole exchange by
dropping the setup future, so a Stop during the delay or the second
attempt sends nothing further and no prompt is released. On success the
existing effort-selection, stale-effort skip, and fallback-announcement
logic continue unchanged and the queued prompt is released exactly once.

Exhausted recovery fails with a plain sentence ("Claude couldn't confirm
the selected model after two attempts. Your selection is unchanged; try
again.") that deliberately does not satisfy
`is_config_selection_unavailable_error`, so the saved model/effort
selection is kept. The matcher is isolated and keyed to the requested
(resolved) model and the adapter's `initialize` name; authentication,
invalid-selection, transport, and generic errors — and every other
adapter — keep their single attempt and Debug-shaped error. Each attempt
logs a sanitized record (phase, adapter, requested/resolved value,
attempt, duration, outcome) with no prompts, credentials, or raw
responses.

The runner's post-run selection clearing is extracted into
`clear_unavailable_acp_config_selection` so the retention contract can be
tested through the real path; its behaviour is unchanged.

Tests drive `run_acp_session` end to end against a scripted in-process
agent under paused time: fail-then-succeed, both fail, cancellation
during the delay and during the second attempt, non-matching errors and
another adapter, an already-current model, effort options changed by the
retried model, a retried fallback value, and setup-deadline expiry — plus
the matcher, the adapter key, and a runner persistence regression.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T03:42:00.578838Z 1aef5a6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant