Skip to content

fix: honor Effort Off for qualified Qwen local AI models - #2012

Open
turutupa wants to merge 5 commits into
openclaw:mainfrom
turutupa:fix/local-ai-reasoning-1976
Open

turutupa wants to merge 5 commits into
openclaw:mainfrom
turutupa:fix/local-ai-reasoning-1976

Conversation

@turutupa

@turutupa turutupa commented Nov 4, 2026 •

Copy link
Copy Markdown

Related: #1976

What Problem This Solves

Effort Off does not stop managed Qwen Local AI models from thinking. The llama-server preset forces reasoning = on, and the Gateway provider entry has no effort vocabulary, so Off is never sent as none. Canvas actions also send thinking: "low", which switches the session's Effort.

User Impact

With a qualified Qwen model, Effort Off now sends reasoning_effort: "none" and the model answers without reasoning. Qwen3.8 offers Off, Low, Medium, High and Ultra. Qwen3.6 and Qwen3.5 offer Off, Low, Medium and Ultra: Low and Ultra are sent as medium, the only other effort these templates accept. Canvas actions keep the user's Effort. Existing installs are upgraded automatically without changing their credentials. Bonsai, Nemotron Bolt and custom models keep their current provider entry (tracked in #1976).

Why This Change Was Made

  • The router preset no longer forces reasoning, so llama-server's default auto mode lets each request turn it off. Models without a reasoning profile (Bonsai, custom) get no effort vocabulary, so the Gateway never asks them for none, and auto mode keeps a thinking template reasoning as before. A test now pins this down.
  • Qualified Qwen catalog entries carry a reasoning profile. The provider entry declares supportedReasoningEfforts, reasoningEffortMap (off to none) and thinkingLevelMap for those templates only.
  • Ownership, recovery, rollback and teardown recognize the exact pre-upgrade provider shape, so an unchanged existing route is not treated as edited outside the companion.
  • Upgrading an exact pre-upgrade provider:
    • Normal path, WSL and native: every runtime start (and every non-ready refresh) withdraws the route before publishing (LlamaServerRuntimeService, endpoint-cycle quiesce). The new shape is then published from scratch, with the API key from the companion's credential store, never from a redacted readback.
    • WSL: if a publication still meets the pre-upgrade shape, it is withdrawn the same way (unset, Gateway restart, removal read back) and published fresh. A failed withdrawal fails the publication and writes nothing new.
    • Native: any publication that meets the pre-upgrade shape patches only the owned models array, atomically (config.patch against the captured hash), then reads it back. In practice this happens on explicit endpoint recovery ("Recover and use Local AI"), which deliberately keeps the verified route; startup has already withdrawn it otherwise. If the readback doesn't match, the failure is reported and the upgraded route is left in place.
  • The provider shape follows what the installed Gateway accepts, decided before anything is written (commits 4 and 5, from @joelagnel's review). The companion admits any protocol-4 Gateway, but thinkingLevelMap only exists from Gateway 2026.6.5. Gateway 2026.6.1 rejects this PR's entry, and the earlier code published it anyway, so Local AI ended with no route.
    • Every publication decides the shape first. WSL dry-runs the new entry (openclaw config set --batch-file … --dry-run), then the pre-upgrade entry if needed; a rejection counts only when the dry-run fails twice. Native reads config.schema (config.patch has no dry-run).
    • Gateways that reject effort metadata keep, or get, the exact pre-upgrade entry. Local AI keeps working; Effort Off behaves as on main.
    • If neither shape validates, nothing is written.
    • Withdrawals never wait on this check: the runtime also withdraws to take the route off an untrusted or conflicting listener, exactly as on main (commit 5).
    • This covers the WSL migration, startup publication, native publication and setup's WSL step.
    • The previous description said "there is no Gateway version floor"; that was wrong. Measured behavior per Gateway release is under Gateway versions.

Five commits on current main (17794662):

  1. The fix.
  2. Removal of the WSL in-place migration, which can't run in a real WSL upgrade. Every start withdraws the route first, and the router is in a kill-on-close job object (LocalAiManagedProcessHost). A pre-upgrade WSL provider is withdrawn and republished instead.
  3. Tests from an independent review: a native migration whose readback fails verification, and the preset for models without a reasoning profile. Test-only.
  4. The Gateway compatibility guard above, with regression tests (7bdbfe9d).
  5. Fixes from an independent review of commit 4 (4c902b0b): withdrawal no longer waits on the shape check, the decision is made once per publication, one failed dry-run can't downgrade a route, and the two failure messages say nothing was written.

Default effort (the open decision in #1976)

Measured on every stable OpenClaw Gateway release from 2026.6.1 to 2026.11.6 (50 releases, none skipped), with Qwen3.8 27B and the companion's GGUF and preset. The method and every release's numbers are under Gateway versions.

Installed Gateway Default turn, main Default turn, this branch Effort Off, main Effort Off, this branch
2026.6.1 nothing sent nothing sent (pre-upgrade entry kept) nothing sent, reasons same as main
2026.6.5 – 2026.7.35 (13 releases) nothing sent medium nothing sent, reasons high sent, reasons
2026.8.1 – 2026.10.4 (23) nothing sent medium nothing sent, reasons none, no reasoning
2026.10.5 – 2026.11.6 (13) medium medium nothing sent, reasons none, no reasoning

"Nothing sent" means the chat template's default applies. Its system text reads "Reasoning effort is set to xhigh".

  • On Gateway 2026.10.5 and later (setup installs the latest Gateway; 2026.11.6 in the runs below), this PR does not change the default effort: main already sends medium. It fixes Off.

  • On Gateway 2026.8.1 – 2026.10.4, the default changes from the template's xhigh to medium. This is the trade-off Local AI: Effort Off doesn't stop llama-server thinking, and newer gateways return HTTP 500 for unsupported levels #1976 leaves open.

  • On 2026.6.5 – 2026.7.35, those Gateways send Off as high, not none, so Off still reasons. main sends nothing there, so Off reasons on main too.

  • With main's preset (reasoning = on), llama-server also ignores none: an Off request spent 98 tokens (median) reasoning. This branch removes that, and none gives 4.

  • 12-prompt benchmark (details below):

    Correct Median tokens
    No effort sent (template default) 12/12 98
    medium (this branch's default) 12/12 160
    none (this branch's Off) 7/12 4

    Off trades accuracy for speed, as chosen by the user. It is not the default.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Tests, CI, or docs

Required proof pools

  • windows-wsl-dgx-blackwell: Local AI on a Blackwell GPU. Inventory recipe run, plus fixed-prompt inference, below.
  • windows-wsl-gateway-e2e: a real managed WSL Gateway upgraded from v2026.11.3 (refused restarts and recovery included), plus the inventory recipe, below.
  • windows-clean-installer-upgrade: v2026.11.3's own setup (headless SetupEngine for WSL; the release app's onboarding UI for native), then this branch on the same data, plus the installer source assertions recipe. Signed installers: see Not verified.
  • windows-11-arm64: every run below is native ARM64 (RTX Spark N1X).
  • windows-winui-interactive: Effort menu, chat, recovery UI and canvas in this branch's app, plus the inventory recipe, below.

Validation

Windows 11 ARM64 (NVIDIA RTX Spark N1X, Blackwell), head 4c902b0b:

  • WinUI app (OpenClaw.Tray.WinUI, Debug win-arm64): 0 warnings, 0 errors. ProductVersion …Sha.4c902b0b3310….
  • Focused tests, all passing:
    • Tray (GatewayActionTransportTests, LocalAiGatewayProviderCoordinatorTests, LocalAiGatewayRpcConfigurationTransportTests, LocalAiPageViewModelTests): 147 passed. Includes the shape tests:

      • a rejected replacement keeps the legacy route with no unset or restart;
      • startup republishes the legacy shape;
      • nothing is written when neither shape validates;
      • one failed dry-run doesn't downgrade;
      • endpoint-cycle withdrawal never waits on the check (WSL, and native with publication through a separate coordinator);
      • native schema checks, including a Gateway without config.schema.

      It also includes RpcPublish_LegacyMigrationThatFailsReadbackKeepsTheUpgradedRouteAndReportsLocalizedFailure.

    • Connection (LocalAiPortLifecycleTests): 131 passed, 1 skipped. Includes the Bonsai preset assertions.

    • SetupEngine (NativeLocalAiLifecycleTests, LocalAiGatewayUninstallTests, LocalAiOnboardingTests, SetupPipelineTests): 296 passed. Includes the setup fallback to the legacy batch and its rollback.

    • scripts/Test-Localization.ps1: 44 passed.

  • git diff --check: clean.
  • Proof-pool inventory recipes (scripts/run-proof-tests.ps1, checked with scripts/validate-proof-evidence.ps1), at 4c902b0b:
    • windows-wsl-dgx-blackwell/run-local-ai-gpu-tests, exactly as inventoried: 15 discovered, 15 passed, 0 skipped. validExecutionEvidence: true. The summary also binds the app binary and the eight screenshots below, including the two older-Gateway ones.

    • windows-wsl-dgx-blackwell/record-gpu-model: NVIDIA RTX Spark N1X (6144-core Blackwell RTX GPU), 621.24.

    • The win-x64 recipes ran exactly as inventoried, on an x64 .NET runtime installed side by side (DOTNET_ROOT_X64):

      • windows-wsl-gateway-e2e/run-gateway-product-path (OPENCLAW_RUN_E2E=1, OPENCLAW_RUN_OLLAMA_E2E=1, SetupAndConnectTests, win-x64): 78 discovered, 78 passed, 0 skipped, validExecutionEvidence: true. It used a fresh app-owned WSL distro, removed afterwards. Two earlier runs at this head failed 1 and 4 cases (an MCP connection refused, then node pairing not completing). Both ran while the test distro from the live runs below still had its own Gateway running. With that distro stopped, the next run passed 78/78. I did not isolate the cause further; at 7bdbfe9d this recipe passed 78/78 on its first run.
      • windows-clean-installer-upgrade/run-installer-source-assertions (InstallerIssAssertionTests): 21 passed, 0 skipped, validExecutionEvidence: true.
    • windows-winui-interactive/run-accessibility-scan (Category=Accessibility, win-x64, private desktop), run at f096746c (commits 4 and 5 change no XAML), repeated on both builds:

      Build Run 1 Run 2 Run 3
      This branch f096746c 24 passed, 1 failed 24 passed, 1 failed 25 passed
      main 17794662 25 passed 24 passed, 1 failed

      The one failure is the same intermittent DashboardsPage ButtonShouldHavePatterns finding on both builds, and this PR doesn't touch that page. Every run also has 14 Gateway-fixture cases skipped (opt-in through scripts/test-gateway-fixture.ps1). The runner treats skips as incomplete proof, so it writes no valid summary for this recipe on either build.

  • Base-failing check (this branch's tests against main's production code, 17794662):
    • Connection: 2 failures, RouterPreset_BoundsOmittedGenerationAtGatewayMaximum and Qwen38Provider_HidesUnsupportedMinimalButKeepsFourSupportedEfforts.
    • Tray (GatewayActionTransportTests): 2 failures, BuildAgentRequestPayload_InheritsSessionEffort and …DeliverIsFalse_WithoutChangingEffort.
    • The coordinator and SetupEngine migration tests don't compile on main: legacy-shape recognition doesn't exist there.
  • Commit 4's tests against commit 3's production code (f096746c):
    • SetupEngine (LocalAiGatewayUninstallTests): 1 failure, Configure_GatewayRejectingEffortMetadataPublishesAndRollsBackLegacyShape (setup fails instead of falling back); 29 passed.
    • Tray: the coordinator and RPC transport tests don't compile there (SupportsEffortMetadataAsync and the shape check don't exist).
  • Commit 5's tests against commit 4's production code (7bdbfe9d): 6 of 68 coordinator tests fail. They are the two endpoint-cycle withdrawal tests, the one-failed-dry-run test, the rejected-both-shapes test, and the two legacy-shape tests whose dry-run count changed.

Independent review

An independent reviewer (a separate Claude Code agent given only the diff, with no access to this description) reviewed 17794662..34f80858. It found no blocking bugs and checked:

  • credentials: a migration patch carries only models, WSL republishes from the credential store, and rollback rebuilds from the known key;
  • partial failures and races: hash re-checks on every journaled capture;
  • providers in neither shape: still rejected as changed outside the companion;
  • localization: all six .resw files.

Its three P3 notes are addressed:

  1. A test for a migration whose readback fails: commit 3.
  2. Pinning the preset for models without a reasoning profile: commit 3.
  3. A more precise description of where the native migration runs: above.

It also flagged one thing it could not check without a Gateway: whether a live Gateway keeps thinkingLevelMap's null values. The readbacks below show it does ("thinkingLevelMap":{"minimal":null}).

A second independent review, again with only the diff, covered commit 4 (f096746c..7bdbfe9d). It found one P1, three P2s and two P3s; commit 5 addresses them:

  • P1: endpoint-cycle withdrawal waited on the shape check. The runtime also withdraws that way from an untrusted or conflicting listener (LlamaServerRuntimeService ownership paths), so a failed check left the route on that listener. Withdrawal is unconditional again, as on main.
  • P2: native quiesce and publish use separate coordinator instances, so the cached decision was lost and publish checked again after the withdrawal; its failure message said the route was kept when it wasn't. There is no cache now: one decision per publication, before any write. The messages say nothing was written.
  • P2: the native check matches the effort key names anywhere in the config.schema text. Measured instead of assumed: on all 50 releases, a live config.schema response gives the same verdict as the dry-run. 2026.6.1 doesn't name thinkingLevelMap anywhere; every later release names it once, as a model property (Gateway versions). The set of Gateways older than the effort keys is closed, so the match can't misfire on them.
  • P2: a Gateway that doesn't list config.schema is treated as newer. Checked:
    • every release from 2026.6.1 serves config.schema, measured on all 50;
    • in 2026.6.1 and 2026.6.5 the hello method list is the Gateway's base methods plus plugin methods, the same for every client and not filtered by scope (source: server-plugins.ts);
    • so the one Gateway that rejects the keys always lists it.
  • P3: WSL treated any failed dry-run as a schema rejection. A rejection now has to repeat.
  • P3: stale cached shape after a failed quiesce. The cache was removed.

It confirmed the rest held: no early withdrawal on publish, setup fallback and rollback, the ownership matching, no key in dry-runs or RPC checks, and the six translations.

Real Behavior Proof

All app runs are on a private desktop (scripts/Invoke-PrivateDesktop.ps1), driven through UI Automation. API keys are shown only as SHA-256 prefixes.

Before and after on the same machine, Gateway and runtime (heads 17794662 and f096746c)

  • Environment:
    • Native Gateway: Store package, isolated session.
    • Local AI: Qwen3.8 27B (UD-Q4_K_M) on the companion-pinned llama.cpp b11320.
  • Steps, the same for both builds:
    1. main sets up Local AI and writes its provider.
    2. This branch upgrades it.
    3. Each build gets a fresh conversation: a default turn, then Effort Off with What is 19 times 21? Reply with only the number.
main 17794662 This branch f096746c
Router preset reasoning = on no forced reasoning
Provider entry compat: {supportsTools, supportsUsageInStreaming} adds supportedReasoningEfforts: ["none","low","medium","high"], reasoningEffortMap: {"off":"none"}, thinkingLevelMap: {"minimal":null}; same key 98ea79e6020f and primary
Effort menu off, minimal, low, medium, high, ultra off, low, medium, high, ultra
Effort Off turn reply 399, 18 tokens generated (reasoning) reply 399, 4 tokens (answer only)

main, Effort menu:

main-17794662-effort-menu

main, Effort Off:

main-17794662-effort-off-chat

This branch, Effort menu:

branch-f096746c-effort-menu

This branch, Effort Off:

branch-f096746c-effort-off-chat

This branch, Local AI running after the upgrade:

branch-f096746c-localai-running-after-upgrade

Older Gateway: 2026.6.1 before and after the fix (heads f096746c and 4c902b0b)

The case from @joelagnel's review, on a real install.

  • Environment: the v2026.11.3 release's own SetupEngine (headless, isolated data roots) with Gateway.Version set to 2026.6.1: a new app-owned WSL distro, OpenClaw Gateway 2026.6.1 as a systemd service, Local AI with Qwen3.8 27B. The v2026.11.3 app wrote the pre-upgrade provider (key 8f93e66f6416). Then each branch build was started on that data.
  • Schema, checked independently (openclaw config set --batch-file … --dry-run, throwaway profiles):
    Gateway 2026.6.1  pre-upgrade entry: Dry run successful
    Gateway 2026.6.1  this PR's entry:   Dry run failed: config schema validation failed.
                                         - models.providers.llamacpp.models.0: Unrecognized key: "thinkingLevelMap"
    Gateway 2026.6.5  both entries:      Dry run successful
    
Gateway f096746c (before the fix) 4c902b0b
2026.6.1 Startup withdraws the route, then writes this PR's entry, which the Gateway rejects: The verified Local AI route could not be published to the app-owned gateway. The just-written route was removed. No llamacpp provider left; router Failed Startup withdraws the route (as on main), decides the shape, then republishes the pre-upgrade entry: The Gateway does not accept Local AI effort metadata (it needs OpenClaw Gateway 2026.6.5 or later). Publishing the earlier provider shape, so Effort Off cannot turn reasoning off on this Gateway. Same key 8f93e66f6416, same primary; router Healthy. The same on a second start
2026.11.6 Not run with this head; at f096746c the upgrade was shown on Gateway 2026.11.4 (WSL upgrade below) Upgraded: the effort fields added, same key 8f93e66f6416 and primary; router Healthy

f096746c on Gateway 2026.6.1, Local AI page after startup (red toast: an unrelated private-desktop notification):

4c902b0b on Gateway 2026.6.1, Local AI page after startup (same unrelated toast):

On Gateway 2026.6.1, v2026.11.3's own setup fails at its pair-node step (Node approval failed), in both runs. That step is in the released setup, not in this PR; the operator connection and Local AI were set up.

Native endpoint recovery: models-only in-place patch (head f096746c)

Step Who Result
1. Router started, app process ended main 17794662 Pre-upgrade shape, key 98ea79e6020f, primary llamacpp/qwen3.8-27b-mtp-ud-q4-k-m
2. Unrelated Gateway edit while the companion is closed openclaw config set agents.defaults.timeoutSeconds 601 Provider untouched; the config hash no longer matches the ownership receipt
3. Start this branch No automatic write. The Local AI page shows the recovery guidance (screenshot)
4. Retry setup, then the Local AI card "The owned Gateway route needs recovery · Recover and use Local AI" this branch Gateway log: one config.patch with changedPaths=models.providers.llamacpp.models
5. Readback Key, endpoint and primary unchanged; effort fields added; agents.defaults.timeoutSeconds kept
6. Setup's normal post-setup restart this branch Regular startup cycle (withdraw, publish); key and primary unchanged

Recovery guidance (test Gateway id blacked out):

branch-f096746c-recovery-guidance

The recovery card (captured at 34f80858; commit 3 changes only tests):

branch-34f80858-recover-and-use-local-ai-card
Gateway log, config writes (redacted)
# Native Gateway (Store package) config writes, head f096746c. Device and connection ids redacted.
# 16:56-16:57  branch app upgrades main's provider at startup: withdraw, then publish.
# 17:01:55     test reset: provider removed with the CLI so main can write it again.
# 17:03:11     main (17794662) writes the pre-upgrade provider.
# 17:03:53     unrelated CLI edit while the companion is closed.
# 17:09:08     branch app: "Recover and use Local AI" -> one models-only patch.
# 17:09:45     the setup's normal post-setup app restart: regular startup cycle.
16:56:47.906 [gateway] config.patch write actor=gateway-client changedPaths=models.providers.llamacpp
16:57:23.381 [gateway] config.patch write actor=gateway-client changedPaths=models.providers.llamacpp
17:01:55.495 [gateway] config.patch write actor=cli changedPaths=models.providers.llamacpp
17:03:11.661 [gateway] config.patch write actor=gateway-client changedPaths=models.providers.llamacpp
17:03:53.149 [gateway] config.patch write actor=cli changedPaths=agents.defaults.timeoutSeconds
17:09:08.457 [gateway] config.patch write actor=gateway-client changedPaths=models.providers.llamacpp.models
17:09:09.287 [reload] config hot reload applied (models.providers.llamacpp.models)
17:09:45.554 [gateway] config.patch write actor=gateway-client changedPaths=models.providers.llamacpp,agents.defaults.model.primary
Provider readbacks (hashes only)
## main 17794662 wrote it (before the upgrade)
apiKey sha256_12=98ea79e6020f  primary=llamacpp/qwen3.8-27b-mtp-ud-q4-k-m
model {"id":"qwen3.8-27b-mtp-ud-q4-k-m","reasoning":true,"contextWindow":262144}
compat={"supportsTools":true,"supportsUsageInStreaming":true}
## branch f096746c, after the startup upgrade
apiKey sha256_12=98ea79e6020f  primary=llamacpp/qwen3.8-27b-mtp-ud-q4-k-m
model {"id":"qwen3.8-27b-mtp-ud-q4-k-m","reasoning":true,"contextWindow":262144,"thinkingLevelMap":{"minimal":null}}
compat={"supportsTools":true,"supportsUsageInStreaming":true,"supportsReasoningEffort":true,"supportedReasoningEfforts":["none","low","medium","high"],"reasoningEffortMap":{"off":"none"}}
## main 17794662 wrote it again (before the recovery); baseUrl sha256_12=811199df9a82
apiKey sha256_12=98ea79e6020f  primary=llamacpp/qwen3.8-27b-mtp-ud-q4-k-m
compat={"supportsTools":true,"supportsUsageInStreaming":true}
## branch f096746c, after "Recover and use Local AI"; baseUrl sha256_12=811199df9a82 (unchanged)
apiKey sha256_12=98ea79e6020f  primary=llamacpp/qwen3.8-27b-mtp-ud-q4-k-m
model {"id":"qwen3.8-27b-mtp-ud-q4-k-m","reasoning":true,"contextWindow":262144,"thinkingLevelMap":{"minimal":null}}
compat={"supportsTools":true,"supportsUsageInStreaming":true,"supportsReasoningEffort":true,"supportedReasoningEfforts":["none","low","medium","high"],"reasoningEffortMap":{"off":"none"}}
agents.defaults.timeoutSeconds: kept

Upgrade from v2026.11.3, managed WSL Gateway (head 34f80858, production code unchanged since)

  • Environment:
    • The v2026.11.3 release's own SetupEngine, run headless (as tests/OpenClaw.E2ETests does) with isolated data roots: a new app-owned WSL distro, OpenClaw Gateway 2026.11.4 as a systemd service, and Local AI with Qwen3.8 27B (UD-Q4_K_M).
    • Then the v2026.11.3 portable app, then this branch's app, on the same data.
Step App Result
1. Fresh install, app started v2026.11.3 Pre-upgrade shape, key 8f93e66f6416, router Healthy
2. Upgrade with Gateway restarts refused (systemd drop-in in the test distro) this branch Route unset; The managed Local AI route was removed from Gateway configuration, but the Gateway restart was not confirmed…; startup Failed; nothing new written
3. Restarts allowed, app started again this branch Restart, write, restart, readback, Healthy; effort fields present; same key and primary; preset without reasoning = on

Stored data: every value in the seven files v2026.11.3 wrote was hashed by key path (values never printed). 147 of 148 values were unchanged. The only change is gateways.json :: gateways[0].lastConnected. No new stored fields or files.

Effort Off chat on the upgraded WSL install (head f096746c):

  1. A fresh v2026.11.3 install, started once.
  2. Then this branch's app on the same data. The provider gained the effort fields with the same key 8f93e66f6416 and the same primary.
  3. Effort Off: the reply 399, 16 tokens generated, no reasoning. The Gateway session reports thinkingLevel=off.
branch-f096746c-wsl-upgraded-effort-off-chat

The default-effort turns in these WSL runs got "Agent error" because llama-server could not load the model on this test laptop. It has 31 GB of GPU memory, and the model at its preset's 262k context needs about 37 GB: 15 GB of weights, 16 GB of KV cache and the compute buffers. It loads only when Windows can spill into shared memory. The log line is failed to allocate CUDA0 buffer of size 2718040192 … exiting due to model loading error. The preset's size settings are the same on main and v2026.11.3; this PR only removes reasoning = on. Default turns are covered natively above and on Linux below.

Upgrade from v2026.11.3, native Gateway (head 34f80858)

v2026.11.3's own onboarding: native Gateway, "Set up a local model" with Qwen3.8 27B. Then this branch: route withdrawn at startup and republished with the effort fields, same key 98ea79e6020f. The same happened with the native Gateway stopped beforehand (the app starts it).

Rollback: v2026.11.3 started after this branch sees the new shape as "changed outside the companion". It keeps the provider untouched and refuses to cycle it (Error on the Local AI page); nothing is overwritten or deleted. "Release Gateway ownership", or removing the provider, lets it set up again.

Effort menu, canvas and request path (head 4ec8e743; the production code for these is unchanged since)

main provider entry This branch's entry
Off selected no reasoning_effort, 14 reasoning chars reasoning_effort: "none", 0 reasoning chars
Ultra selected high sent high sent
Canvas action after choosing Off main app This branch's app
Canvas turn low sent, 638 reasoning chars none sent, 0 reasoning chars
Session thinkingLevel afterwards low off

This branch's app after the canvas action (Effort still "off"; host name and device instance id blacked out):

pr-canvas-effort-stays-off

main's app after the canvas action:

main-canvas-switches-effort

Gateway request path for every level (openclaw agent --thinking <level>, logging proxy):

  • Before, Effort Off: no reasoning_effort, and the reply carried reasoning.
  • After, Effort Off: reasoning_effort: "none", 0 reasoning chars.
  • Qwen3.8 sent low / medium / high as chosen; Qwen3.6 sent medium for Low and Medium.
  • minimal and xhigh were rejected by the Gateway.

Gateway-side check on Linux (DGX Spark)

The same model file and Gateway version on Linux, with no memory limits. This shows the Gateway request and the model behavior for both provider entries; it is not the Windows product path, which is covered above.

  • Setup:
    • OpenClaw Gateway 2026.11.4 in an isolated profile, on loopback.
    • The companion's GGUF: unsloth/Qwen3.8-27B-GGUF UD-Q4_K_M, revision 313447f, SHA-256 322e194f…, as the Hub's LFS hash.
    • llama-server with the companion preset (MTP draft, reasoning-format deepseek, sampling as shipped). ctx-size 65536 instead of 262144, on llama.cpp b12367.
    • The provider entry is this branch's entry exactly as read back from the upgraded WSL Gateway, without the key. main's entry is the same without the effort fields.
    • A logging proxy records each request's reasoning_effort and the reply's reasoning and answer sizes (no message text).

openclaw agent --thinking <level>, the prompt "What is 19 times 21? Reply with only the number.":

Level main entry, main preset (reasoning = on) This branch's entry and preset
Default medium sent; 18 tokens medium sent; 285 tokens
Off nothing sent; 29 tokens, 61 reasoning chars none sent; 4 tokens, 0 reasoning chars
Low low; 63 tokens low; 40 tokens
Medium medium; 22 tokens medium; 19 tokens
High high; 76 tokens high; 279 tokens

Single turns at temperature 1, so the token counts vary from turn to turn; what matters is what is sent and what Off does. Every reply was 399 except this branch's Off turn, which answered 429 without reasoning.

Benchmark. 12 arithmetic and logic prompts with known answers, one request per prompt and effort, sent straight to llama-server:

reasoning_effort sent main preset: correct, median tokens This branch's preset: correct, median tokens
none sent (main's Off) 12/12, 113 12/12, 98
none (this branch's Off) 12/12, 98 (ignored: still reasons) 7/12, 4
low 12/12, 161 11/12, 159
medium (default) 12/12, 182 12/12, 160
high 12/12, 94 12/12, 124

Gateway versions: what Effort Off and the default send

Every stable OpenClaw Gateway release from 2026.6.1 to 2026.11.6: 50 releases from npm, none skipped. Each ran in an isolated loopback profile on the Linux setup above (same GGUF, preset and logging proxy), once with main's provider entry and once with this branch's. Each got a default turn and an Off turn (openclaw agent --thinking off), "What is 19 times 21? Reply with only the number.". The proxy records the reasoning_effort the Gateway sent and the reply's reasoning size.

  • Which entry each release accepts (dry-run of this branch's entry, and the config.schema RPC on a live Gateway, the native check's input):
    • 2026.6.1: dry-run Unrecognized key: "thinkingLevelMap"; the schema doesn't name it. A config that already contains it stops 2026.6.1 from starting (Invalid config). The fix keeps main's entry there.
    • 2026.6.5 – 2026.11.6: the dry-run passes, and the schema names thinkingLevelMap exactly once, as a property of models.providers.*.models[].
    • The native check (all effort keys named in the config.schema response) gives the same answer as the dry-run on all 50 releases. Every release serves config.schema.
  • Single turns at temperature 1: token counts vary from turn to turn. The pattern is in what is sent and whether Off reasons. Every branch Off turn that sent none had 0 reasoning chars.
  • npm install openclaw@2026.7.33 leaves out its declared dependency @openclaw/ai@2026.7.33, so its CLI doesn't start. It was measured with that exact declared version installed next to it.
  • Three cells were rerun for harness reasons, and the table shows the rerun:
    • 2026.11.6 (both entries): it refuses a group-writable state directory as its temp root, so the test profile is now chmod 700;
    • 2026.10.1's branch Off turn: the first attempt sent none and then hit the 500 s turn timeout.
Every release
Gateway main entry: default main entry: Off Branch entry: default Branch entry: Off
2026.6.1 nothing sent; 245 tokens, 1084 reasoning chars nothing sent; 263 tokens, 1140 reasoning chars Gateway refuses to start: Invalid config (the fix keeps main's entry here) same
2026.6.5 nothing sent; 1068 tokens, 4753 reasoning chars nothing sent; 314 tokens, 1403 reasoning chars medium; 36 tokens, 129 reasoning chars high; 126 tokens, 547 reasoning chars
2026.6.6 nothing sent; 2627 tokens, 11657 reasoning chars nothing sent; 473 tokens, 1953 reasoning chars medium; 262 tokens, 1197 reasoning chars high; 280 tokens, 1265 reasoning chars
2026.6.8 nothing sent; 65 tokens, 206 reasoning chars nothing sent; 2640 tokens, 11792 reasoning chars medium; 1046 tokens, 4294 reasoning chars high; 841 tokens, 3537 reasoning chars
2026.6.9 nothing sent; 255 tokens, 1128 reasoning chars nothing sent; 88 tokens, 349 reasoning chars medium; 368 tokens, 1555 reasoning chars high; 230 tokens, 990 reasoning chars
2026.6.10 nothing sent; 19 tokens, 14 reasoning chars nothing sent; 751 tokens, 3230 reasoning chars medium; 141 tokens, 515 reasoning chars high; 322 tokens, 1390 reasoning chars
2026.6.11 nothing sent; 389 tokens, 1835 reasoning chars nothing sent; 153 tokens, 552 reasoning chars medium; 28 tokens, 59 reasoning chars high; 783 tokens, 3203 reasoning chars
2026.6.33 nothing sent; 369 tokens, 1637 reasoning chars nothing sent; 643 tokens, 2707 reasoning chars medium; 252 tokens, 1150 reasoning chars high; 533 tokens, 2350 reasoning chars
2026.6.34 nothing sent; 1892 tokens, 8198 reasoning chars nothing sent; 405 tokens, 1676 reasoning chars medium; 128 tokens, 548 reasoning chars high; 201 tokens, 885 reasoning chars
2026.6.35 nothing sent; 399 tokens, 1778 reasoning chars nothing sent; 93 tokens, 366 reasoning chars medium; 1474 tokens, 6514 reasoning chars high; 40 tokens, 40 reasoning chars
2026.7.1 nothing sent; 69 tokens, 227 reasoning chars nothing sent; 787 tokens, 3206 reasoning chars medium; 449 tokens, 1817 reasoning chars high; 386 tokens, 1673 reasoning chars
2026.7.33 nothing sent; 307 tokens, 1388 reasoning chars nothing sent; 78 tokens, 167 reasoning chars medium; 458 tokens, 1800 reasoning chars high; 427 tokens, 1853 reasoning chars
2026.7.34 nothing sent; 473 tokens, 2005 reasoning chars nothing sent; 259 tokens, 1181 reasoning chars medium; 194 tokens, 860 reasoning chars high; 345 tokens, 1468 reasoning chars
2026.7.35 nothing sent; 39 tokens, 77 reasoning chars nothing sent; 295 tokens, 1347 reasoning chars medium; 179 tokens, 656 reasoning chars high; 59 tokens, 167 reasoning chars
2026.8.1 nothing sent; 292 tokens, 1145 reasoning chars nothing sent; 182 tokens, 755 reasoning chars medium; 92 tokens, 344 reasoning chars none; 4 tokens, 0 reasoning chars
2026.8.2 nothing sent; 224 tokens, 891 reasoning chars nothing sent; 459 tokens, 2004 reasoning chars medium; 22 tokens, 15 reasoning chars none; 4 tokens, 0 reasoning chars
2026.8.33 nothing sent; 37 tokens, 98 reasoning chars nothing sent; 70 tokens, 257 reasoning chars medium; 11 tokens, 4 reasoning chars none; 4 tokens, 0 reasoning chars
2026.8.34 nothing sent; 30 tokens, 61 reasoning chars nothing sent; 83 tokens, 305 reasoning chars medium; 24 tokens, 36 reasoning chars none; 4 tokens, 0 reasoning chars
2026.8.35 nothing sent; 175 tokens, 524 reasoning chars nothing sent; 89 tokens, 327 reasoning chars medium; 112 tokens, 410 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.1 nothing sent; 35 tokens, 85 reasoning chars nothing sent; 237 tokens, 932 reasoning chars medium; 30 tokens, 84 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.2 nothing sent; 111 tokens, 363 reasoning chars nothing sent; 423 tokens, 1767 reasoning chars medium; 68 tokens, 247 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.3 nothing sent; 66 tokens, 137 reasoning chars nothing sent; 200 tokens, 773 reasoning chars medium; 40 tokens, 82 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.4 nothing sent; 24 tokens, 40 reasoning chars nothing sent; 185 tokens, 583 reasoning chars medium; 23 tokens, 53 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.5 nothing sent; 827 tokens, 3580 reasoning chars nothing sent; 169 tokens, 622 reasoning chars medium; 27 tokens, 54 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.6 nothing sent; 70 tokens, 135 reasoning chars nothing sent; 165 tokens, 700 reasoning chars medium; 278 tokens, 994 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.7 nothing sent; 218 tokens, 878 reasoning chars nothing sent; 222 tokens, 863 reasoning chars medium; 61 tokens, 115 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.8 nothing sent; 171 tokens, 672 reasoning chars nothing sent; 247 tokens, 870 reasoning chars medium; 136 tokens, 525 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.9 nothing sent; 180 tokens, 627 reasoning chars nothing sent; 28 tokens, 54 reasoning chars medium; 271 tokens, 1130 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.33 nothing sent; 139 tokens, 424 reasoning chars nothing sent; 271 tokens, 1009 reasoning chars medium; 385 tokens, 1490 reasoning chars none; 45 tokens, 0 reasoning chars
2026.9.34 nothing sent; 211 tokens, 751 reasoning chars nothing sent; 111 tokens, 402 reasoning chars medium; 153 tokens, 455 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.35 nothing sent; 159 tokens, 582 reasoning chars nothing sent; 189 tokens, 704 reasoning chars medium; 105 tokens, 396 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.36 nothing sent; 132 tokens, 466 reasoning chars nothing sent; 320 tokens, 927 reasoning chars medium; 93 tokens, 379 reasoning chars none; 4 tokens, 0 reasoning chars
2026.9.37 nothing sent; 270 tokens, 870 reasoning chars nothing sent; 181 tokens, 628 reasoning chars medium; 30 tokens, 57 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.1 nothing sent; 232 tokens, 878 reasoning chars nothing sent; 192 tokens, 591 reasoning chars medium; 169 tokens, 650 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.2 nothing sent; 130 tokens, 463 reasoning chars nothing sent; 200 tokens, 698 reasoning chars medium; 23 tokens, 40 reasoning chars none; 87 tokens, 0 reasoning chars
2026.10.3 nothing sent; 222 tokens, 822 reasoning chars nothing sent; 199 tokens, 744 reasoning chars medium; 34 tokens, 78 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.4 nothing sent; 118 tokens, 430 reasoning chars nothing sent; 377 tokens, 1483 reasoning chars medium; 99 tokens, 354 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.5 medium; 112 tokens, 412 reasoning chars nothing sent; 139 tokens, 525 reasoning chars medium; 32 tokens, 66 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.6 medium; 30 tokens, 69 reasoning chars nothing sent; 153 tokens, 470 reasoning chars medium; 65 tokens, 108 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.7 medium; 144 tokens, 530 reasoning chars nothing sent; 154 tokens, 527 reasoning chars medium; 134 tokens, 536 reasoning chars none; 3 tokens, 0 reasoning chars
2026.10.8 medium; 144 tokens, 457 reasoning chars nothing sent; 144 tokens, 515 reasoning chars medium; 27 tokens, 49 reasoning chars none; 91 tokens, 0 reasoning chars
2026.10.9 medium; 112 tokens, 451 reasoning chars nothing sent; 369 tokens, 1442 reasoning chars medium; 99 tokens, 388 reasoning chars none; 28 tokens, 0 reasoning chars
2026.10.10 medium; 64 tokens, 246 reasoning chars nothing sent; 97 tokens, 340 reasoning chars medium; 105 tokens, 259 reasoning chars none; 4 tokens, 0 reasoning chars
2026.10.33 medium; 25 tokens, 43 reasoning chars nothing sent; 95 tokens, 338 reasoning chars medium; 58 tokens, 186 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.1 medium; 19 tokens, 14 reasoning chars nothing sent; 132 tokens, 497 reasoning chars medium; 87 tokens, 307 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.2 medium; 102 tokens, 367 reasoning chars nothing sent; 263 tokens, 1063 reasoning chars medium; 38 tokens, 72 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.3 medium; 77 tokens, 255 reasoning chars nothing sent; 105 tokens, 418 reasoning chars medium; 78 tokens, 307 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.4 medium; 25 tokens, 42 reasoning chars nothing sent; 211 tokens, 797 reasoning chars medium; 366 tokens, 1593 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.5 medium; 88 tokens, 266 reasoning chars nothing sent; 147 tokens, 466 reasoning chars medium; 28 tokens, 59 reasoning chars none; 4 tokens, 0 reasoning chars
2026.11.6 medium; 19 tokens, 14 reasoning chars nothing sent; 892 tokens, 3639 reasoning chars medium; 470 tokens, 1732 reasoning chars none; 4 tokens, 0 reasoning chars

Not verified / blocked

  • Signed-installer upgrade matrix (windows-clean-installer-upgrade: verify-installer-signatures, verify-installed-payload, exercise-upgrade-matrix):
    • The inventory marks this pool maintainer-scheduled, with approval required.
    • It needs official signed installers for the old and new versions on a restorable machine, which a contributor build doesn't have.
    • Its contributor-runnable recipe (installer source assertions) passed above. The release's own setup and portable app were used for the upgrade itself.
  • A default-effort turn on the upgraded WSL install: blocked by the test laptop's GPU memory, as described above.

Security Impact

  • New permissions or capabilities? No.
  • Secrets or tokens handling changed? No new handling. A republished provider takes its API key from the companion's credential store, as on main. The native in-place patch changes only the models array, so the key stays as stored.
  • New or changed network calls? No.
  • Command or tool execution surface changed? No new commands. WSL uses the existing unset / restart / write sequence; native uses the existing config.patch.
  • Data access scope changed? No.

Compatibility and Migration

  • Backward compatible? Yes, measured on every stable Gateway release from 2026.6.1 to 2026.11.6 (Gateway versions). 2026.6.1 rejects the new fields, so it keeps the pre-upgrade entry and today's behavior. 2026.6.5 and newer accept them; Off sends none from 2026.8.1 (2026.6.5 – 2026.7.35 send it as high).
  • Config or environment changes? Qwen provider entries gain effort metadata; the router no longer forces reasoning.
  • Migration needed? Automatic. Only an exact pre-upgrade, companion-owned provider is upgraded; any other provider shape is left untouched. Stored companion data is unchanged. Rolling back to v2026.11.3 leaves the provider untouched.

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Nov 4, 2026
@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex review: needs changes before merge.

What this changes

This PR enables Effort Off for qualified Qwen models, migrates managed provider metadata, and preserves session effort during canvas actions.

Example: Select Effort Off with Qwen3.8 27B and ask “What is 19 times 21?”

  • Before: The forced reasoning preset ignores Off and generates reasoning.
  • After: On a compatible Gateway, the request sends reasoning_effort=none and returns an answer without reasoning.

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) Strong supplied product-path evidence supports the intended repair, but a concrete older-Gateway compatibility failure blocks merge.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (logs): Contributor evidence exercises the Windows ARM64 app with Qwen3.8 and native/WSL Gateways, shows Off and canvas request behavior, and records native models-only recovery plus v2026.11.3 upgrades retaining credentials and unrelated settings; inspected screenshots support the visible states and request/log evidence supports reasoning suppression.
Patch quality 🦪 silver shellfish (2/6) 1 actionable review finding remain.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Partial
User problem: Choosing Effort Off still spends time reasoning, unsupported effort levels can fail, and canvas actions change the user's selected effort.
Reason: Honoring an explicit user choice has clear value; retaining Related wording correctly leaves non-Qwen coverage in the broader issue.

Merge readiness

⛔ Needs changes before merge - 1 item remains

Keep this PR open for a focused compatibility repair; it addresses a confirmed Local AI defect that current main still contains.

Priority: P2
Reviewed head: f096746cd8bfd9307d6f104e9783d4fab849631c

Before merge

  • [P1] Check replacement compatibility before withdrawing the legacy route (P1) - A working managed Qwen route on Gateway 2026.6.1 is admitted here, but QuiesceAsync removes its provider and restarts Gateway before ApplyBatchAsync performs the replacement dry-run. That released Gateway advertises supported protocol 4 while its strict ModelDefinitionSchema rejects thinkingLevelMap. Publication therefore fails after the working route is gone, and retrying generates the same unsupported replacement. Normal startup also withdraws before publication, so checking only this direct migration branch would leave the same failure reachable. Corroborating joelagnel's report at fix: honor Effort Off for qualified Qwen local AI models #2012 (comment), validate installed-Gateway support before any startup or migration withdrawal, preserve the legacy route on rejection, and add a regression covering rejected replacement without removal.

Findings

  • [P1] [P1] Check replacement compatibility before withdrawing the legacy route — src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayProviderCoordinator.cs:194-195
Agent review details

How this fits together

Companion generates llama-server presets and managed Gateway provider configuration; Gateway translates chat and canvas effort choices into inference requests.

flowchart LR
 A[Chat effort choice] --> B[Gateway]
 C[Canvas action] --> B
 D[Model catalog] --> E[Provider coordinator]
 E --> B
 B --> F[llama-server]
 F --> G[Model response]
Loading

Technical review

Best possible solution:

Retain the focused effort fix and validate replacement compatibility through the existing lifecycle and publication owners before withdrawing any working legacy route.

Do we have a high-confidence way to reproduce the issue?

Current-main source establishes the effort defects, and contributor runtime evidence demonstrates them; the older-Gateway upgrade failure is source-reproducible against the released strict schema, without a live reproduction in this review.

Is this the best way to solve the issue?

The catalog-owned effort vocabulary and existing migration owners fit the bug, but replacement compatibility must be established before route withdrawal; recorded requests also resolve the previous default-effort concern by showing Medium before and after.

Full review comments:

  • [P1] [P1] Check replacement compatibility before withdrawing the legacy route — src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayProviderCoordinator.cs:194-195
    A working managed Qwen route on Gateway 2026.6.1 is admitted here, but QuiesceAsync removes its provider and restarts Gateway before ApplyBatchAsync performs the replacement dry-run. That released Gateway advertises supported protocol 4 while its strict ModelDefinitionSchema rejects thinkingLevelMap. Publication therefore fails after the working route is gone, and retrying generates the same unsupported replacement. Normal startup also withdraws before publication, so checking only this direct migration branch would leave the same failure reachable. Corroborating joelagnel's report at fix: honor Effort Off for qualified Qwen local AI models #2012 (comment), validate installed-Gateway support before any startup or migration withdrawal, preserve the legacy route on rejection, and add a regression covering rejected replacement without removal.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.97

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against aa2089bd64a0.

Provenance checked

  • Forced reasoning preset and provider metadata changes intended behavior with a stated reason (ccd64bb: The original managed Local AI implementation supplied a verified thinking-model runtime and Gateway route; the current PR explains why forcing reasoning prevents per-request Off.)
  • Canvas effort override changes intended behavior with a stated reason (9fa43f3: The existing comment explicitly matched Android's Low budget hint; the patch replaces that hint with session inheritance because Gateway persists the override.)
  • Recovery rollback and teardown keeps the original intent (652a20a: Non-destructive recovery preserves exact owned state and restores the original route only after verified compensation; legacy-shape recognition retains that boundary.)
  • Native ownership and guarded publication keeps the original intent (7dcbc2f: Native Local AI uses authenticated, revision-guarded configuration changes and exact ownership; the migration retains those owners and updates only the owned model array.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

Acceptance criteria:

  • ./build.ps1
  • dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj --no-restore
  • dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore
  • dotnet test ./tests/OpenClaw.Connection.Tests/OpenClaw.Connection.Tests.csproj --filter FullyQualifiedName~LocalAiPortLifecycleTests
  • dotnet test ./tests/OpenClaw.SetupEngine.Tests/OpenClaw.SetupEngine.Tests.csproj --filter FullyQualifiedName~NativeLocalAiLifecycleTests

What I checked:

Likely related people:

  • Joel: Raw commit ccd64bb adds src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayProviderCoordinator.cs:32 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: ccd64bbb7d68; files: src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayProviderCoordinator.cs)
  • Karen: Raw commit 7dcbc2f adds src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayLifecycle.cs:276 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 7dcbc2f798a8; files: src/OpenClaw.Tray.WinUI/Services/LocalAiGatewayLifecycle.cs)
  • Scott Hanselman: Raw commit 652a20a adds src/OpenClaw.SetupEngine/LocalAiGatewayConfiguration.cs:61 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 652a20aa01be; files: src/OpenClaw.SetupEngine/LocalAiGatewayConfiguration.cs)
  • Chris Anderson: Raw commit 9fa43f3 adds src/OpenClaw.Tray.WinUI/A2UI/Actions/GatewayActionTransport.cs:110 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: 9fa43f347703; files: src/OpenClaw.Tray.WinUI/A2UI/Actions/GatewayActionTransport.cs)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Review metrics

Metric Value Why it matters
Introduced change size Production/resources +214/-41; tests +406/-5; documentation +3/-1 Growth primarily preserves legacy ownership, native migration, WSL replacement, rollback, and teardown across distinct existing boundaries.

Labels

Label changes:

  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🦪 silver shellfish, so this older rating label is no longer current.
  • remove status: ⏳ waiting on maintainer: Current PR status label is status: ⏳ waiting on author.

Label justifications:

  • P2: The underlying effort-control bug affects managed Local AI, with a focused repair and limited product scope.
  • merge-risk: 🚨 compatibility: An older protocol-compatible Gateway rejects the added model field after the changed WSL path has withdrawn its working provider.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦞 diamond lobster and patch quality is 🦪 silver shellfish.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.
  • proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (9 earlier review cycles; latest 8 shown)
  • reviewed 2026-11-11T01:44:57.295Z sha f4b2dc0 :: needs real behavior proof before merge. :: [P2] Retain unconfirmed WSL migration application across publication retries
  • reviewed 2026-11-16T20:05:56.870Z sha 4ec8e74 :: needs real behavior proof before merge. :: none
  • reviewed 2026-11-16T23:58:11.297Z sha 4ec8e74 :: needs real behavior proof before merge. :: [P2] [P2] Give migration validation failures an actionable recovery step | [P3] [P3] Reuse the shared temporary-directory fixture
  • reviewed 2026-11-17T18:53:28.034Z sha ed6d88f :: needs real behavior proof before merge. :: none
  • reviewed 2026-11-18T00:17:08.974Z sha ed6d88f :: needs real behavior proof before merge. :: none
  • reviewed 2026-11-18T22:40:23.929Z sha 34f8085 :: needs real behavior proof before merge. :: none
  • reviewed 2026-11-18T23:44:21.402Z sha 34f8085 :: ready, awaiting a maintainer decision. :: none
  • reviewed 2026-11-19T18:13:10.689Z sha f096746 :: ready, awaiting a maintainer decision. :: none

Reviewed November 19, 2026, 5:18 PM ET / 22:18 UTC (Revision 10).

@turutupa turutupa changed the title fix: local AI ignores Effort Off and errors on unsupported levels fix: honor Effort Off for qualified Qwen local AI models Nov 4, 2026
@turutupa
turutupa marked this pull request as ready for review November 5, 2026 07:57
@clawsweeper

clawsweeper Bot commented Nov 5, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@roboclaw-bot

Copy link
Copy Markdown
Contributor

PR cleanup review

Reviewed head: 03fdd2fc2545f23d1ad7d4bb3ecd90e545208a70. Main comparison: e59374c064f43d68f5c72d070d4c89ef2d20ec3b.

Thanks for the follow-up work on legacy recovery, exact rollback and localized guidance. This still addresses behavior present on main: the router forces reasoning on, provider entries lack effort metadata, and canvas actions explicitly request Low, so I would keep the PR open.

The latest head needs Windows validation and product proof before landing. The currently visible successful checks are ClawSweeper dispatch jobs, and the substantive review targets the older d8f5e32 head.

One remaining source path needs attention: direct WSL Settings Start/Restart returns from LocalAiGatewayLifecycle.PrepareStartAsync after owner resolution without the new version check. Startup can therefore withdraw/restart the route and start the listener before publication rejects an older Gateway. The separate WSL setup/recovery batch writer also appears to publish the new metadata without enforcing the floor. Please cover these entry points with early compatibility admission and regression tests, while keeping cleanup available.

After reconciling with main, please provide exact-head Windows build/test results and native/WSL migration plus Effort/canvas proof. Maintainer confirmation of the 2026.9.8 floor, Qwen3.8 default change and deferred non-Qwen scope is also still needed.

Action in this cleanup pass: leaving this PR open. No branch changes or runtime-proof waiver. The findings above come from source, discussion and recorded-CI review; Windows validation was not rerun in this pass.

@turutupa
turutupa force-pushed the fix/local-ai-reasoning-1976 branch 2 times, most recently from e249e75 to f4b2dc0 Compare November 11, 2026 01:38
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. labels Nov 11, 2026
Effort Off still produced reasoning for managed Qwen models because the
llama-server preset forced `reasoning = on` and the Gateway provider had
no effort vocabulary to map Off onto.

- Drop `reasoning = on` from the router preset so llama-server's default
  auto mode lets each request turn reasoning off.
- Tag the qualified Qwen catalog entries with a reasoning profile
  (Qwen38 / QwenOnOff) and publish effort metadata for those templates
  only (supportsReasoningEffort, supportedReasoningEfforts,
  reasoningEffortMap off->none, thinkingLevelMap). Other templates keep
  the previous provider shape.
- Recognize the exact pre-upgrade provider shape in ownership, recovery,
  rollback and teardown (SetupEngine step, native lifecycle, coordinator)
  so an unchanged legacy route is not treated as externally edited.
- Migrate an exact legacy provider by replacing only its owned `models`
  array: guarded config.patch on native, dry-run then
  --expect-current-json write then restart on WSL. The API key is never
  replayed from a redacted readback.
- Native endpoint recovery keeps its early return for a current-shape
  route and continues through the guarded coordinator only for the
  exact legacy shape.
- Canvas actions no longer send `thinking: "low"`, so they inherit the
  session's effort.

No Gateway version gate: every field written has been in the Gateway
config schema since 2026.6.5. Gateways 2026.6.5 to 2026.9.7 accept it and
keep today's behavior (Off still reasons); older Gateways reject the
write through the existing guarded paths with no config change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@turutupa
turutupa force-pushed the fix/local-ai-reasoning-1976 branch from aa4bc38 to 4ec8e74 Compare November 16, 2026 18:36
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot removed the proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. label Nov 17, 2026
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

…igration

A provider in the exact pre-effort shape needs its model metadata upgraded.

WSL: the companion never edits it in place. Every runtime start (and every
non-ready refresh) quiesces the route first: the provider is unset, the
Gateway restarted and the removal read back before any publication, and the
llama-server router is in a kill-on-close job object, so it never outlives
the app. If a publication still finds the legacy shape, it is withdrawn the
same way and published from scratch (API key from the credential store,
never from a redacted readback); a failed withdrawal fails the publication
and writes nothing new.

Native: endpoint recovery deliberately keeps a verified live route and
passes the legacy provider to publication, so only the owned model array is
patched atomically and read back, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@turutupa
turutupa force-pushed the fix/local-ai-reasoning-1976 branch from ed6d88f to 34f8085 Compare November 18, 2026 22:33
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Nov 18, 2026
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on maintainer Only explicit maintainer approvals or decisions remain; merge is still blocked. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Nov 18, 2026
…hout effort metadata

From an independent review of the previous head:

- A native legacy migration whose write lands but whose readback does not
  match (here, a Gateway that drops thinkingLevelMap) now has a test. It
  reports the localized LocalAi_GatewaySavedRouteVerification failure,
  makes no second write, and keeps the upgraded route, its credential and
  the managed primary in place.
- The Bonsai receipt test now asserts what removing the forced
  `reasoning = on` means for a model without a reasoning profile: the
  preset uses llama-server's auto mode, and the provider entry carries no
  effort vocabulary, so it is unchanged from the pre-upgrade shape and the
  Gateway never requests "none" for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@turutupa

Copy link
Copy Markdown
Author

asking for a review @joelagnel

@joelagnel

Copy link
Copy Markdown
Collaborator

@turutupa could you check whether upgrading a legacy WSL Local AI route against an older Gateway can leave the working provider removed when the new metadata is rejected?

At the reviewed head (f096746c), the WSL migration withdraws the legacy provider and restarts the Gateway before publishing its replacement. The replacement batch's dry-run happens afterward in ApplyBatchAsync.

A concrete version to check is Gateway 2026.6.1: it advertises protocol 4, which passes the Companion compatibility policy, but its strict model schema does not include thinkingLevelMap. A focused schema probe accepted the legacy model shape and rejected the new shape with Unrecognized key: "thinkingLevelMap".

My reading is that publication would then fail after the old route has already been removed, and retrying would generate the same rejected configuration. Normal runtime startup also quiesces before publication, so this may extend beyond the direct migration branch.

Is there a minimum-version or capability guard elsewhere that rules this out? If not, could we check compatibility before withdrawal or preserve the legacy shape for those Gateways, with a regression test for replacement rejection? I have not reproduced this on a live WSL installation; this concern is based on source review and the schema probe.

@turutupa

Copy link
Copy Markdown
Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on maintainer Only explicit maintainer approvals or decisions remain; merge is still blocked. labels Nov 19, 2026
turutupa and others added 2 commits November 19, 2026 14:49
…tadata

The companion admits any protocol-4 Gateway, but the effort metadata this
PR publishes needs newer model schemas: supportsReasoningEffort arrived in
2026.1.14, supportedReasoningEfforts and reasoningEffortMap in 2026.4.26,
and thinkingLevelMap only in 2026.6.5. Gateway 2026.6.1-2026.6.4 therefore
rejects the new provider ("Unrecognized key"). PublishAsync withdrew a
pre-effort WSL provider (config unset + Gateway restart) before the
replacement was dry-run, and normal startup also withdraws on its endpoint
cycle before publishing (native: config.patch, same order). On those
Gateways the working route was removed, publication failed, and every retry
generated the same rejected config.

Decide the provider shape before anything is withdrawn or written:
- WSL: dry-run the full batch (new ValidateBatchAsync, dry-run only); if it
  is rejected, dry-run the exact pre-effort batch.
- Native: read-only config.schema check (SupportsEffortMetadataAsync); a
  Gateway that does not list config.schema is newer than 2026.6.5.
- Full shape when accepted; otherwise the pre-effort shape, as on main, with
  one warning that Effort Off cannot disable reasoning on that Gateway. A
  pre-effort provider there counts as current: no withdraw/republish churn
  and no in-place migration.
- If neither shape validates or the check fails, nothing is withdrawn and
  the operation fails with a localized message, keeping the current route.

An endpoint-cycle quiesce decides the shape and the following publication
reuses it; teardown needs no check. Publication, verification and native
migration use the decided shape. Setup's WSL step already dry-runs before
applying; it now falls back to the pre-effort batch when a dry-run confirms
the Gateway rejects the full one, and its rollback recognizes that shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An independent review of 7bdbfe9 found that endpoint-cycle quiesce ran the
provider-shape check before withdrawing. The runtime also withdraws through an
endpoint cycle to take the route off an untrusted or conflicting listener
(LlamaServerRuntimeService ownership-incomplete and conflict paths), so a
failed check (config.schema timeout, dry-run indeterminate) left the Gateway
routing prompts to that listener.

- Quiesce withdraws unconditionally again, as on main. The shape is decided
  only in PublishAsync, once per publication, before anything is written.
  This also removes the cached decision, so native quiesce and publish on
  separate coordinator instances no longer re-check after a withdrawal, and
  a failed quiesce cannot leave a stale shape behind.
- WSL treats the effort shape as rejected only when its dry-run fails twice
  and the pre-effort shape validates, so one failed CLI run cannot downgrade
  a working route.
- The rejected and check-failed messages now say nothing was written instead
  of claiming the current route was kept (all six languages).

Tests: endpoint-cycle withdrawal never waits on the check (WSL and native,
native publish through a separate coordinator), publication rejecting both
shapes writes nothing, and a one-off dry-run failure keeps the current shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@turutupa

Copy link
Copy Markdown
Author

@joelagnel thanks, you were right, and it reproduces on a real install.

Repro (Gateway 2026.6.1, managed WSL, fresh v2026.11.3 install, then this branch): at f096746c the startup withdrew the working route, the replacement was rejected, and Local AI ended Failed with no provider left. A dry-run on 2026.6.1 rejects this PR's entry (Unrecognized key: "thinkingLevelMap") and accepts the pre-upgrade one; 2026.6.5 accepts both.

Root cause: the companion admits any protocol-4 Gateway, but thinkingLevelMap only exists from 2026.6.5, and publication wrote the new entry without checking. My "no version floor" claim in the description was wrong; it's corrected now.

Fix (7bdbfe9d + 4c902b0b): every publication decides the provider shape before it writes anything.

  • WSL: a dry-run of the new entry (openclaw config set --batch-file … --dry-run), then of the pre-upgrade entry; a rejection has to repeat.
  • Native: a read-only config.schema check (config.patch has no dry-run).
  • A Gateway that rejects effort metadata keeps or gets the exact pre-upgrade entry. Local AI keeps working, and Off behaves as on main.
  • If neither shape validates, nothing is written. Setup's WSL step falls back the same way.
  • An independent review of 7bdbfe9d found that I had also gated withdrawals on that check. That would have left a route on an untrusted listener when the check failed. 4c902b0b makes withdrawal unconditional again, as on main.

Live, same machine:

Gateway f096746c 4c902b0b
2026.6.1 route removed, Local AI Failed pre-upgrade entry republished (same key and primary), Healthy, also on a second start
2026.11.6 upgraded to the effort fields, Healthy

I also measured every stable Gateway release from 2026.6.1 to 2026.11.6 (50), with both entries. Off sends none from 2026.8.1; 2026.6.5–2026.7.35 send it as high. The table and screenshots are in the description.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants