Repository navigation
Insist on native tool calls in the dialog prompts - #290
Merged
Merged
Conversation
The dialog model still writes about a quarter to two fifths of its tool
calls as reply text (react_to_message(...), <action><tool_call>...,
draw_image{...}) instead of native tool_calls.
The rule was a single sentence at the end of the system prompt. Now the
tool mode line, the tool contract, the silent tool call rule, the reaction
rule and the resample notes all state that a tool runs only through
native tool_calls and that a call written as text does nothing. The
shared core adds its lines only when tools are available.
Owner
Author
|
Re Danger's prompt warning: the shape is preserved intentionally.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The dialog model (Gemma on NInfer GPU0) still writes many tool calls as reply text instead of native
tool_calls. In the last three days of dialog requests with stored raw responses, 23–42% of tool calls per day were text: code formreact_to_message(emoji=…),<action><tool_call>…markup and Gemma's baredraw_image{…}. The bot salvages some of these, but each one risks a lost action, a rejected turn or a broken argument (see #288).The rule existed as one sentence in
<tool_contract>at the end of the system prompt.Change
aifarm/system.prompt: the native<tool_mode>line says tools are called only through nativetool_calls, never as reply text.<tool_contract>states the rule as a hard rule without exceptions, names the forbidden written forms in general terms, says that a written call does nothing, and covers every tool, includingreact_to_messageandsend_message.chat/_shared_core.prompt:<silent_tool_calls>and<semantic_reactions>repeat the rule, only whenhasToolsis true, so the tool-less prompt stays unchanged.aifarm/resample_note.prompt: theprotocol_only,no_tool_callsandunexecuted_actionretry notes ask for nativetool_callsinstead of "as described in the catalog".The prompts do not quote any concrete text-call syntax, so they give the model no form to copy.
Not changed: the memory prompts. Their tool is the engine's structured-output mechanism (
response_formatsent as a forcedstructured_responsecall); they ask for one JSON object, the same prompt serves the Gemini schema path, and 820 of 820 responses in the last day were native.The system prompt prefix changes, so the dialog prefix cache warms up again once after deploy.
Tests
New test
native_tool_calls_rule_appears_only_when_tools_are_availablerenders the system prompt with and without tools.cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo test -p openplotva-prompts(10 passed)cargo test -p openplotva-llm(225 passed, 2 ignored)cargo test -p openplotva-dialog(109 passed)cargo test -p openplotva-app(1460 passed, 7 ignored)