Skip to content

Run CLI summaries tool-free and require explicit opt-in - #431

Merged
spalen0 merged 6 commits into
mainfrom
cli-guardrails
Oct 21, 2026
Merged

spalen0 merged 6 commits into
mainfrom
cli-guardrails

Conversation

@spalen0

@spalen0 spalen0 commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

  • Make LLM_CLI_PROVIDERS=none the default. Operators explicitly choose codex, claude, or an ordered list such as codex,claude; only listed CLIs are tried before the API provider.
  • Run both CLIs as text-only summarizers. The explainer already supplies decoded calls, verified source, on-chain state, simulation and protocol adapter context, so the agent needs no tools.
    • Codex: replace --yolo with --sandbox read-only, --ignore-user-config, --ignore-rules, disabled shell/browser/apps/plugins/hooks/multi-agent/image features, and web_search="disabled". No MCP servers load.
    • Claude: replace --dangerously-skip-permissions with --permission-mode dontAsk, --tools "" and --strict-mcp-config with no MCP config, so no built-in tools and no MCP servers.
  • Give the CLI subprocess only process basics and its own authentication variables. RPC, Etherscan, Telegram, Tenderly and application LLM API credentials stay out of the child environment.
  • Audit Codex JSONL and Claude stream JSON after each run. Any tool event disables that provider for the process and falls through to the next one. Claude's internal StructuredOutput event (schema calls only) and Codex todo_list plan items are not tool use. Non-JSON warning lines are skipped; output without events fails only that call.
  • Error messages for a non-zero exit quote stderr only, never the event stream.
  • Pass cli_auth_credentials_store="auto" to Codex: --ignore-user-config also drops that setting, and the file-only default misses a login kept in the OS keyring (401).
  • Docs, .env.example and the systemd unit state the new none default.

If a summary turns out to need more chain data, add the read to the explainer's context gathering rather than giving the agent a tool.

Dependency audit

The certifi fix for PYSEC-2026-4320 (pin certifi==2026.10.19, certifi-only exclude-newer-package override, no audit ignore) is split out to #432 so main goes green first. This branch carries identical pyproject.toml, uv.lock and CI files, so merging main after #432 is conflict-free and leaves only the CLI changes in this diff.

Validation

  • uv run pytest -q tests/test_llm_cli.py: 40 passed (tool audit, environment allowlist, stderr-only errors, warning-line tolerance, fallback and factory wiring).
  • Ruff lint/format and ty pass.
  • Live local runs of both CLIs through the providers: asked to run pwd, each answered NO_TOOLS; schema-constrained output works on both.

Rollout

Keep LLM_CLI_PROVIDERS=none in production until this version is deployed. Enable one CLI explicitly for a trial, check the AI summary using <model> logs, then choose the order.

Both CLIs still run as the monitoring service's Unix user, and the restrictions are CLI flags; event auditing detects unexpected tool calls after the fact and cannot undo one.

🤖 Generated with Claude Code

@spalen0 spalen0 changed the title Guard CLI summaries with tool-free execution and explicit opt-in Guard CLI summaries with read-only chain tools and explicit opt-in Oct 21, 2026
spalen0 and others added 2 commits October 21, 2026 10:31
The explainer already gathers decoded calls, source, state and simulation
before the LLM call, so the CLIs run as text-only summarizers. This removes
the monitoring_chain MCP server, the mcp dependency and the RPC/Etherscan
credentials passed to the CLI child.

- Any tool event disables that CLI (Claude StructuredOutput on schema calls
  and Codex todo_list plan items excepted).
- Non-JSON output lines are skipped; output with no events fails only that
  call instead of disabling the provider.
- Non-zero exit errors quote stderr only, never the event stream.
- Pass cli_auth_credentials_store="auto" to Codex: --ignore-user-config drops
  it and the file-only default misses keyring logins (401).
- Docs, .env.example and the systemd unit now state the default is "none".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spalen0 spalen0 changed the title Guard CLI summaries with read-only chain tools and explicit opt-in Run CLI summaries tool-free and require explicit opt-in Oct 21, 2026
spalen0 and others added 2 commits October 21, 2026 13:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@spalen0
spalen0 merged commit b3adb37 into main Oct 21, 2026
3 checks passed
@spalen0
spalen0 deleted the cli-guardrails branch October 21, 2026 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant