[spalen0]

Summary

The AI explainer now tries local agent CLIs before the metered API:

1. Codex CLI: codex exec --yolo
2. Claude Code CLI: claude -p --dangerously-skip-permissions (bypass-permissions mode)
3. API provider: the existing LLM_PROVIDER / LLM_API_KEY config, used when both CLIs fail (no credits, auth, timeout) or aren't installed

How it works

• utils/llm/cli_provider.py adds CodexCLIProvider and ClaudeCLIProvider.
  • The prompt goes in on stdin.
  • Structured output uses each CLI's native schema flag: --output-schema for Codex, --json-schema for Claude (which returns structured_output). The risk_tag enum is enforced the same way as on the API path.
  • Every call runs in a fresh temp dir with --ephemeral / --no-session-persistence / --strict-mcp-config, so this repo's AGENTS.md/CLAUDE.md and MCP servers never reach the analysis.
  • Codex has no system-prompt flag, so the system prompt is prepended in a <system_instructions> block. Claude gets it via --system-prompt-file.
• utils/llm/fallback.py adds FallbackProvider, which tries providers in order.
  • Process-level failures (non-zero exit, timeout, missing binary, Claude is_error) raise a new LLMUnavailableError. That provider is then disabled for the rest of the run, so a CLI with no credits doesn't add latency to every stage of every alert.
  • A bad response (empty text, invalid JSON) only falls through for that one call.
  • model_name reports the provider that actually answered, so the AI summary using … log line is accurate.
• factory.py builds the chain.
  • CLIs not on PATH are skipped.
  • A single available provider is returned unwrapped.
  • LLM_API_KEY is now optional when a CLI exists.

New env vars: LLM_CLI_PROVIDERS (default codex,claude; none turns the CLIs off), LLM_CLI_TIMEOUT (default 300s), LLM_CODEX_MODEL, LLM_CLAUDE_MODEL. When the model vars are unset, each CLI uses its own configured default.

Context parity with the API

The CLIs get exactly the same per-transaction prompt and system prompts as the API path. Host-level agent config is stripped where the CLI supports it, verified live:

• Claude: --setting-sources "" --disable-slash-commands --strict-mcp-config. No user rules, CLAUDE.md, skills or MCP servers reach the model.
• Codex: every server from codex mcp list --json is disabled by name (--config mcp_servers.<name>.enabled=false), plus --disable apps --disable plugins. mcp_servers={} turned out to be a no-op merge. Listing failures and names the -c parser can't address fail closed: Codex is skipped. Verified live: a probe stdio server starts when enabled and doesn't when disabled. A regression test runs the real codex binary against a synthetic config. Codex has no supported switch for the global ~/.codex/AGENTS.md or ~/.agents/skills, and both still load. Keep them empty for the user that runs monitoring.

⚠️ Things to decide before merging

• Prompt injection with approvals off. Both CLIs run as full agents with approvals and sandbox disabled, as requested. The prompt contains attacker-influenced text: proposal descriptions, verified source, labels. An injection could run shell commands as the monitoring user, which has access to .env secrets.
  • Suggested hardening, if acceptable: for Claude, --tools "" disables tools entirely. It never asks for approval either way, so nothing hangs. Codex would need --sandbox read-only instead of --yolo.
  • Happy to switch to either.
• VPS deploy is unchanged, so prod still uses the API. deploy/systemd/monitoring.service sets PATH=<repo>/.venv/bin:/usr/local/bin:/usr/bin:/bin, which excludes ~/.local/bin, and sets ProtectHome=read-only. Both CLIs need write access to ~/.codex / ~/.claude.
  • Turning this on in prod needs a unit change (PATH + ReadWritePaths) plus a logged-in CLI on the host. I left that out because it loosens the unit's hardening.
• Venice credit alert (credits.py) still runs. Venice is the last-resort fallback, so it should stay funded.

Test plan

• uv run pytest -q tests: 1825 passed, 4 skipped. That includes the new tests/test_llm_cli.py (CLI argv/stdin/schema handling, error classification, fallback ordering/disabling, factory chain wiring). The existing factory tests now patch shutil.which so they don't depend on installed CLIs.
• ruff check, ruff format --check, ty check are clean.
• Live, using the real SUMMARY_SCHEMA and system prompts:
  • Codex answers structured (~13s) and text (~6s).
  • With CODEX_HOME pointing at an empty dir (401), Codex is disabled after one failure and Claude answers both calls.
  • With Codex broken and LLM_CLI_PROVIDERS=codex, it falls through to Venice.

🤖 Generated with Claude Code