Repository navigation

batch: benchmark, scoreboard and README (#1340 #1352 #1380 #1365 #1338) — after #1384 - #1385

Merged

abrichr merged 157 commits into

mainfrom

batch/bench-scoreboard-docs

Oct 20, 2026

Merged

batch: benchmark, scoreboard and README (#1340 #1352 #1380 #1365 #1338) — after #1384#1385
abrichr merged 157 commits into
mainfrom
batch/bench-scoreboard-docs

Commits on Oct 18, 2026
- Merge remote-tracking branch 'origin/main' into feat/narration-rules
  abrichr
  committed
  49b60dd
- feat(capture): scrub narration and store it as timed sentences
  
  abrichr
  and
  claude
  committed
  8f90a16
- feat(flow): narration proposes branch guards; recordings or a person certify them
  
  abrichr
  and
  claude
  committed
  6da5aab
- feat(flow): probe the narrated guard on a test copy; induce and narration CLI
  
  abrichr
  and
  claude
  committed
  3824549
- feat(flow): show narrated guards in visualize and repair show; narration in teach-branch
  
  abrichr
  and
  claude
  committed
  f3e4a50
- docs: narration in capture and flow, capability row in review
  
  abrichr
  and
  claude
  committed
  3d5f3d5
- Merge remote-tracking branch 'origin/flow/probe-induction' into feat/narration-rules
  abrichr
  committed
  b6ec4e8
- fix(flow): convert a capture without narration when its file is refused
  
  abrichr
  and
  claude
  committed
  9058afb
- fix(flow): review findings on narrated guards
  
  abrichr
  and
  claude
  committed
  1b15d54
- flow: keep the server's answer to a save, labelled same_screen
  
  abrichr
  and
  claude
  committed
  e387c81
- Merge branch 'flow/desktop-readback' into flow/save-response-evidence
  abrichr
  committed
  27e3569
- flow: server answer evidence review fixes
  
  abrichr
  and
  claude
  committed
  354cbc3
- Merge branch 'flow/desktop-readback' into flow/save-response-evidence
  abrichr
  committed
  d5a2501
- Merge flow/desktop-readback; regenerate claims report and artifact inventory
  
  abrichr
  and
  claude
  committed
  21105d2
- Merge branch 'flow/desktop-readback' into flow/save-response-evidence
  abrichr
  committed
  ac1f101
Commits on Oct 19, 2026
- Merge remote-tracking branch 'origin/main' into feat/narration-rules
  abrichr
  committed
  86e2e3c
- Merge branch 'flow/desktop-readback' into flow/save-response-evidence
  abrichr
  committed
  c2a6f99
- Merge remote-tracking branch 'origin/main' into feat/narration-rules
  abrichr
  committed
  2c99d59
- fix(flow): lint and artifact inventory for narration
  
  abrichr
  and
  claude
  committed
  534265b
- Merge flow/desktop-readback; arm the server-answer log at the first save
  
  abrichr
  and
  claude
  committed
  0533f97
- Merge remote-tracking branch 'origin/main' into claude/mcp-teach-compile-a652e8
  abrichr
  committed
  665d8a8
- feat(agent): teach a workflow over MCP, then call it with no model calls
  
  abrichr
  and
  claude
  committed
  e8ec967
- docs: link the teach-over-MCP PR in CAPABILITIES.md
  
  abrichr
  and
  claude
  committed
  a8e1eb0
- Merge remote-tracking branch 'origin/claude/crystallize-from-prompt-89950e' into claude/mcp-teach-compile-a652e8
  abrichr
  committed
  568a111
- Merge remote-tracking branch 'origin/main' into feat/narration-rules
  abrichr
  committed
  d9dbaee
- docs: an agent's claims versus the read-back, 30 of 30 bad saves
  
  abrichr
  and
  claude
  committed
  e25c447
- scoreboard: OpenEMR claims runner (faults in the fixture database only: triggers and a poller)
  
  abrichr
  and
  claude
  committed
  97effbd
- fix(agent): bind each person's approval to one action; gate more off the web
  
  abrichr
  and
  claude
  committed
  328e4fb
- Fix four false stops in the extra identity looks
  
  abrichr
  and
  claude
  committed
  4a1d781
- Merge branch 'claude/hover-identity' into claude/pixel-identity-measure
  abrichr
  committed
  76eaaab
- Measure the extra identity looks and the hover look, before and after
  
  abrichr
  and
  claude
  committed
  f73e8e8
- Register the new evidence files in the public artifact inventory
  
  abrichr
  and
  claude
  committed
  e103f68
- ci: pin two reviewed gitleaks false positives (key_dir in MCP teach tests)
  
  abrichr
  and
  claude
  committed
  41dacc2
- Merge remote-tracking branch 'origin/main' into flow/save-response-evidence
  abrichr
  committed
  a3ad5e0
- Merge branch 'claude/hover-identity' into claude/pixel-identity-measure
  abrichr
  committed
  9f097a3
- Mention the check-to-click guard in CAPABILITIES.md
  
  abrichr
  and
  claude
  committed
  5af281c
- Merge origin/main into guard branch (brings in API steps #1269)
  abrichr
  committed
  ca1020d
- bench: silent-saves, score any GUI executor on injected bad saves
  
  abrichr
  and
  claude
  committed
  e4d5e81
- flow: list the silent-saves data in the public artifact inventory
  
  abrichr
  and
  claude
  committed
  dacf2c4
- WIP: seal every bundle and recording file under one index
  
  abrichr
  and
  claude
  committed
  a94ffbe
- Propose undo steps and verify them on a test copy
  
  abrichr
  and
  claude
  committed
  1a73a54
- Merge remote-tracking branch 'origin/main' into claude/pixel-identity-measure
  abrichr
  committed
  d66be00
- Regenerate the public artifact inventory after merging main
  
  abrichr
  and
  claude
  committed
  4ef7f70
- Merge remote-tracking branch 'origin/main' into flow/save-response-evidence
  abrichr
  committed
  275c7f1
- undo propose: form-post and GraphQL reasons, offline rows, undo list hint
  
  abrichr
  and
  claude
  committed
  bf6c96f
- Merge remote-tracking branch 'origin/main' into claude/gracious-chandrasekhar-43a0c5
  abrichr
  committed
  835803f
- WIP: sealed runs, CLI guard, at-rest tests
  
  abrichr
  and
  claude
  committed
  1ceaf08
- undo propose: say what would close each step's gap; offline OpenEMR/openIMIS rows
  
  abrichr
  and
  claude
  committed
  b518949
- feat(flow): guard an agent you already run (commit guard over CDP, run-2 handoff)
  
  abrichr
  and
  claude
  committed
  54d9962
- WIP: keep compile, induce, qualify, admit, scaffold-verifier and undo output sealed
  
  abrichr
  and
  claude
  committed
  bbc452f
- test(flow): agent guard benchmark harness, e2e test, docs draft
  
  abrichr
  and
  claude
  committed
  9774017
- WIP: sealed resume, attended state, nested undo runs
  
  abrichr
  and
  claude
  committed
  59a45e6
- undo propose: fix test-copy snapshot identity; live MockLoan rows
  
  abrichr
  and
  claude
  committed
  929f205
- scoreboard: timed OpenEMR worklist runs (arms 3, 4, 6), per-run breakdown in the tables
  
  abrichr
  and
  claude
  committed
  b46f941
- undo propose benchmark: Markdown table output
  
  abrichr
  and
  claude
  committed
  7a9b20d
- undo propose: results tables in docs, regenerate artifact inventory
  
  abrichr
  and
  claude
  committed
  b7e1617
- CAPABILITIES: link #1346
  
  abrichr
  and
  claude
  committed
  43aa5ea
- scoreboard: Amendment 4 (worklist runs, self-loaded arm 6), per-run engine and throughput lines
  
  abrichr
  and
  claude
  committed
  89bfeed
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  72343f6
- WIP: docs, measurements, record --no-encrypt
  
  abrichr
  and
  claude
  committed
  06c5af1
- Register the at-rest benchmark results in the public artifact inventory
  
  abrichr
  and
  claude
  committed
  9cdba9a
- WIP: bind-record-checks keeps a sealed bundle sealed; approve on sealed runs
  
  abrichr
  and
  claude
  committed
  6fe77bf
- fix(flow): guard on its own connection, recorder on its own thread; run 2 end to end
  
  abrichr
  and
  claude
  committed
  60eebbb
- bench(silent-saves): proof images, reference runs, detection apart from prevention
  
  abrichr
  and
  claude
  committed
  af6f39d
- flow: list the silent-saves proof and reference runs in the public artifact inventory
  
  abrichr
  and
  claude
  committed
  deece7a
- Merge remote-tracking branch 'origin/main' into claude/cool-tesla-63b116
  abrichr
  committed
  a9c4bd0
- scoreboard: Amendment 5 (quiet-window overlap; per-row load gate held)
  
  abrichr
  and
  claude
  committed
  7f16c5a
- Seal look-up check proposals in encrypted recordings; refresh llms-full
  
  abrichr
  and
  claude
  committed
  7c04617
- bench(flow): measure the agent guard on MockMed; docs and capability map
  
  abrichr
  and
  claude
  committed
  6db638f
- chore(flow): regenerate the public artifact inventory for the agent guard results
  
  abrichr
  and
  claude
  committed
  6c7f103
- style: format the agent guard report
  
  abrichr
  and
  claude
  committed
  f8a88ef
- docs: link the agent guard PR in the capability map
  
  abrichr
  and
  claude
  committed
  f045824
- docs: state run 2's profile and the recorded reason no API step was proposed
  
  abrichr
  and
  claude
  committed
  42cec77
- flow: threat model and prompt-injection controls for teach and repair
  
  abrichr
  and
  claude
  committed
  8339efd
- docs: link #1359 in the capability map
  
  abrichr
  and
  claude
  committed
  1e8683f
- bench(silent-saves): fault set 1.1.0, repeated runs under UI change
  
  abrichr
  and
  claude
  committed
  1c6f425
- bench(silent-saves): arms 2 and 3r for fault set 1.1.0
  
  abrichr
  and
  claude
  committed
  031836f
- bench(silent-saves): arm 2 run on fault set 1.1.0; keep the item's fault through admission
  
  abrichr
  and
  claude
  committed
  3091155
- scoreboard: arm 5 (OpenEMR API step, #1269) and its interleaved API-off runs, Amendment 6
  
  abrichr
  and
  claude
  committed
  dad5d57
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  cac6cf4
- undo propose: live MockMed row
  
  abrichr
  and
  claude
  committed
  2e2e176
- Regenerate public artifact inventory
  
  abrichr
  and
  claude
  committed
  bfa7d55
- Merge remote-tracking branch 'origin/main' into claude/gracious-chandrasekhar-43a0c5
  abrichr
  committed
  67eafc2
- scoreboard: OpenEMR agent/screen/API comparison section, exact profile labels per run, claims runs shown only when complete
  
  abrichr
  and
  claude
  committed
  77687d9
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  b4fcb9d
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  eefb618
- docs: README leads with the measured OpenEMR times and the 30 of 30 claims result
  
  abrichr
  and
  claude
  committed
  2bdbd2a
- docs: scope the API step to single HTML form submissions
  
  abrichr
  and
  claude
  committed
  0de133d
- scoreboard: the agent's claims versus verify-run on OpenEMR (faults in the fixture database), Amendment 7
  
  abrichr
  and
  claude
  committed
  1ec521c
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  b7db4e3
- docs: the OpenEMR claims result, 25 of 30 caught, and its collateral-edit gap
  
  abrichr
  and
  claude
  committed
  1e1d83e
- scoreboard: arm 7 timed pass (10 of 10 correct, 1 row timed), n beside every median, Amendment 8
  
  abrichr
  and
  claude
  committed
  fb0e85a
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  16d65f9
- verify-run: a record outside the target that changed is collateral, not only one that vanished (#1373)
  
  abrichr
  and
  claude
  committed
  48b0ad0
- bench(silent-saves): match MockMed's fault hook line exactly, not with a regex
  
  abrichr
  and
  claude
  committed
  e98bedc
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  afdc517
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  528e0e4
- bench(silent-saves): match MockMed's fault hook line exactly, not with a regex
  
  abrichr
  and
  claude
  committed
  4497323
- verify-run collateral change: compare values in canonical form (a baseline's nested keys come back sorted)
  
  abrichr
  and
  claude
  committed
  9850b62
- Replayer undo hook: accept no effects (mypy)
  
  abrichr
  and
  claude
  committed
  26ef5db
- Fix CI: mypy in sealed_set, test doubles without encryption hooks, CodeQL findings, gitleaks false positive
  
  abrichr
  and
  claude
  committed
  fe5b410
- Merge remote-tracking branch 'origin/main' into claude/cool-tesla-63b116
  abrichr
  committed
  4ba1401
- Merge remote-tracking branch 'origin/main' into claude/gracious-chandrasekhar-43a0c5
  abrichr
  committed
  5796e05
- bench(silent-saves): per-item timeout, arm 1 run and hand checks, 1.1.0 table generator
  
  abrichr
  and
  claude
  committed
  bb7c97f
- Merge remote-tracking branch 'origin/main' into flow/save-response-evidence
  abrichr
  committed
  34ceb6d
- bench(silent-saves): keep arm 2's admission records with its results
  
  abrichr
  and
  claude
  committed
  4e6878c
- Merge origin/main into claude/mcp-teach-compile-a652e8
  
  abrichr
  and
  claude
  committed
  13e7a80
- Merge remote-tracking branch 'origin/claude/mcp-teach-compile-a652e8' into claude/mcp-teach-compile-a652e8
  abrichr
  committed
  54924e6
- Merge branch 'claude/hover-identity' into claude/pixel-identity-measure
  abrichr
  committed
  a3c2cac
- Merge remote-tracking branch 'origin/main' into claude/pixel-identity-measure
  abrichr
  committed
  6e1b36c
- undo_proposal: narrow selector value type (mypy)
  
  abrichr
  and
  claude
  committed
  1ae6440
- Merge remote-tracking branch 'origin/main' into claude/silly-mirzakhani-0bfadc
  abrichr
  committed
  5a67d8c
- Merge origin/main into the agent guard branch
  
  abrichr
  and
  claude
  committed
  36a5fb7
- scoreboard: re-judge OpenEMR collateral_edit and clean items with #1375 on rebuilt state (5/5 caught, 0 false alarms); claims scripts never read a stale receipt; Amendment 9
  
  abrichr
  and
  claude
  committed
  c38aea4
- flow: refresh the public artifact inventory
  
  abrichr
  and
  claude
  committed
  a0794ce
- scoreboard claims: pass the item's values to verify-run on stdin, never in a file (CodeQL py/clear-text-storage-sensitive-data); label the re-judge block 'rebuilt state, judged by fixed verify-run'
  
  abrichr
  and
  claude
  committed
  c72d21f
- test(flow): don't load mcp when collecting the MCP teach tests
  
  abrichr
  and
  claude
  committed
  3553c9b
- verify-run collateral change: compare driver values with their stored JSON form; say it may be another user's edit
  
  abrichr
  and
  claude
  committed
  2a302c6
- bench(silent-saves): fault set 1.1.0 results for arms 1, 1+read-back, 2, 3 and 3r
  
  abrichr
  and
  claude
  committed
  250d3c8
- flow: list the silent-saves 1.1.0 data in the public artifact inventory
  
  abrichr
  and
  claude
  committed
  68ae7fc
- Merge remote-tracking branch 'origin/bench/silent-saves' into bench/silent-saves-change
  abrichr
  committed
  b1d52d2
- Merge remote-tracking branch 'origin/main' into claude/silly-mirzakhani-0bfadc
  abrichr
  committed
  1aac2b3
- Merge remote-tracking branch 'origin/main' into claude/cool-tesla-63b116
  abrichr
  committed
  a0ee939
- Merge remote-tracking branch 'origin/main' into flow/save-response-evidence
  abrichr
  committed
  6a0daa9
- Merge origin/main into claude/mcp-teach-compile-a652e8
  
  abrichr
  and
  claude
  committed
  4999d26
- Merge origin/main into the agent guard branch (split files)
  
  abrichr
  and
  claude
  committed
  9c67a6d
- Merge #1335's updated branch into the agent guard branch
  
  abrichr
  and
  claude
  committed
  623b614
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  b7f5eaa
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  aa3757a
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  e8637f1
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  bf8e244
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  c674230
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  dc7d7d8
- Merge remote-tracking branch 'origin/main' into HEAD
  abrichr
  committed
  c3103a3
- bundle validation: omit forbid_collateral_change=False from sealed v2 content
  
  abrichr
  and
  claude
  committed
  a1f06b1
- docs: public copy lists only shipped work; test for status words
  
  abrichr
  and
  claude
  committed
  ee0f348
- Merge #1375 into batch safety
  abrichr
  committed
  d20023e
- Merge #1354 into batch safety
  abrichr
  committed
  cc1b237
- Merge #1298 into batch safety
  abrichr
  committed
  e784cf8
- Merge #1291 into batch safety
  abrichr
  committed
  e968b2d
- Merge #1346 into batch safety
  abrichr
  committed
  b7a25ea
- batch safety: regenerate generated files
  abrichr
  committed
  a7cb4e5
- batch safety: regenerate inventory, verification report, command list and llms-full
  abrichr
  committed
  8db3caa
- Merge #1359 into batch safety
  abrichr
  committed
  89e4eb9
- Merge #1335 into batch safety
  abrichr
  committed
  ae0de30
- Merge #1357 into batch safety
  abrichr
  committed
  3cad704
- batch safety: regenerate generated files
  abrichr
  committed
  2757585
- batch safety: public copy lists only shipped work
  
  abrichr
  and
  claude
  committed
  97537a1
- Merge #1384 into batch bench
  abrichr
  committed
  ba8e74b
- Merge #1340 into batch bench
  abrichr
  committed
  81ba390
- Merge #1352 into batch bench
  abrichr
  committed
  95b808c
- Merge #1380 into batch bench
  abrichr
  committed
  111f2a5
- Merge #1365 into batch bench
  abrichr
  committed
  b91c4af
- batch bench: regenerate generated files
  abrichr
  committed
  5805055
- batch bench: regenerate inventory, verification report, command list and llms-full
  abrichr
  committed
  dfc320b
- Merge #1338 into batch bench (batch 2's identical copy edits kept)
  abrichr
  committed
  99374a6
- batch bench: shipped-only rows; regenerate generated files
  
  abrichr
  and
  claude
  committed
  aeeb915

Commits

Commits on Oct 18, 2026

Commits on Oct 19, 2026