[irvingpop]

Summary

Records the phase 3 live results, corrects four spec claims that the live tests disproved, and switches all three review passes to Sonnet 5.5. The spec (docs/claude-review-design.md) and the build plan hold the detail. Phase 3 is not closed: V26 a and c are partial, and every case that needs account X is NOT YET RUN.

Live results

Target: claude-review-test, ruleset pinned to S1 3aa7ae9 and its merge M1 590a387. Times are UTC, 2026-10-27.

Verification	Result	Evidence
Release eval at S1	Pass, 13 of 13	run 40528717338
M1 tree check	Pass, S1 and M1 have the same tree	git diff S1 M1 is empty
V26 a	Partial: R1 and R2 created at the placeholder and accepted; no read-back recorded	design item 26
V26 b	Pass: token at S1, denied at the probe SHA	runs 40560632129, 40561015054
V26 c, simulation (R2, S2, M2)	Partial: tokens at S2 and M2; no R2 read-backs; no denied exchange at M2	runs 40585724589, 40587964524
Deny at M1 before the second edit	Pass	run 40583290922
Accept at M1 after the second edit	Pass (M1 exchange passed; phase 3 exit criteria remain open)	run 40585413003
V22 a, V22 b	Pass, with a rollback residual	design "Which run counts"
V11, V12	Pass	design V11, V12
Outage: invalid federation rule	Pass, two denied exchanges	runs 40561015054, 40583290922
Outage: exhausted budget	Covered by V9 and unit tests	design V9
Acceptance list without account X	Pass for the listed cases; unit-test-only cases excluded	design acceptance table

Spec corrections

1. The merge box counts only runs at the pinned SHA. A forward pin move blocks at once; a move back can revive an older pass, so start a new run on each open PR after any pin move (new residual in rollout step 7).
2. Repo Actions settings do not affect the required run (V12).
3. A normal cancel before the gate job starts does not stop it, so it writes a pass summary next to a cancelled run; the run still blocks. Force-cancel stopped the gate job after the summary (run 40565910355). Normal cancel while the gate job runs was not tested.
4. Artifacts of earlier attempts are gone after a re-run. The gate reads only its own attempt, so it still fails closed; earlier pass records cannot be read back.

Sonnet 5.5

All three passes (review, security, supply-chain) use claude-sonnet-5-5. The model is now a per-pass setting. Effort, output limits, schemas and gate code are unchanged. This takes effect at the next release, not at merge: production still pins M1.

13-case fixture eval, same prompts:

Run	Matched	Total tokens vs Opus	Output tokens vs Opus	Wall time
Opus (40528717338)	13/13	1.00	1.00	138 s
Sonnet 1 (40589761270)	13/13	0.97	0.71	84 s
Sonnet 2 (40590079545)	13/13	0.98	0.76	82 s

All 39 pass results per Sonnet run had status ok. Input, cache write and cache read ratios are 0.99 to 1.00.

Not yet run

• Account X (non-member fork author): V8, V23, V6 f as a Write user, V25, and the web UI "Re-run all jobs" case.
• V26 a: read R1 and R2 back.
• V26 c remaining steps: R2 read-backs at the placeholder, S2 and M2; a denied exchange at M2 before the R2 edit to M2.
• The four unit-test-only acceptance cases (gate cancelled after inline comments, approver role removed, edited override, artifact upload removed).

Decisions the owner may want to reverse

• Sonnet 5.5 for all three passes: one line in the model map per pass restores Opus for any of them.
• No budget-lowering Console edits (Q7): the exhausted-budget case rests on V9 and unit tests.
• One PR for the results and the model switch: they could be split, but the spec text covers both.

🤖 Generated with Claude Code

[github-actions]

OC Claude review: fail

Result for run 40591583153, attempt 1.
Computed for head SHA 44cb4ff69dd5f788292f2c887b28647971b05ba5, merge base 590a387333c999a5d30dad8022a00383251ce16a, base ref main.
The result that counts is the required workflow in the merge box.

Reasons

• job-result (not waivable): dependency-review: failure
• dependency-review (not waivable): job result failure
• dependency-review (not waivable): step outcome failure
• pass-records (not waivable): pass review failed: authentication failed
• pass-records (not waivable): pass security failed: authentication failed
• pass-records (not waivable): pass supply-chain failed: authentication failed

Prefilter (code, no LLM)

Hard blocks: 0.
Signals: 2.

• long-line: lines longer than 500 characters, in docs/build-plan.md, lines 58 95 157 168
• long-line: lines longer than 500 characters, in docs/claude-review-design.md, lines 175 206 351 514 517 540 631 735 741 763 and more
  Coverage: complete (files in comparison 4, files collected 4, trees complete).

Dependency delta

No dependency changes.

Findings per pass

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

review: failed, authentication failed

No verified findings.

security: failed, authentication failed

No verified findings.

supply-chain: failed, authentication failed

No verified findings.

Token usage

• review: no usage recorded
• security: no usage recorded
• supply-chain: no usage recorded
• total: no usage recorded

Workflow run. The result that counts is the required workflow in the merge box.