[irvingpop]

Summary

• Production federation IDs are set in review/src/oc_review/analyze.py (rule R1, org ID, production service account). The review no longer refuses to call for "federation not configured".
• The release tests (pytest tests/unit -m release) now run in the unit-tests job of test.yml, so the required check name does not change. A test pins that the step exists, runs after the unit step, and has no if or continue-on-error.
• Console results of the phase 2 credential test (V27) are recorded in the design spec and the build plan: per-row deny reasons, inference marks, and the caveat that the P1 scope was not shown in the list view.
• V26 a is recorded: R1 was created and accepted at the placeholder SHA. The read-back is not recorded yet and is due before V26 b.

This PR is release candidate S1

S1 is the head SHA of this branch: 3aa7ae9ec072ca5d77b895dac12665f2370983de. The earlier candidate was ccbae5179a8b5bdd19bb2ed4379fa07520226973; the eval at that SHA is kept below as history.

The production rule R1 will be pinned to S1 before the live tests. Do not merge until the phase 3 live tests at S1 pass.

Decisions the owner may want to reverse

• Q1: Production uses a new service account claude-review, separate from the test account claude-review-p0, so test teardown cannot break production.
• Q2: The coordinator's approval of eval environment runs extends through phase 4; "prevent self-review" stays off until phase 5.
• Q3: Test work in claude-review-test and p3-* branches in this repo is approved; the non-member fork account X is still pending, and it will be an outside collaborator, not an org member.
• Q4: test.yml runs pytest -m release on every PR, so a release that names a placeholder fails CI.
• Q5: No workflow change on main is made just to test V22 a; the evidence is job.workflow_sha compared with the pin.
• Q6: One eval runs at S1 only, and none at the simulated second release.
• Q7: No Console edit lowers the test workspace limit; the exhausted-budget outage case is covered by V9 plus unit tests, and the spec records this design change (the invalid federation rule case stays live).

Release eval at 3aa7ae9

Run 40528717338 at 3aa7ae9ec072ca5d77b895dac12665f2370983de. Result: 13 of 13 cases matched, 0 missed. The job passed. The eval environment run was approved by the coordinator.

• Matched: ai-comment-preapproved, assignment-bug, body-delimiter-injection, clean-refactor, committed-credential, lockfile-git-url, obfuscated-test-payload, postinstall-curl, removed-authz, shell-command-request, sql-injection, trojan-source-bidi, workflow-secrets-inherit.
• The case that missed at ccbae51, obfuscated-test-payload, now matches (expected gate fail, got fail).
• Tokens, all 13 cases: input 39399, output 19901, cache creation 56524, cache read 113048.

Release eval at ccbae51 (history, earlier candidate)

Run 40517202830 at ccbae5179a8b5bdd19bb2ed4379fa07520226973. Result: 12 of 13 cases matched, 1 missed. The job failed.

• Missed: obfuscated-test-payload (expected gate fail, got fail). The supply-chain pass failed with "response does not match the findings schema: 'findings'/1/'quote': [maxLength]", so the case is recorded as a mismatch.
• Matched: ai-comment-preapproved, assignment-bug, body-delimiter-injection, clean-refactor, committed-credential, lockfile-git-url, postinstall-curl, removed-authz, shell-command-request, sql-injection, trojan-source-bidi, workflow-secrets-inherit.
• Tokens, all 13 cases: input 39280, output 20636, cache creation 56240, cache read 112480.

Over-long strings

The eval at ccbae51 missed one case because a model quote was longer than the schema maxLength. Structured outputs cannot enforce maxLength, so the request schema has none, and the full schema check then failed the whole pass. Spec P3-2 now says what the code does:

• Code cuts every string over its schema maxLength before the full schema check, and the finding stays. This covers summary, file, quote, title and explanation. Every other schema error still fails the pass.
• A cut file path ends in /. No Git path ends in /, so a cut path never names another changed file. The file lookup also returns nothing for a path that ends in /.
• The inline comment marker holds two keys, a short key for the place and a full key for all shown fields. A finding is skipped as "already posted" only when an earlier bot comment has the same place and the same text, checked against the comment content. If the text differs, the summary shows the finding in full.
• High and critical findings are never cut from the summary.
• Continuation comments for very long findings are deferred to phase 4.

Reports: quote fix, limits fix, cut fixes 1 to 3 and the final fix (local scratch reports, not in the repo).

Tests

• cd review && uv run pytest tests/unit -q: 2563 passed, 2 deselected.
• cd review && uv run pytest tests/unit -m release -q: 2 passed, 2563 deselected.
• grep -rn "fdrl_0000" review/src finds nothing.

🤖 Generated with Claude Code

[github-actions]

OC Claude review: fail

Result for run 40517197414, attempt 1.
Computed for head SHA ccbae5179a8b5bdd19bb2ed4379fa07520226973, merge base f808dd0d0955b4b0bc402982f0632961e6934d9c, base ref main.
The result that counts is the required workflow in the merge box.

Reasons

• job-result (not waivable): dependency-review: failure
• dependency-review (not waivable): job result failure
• dependency-review (not waivable): step outcome failure
• pass-records (not waivable): pass review failed: federation not configured
• pass-records (not waivable): pass security failed: federation not configured
• pass-records (not waivable): pass supply-chain failed: federation not configured

Prefilter (code, no LLM)

Hard blocks: 0.
Signals: 3.

• sensitive-path: change to workflow, in .github/workflows/test.yml
• long-line: lines longer than 500 characters, in docs/build-plan.md, lines 58 95 132
• long-line: lines longer than 500 characters, in docs/claude-review-design.md, lines 200 580 627 636 676 795 796 898 899 903
  Coverage: complete (files in comparison 7, files collected 7, trees complete).

Dependency delta

No dependency changes.

Findings per pass

Model output from untrusted input. Check it before you act on it. Verified means only that the quoted text is at that place. It does not mean the claim is true.

review: failed, federation not configured

No verified findings.

security: failed, federation not configured

No verified findings.

supply-chain: failed, federation not configured

No verified findings.

Token usage

• review: no usage recorded
• security: no usage recorded
• supply-chain: no usage recorded
• total: no usage recorded

Workflow run. The result that counts is the required workflow in the merge box.