Skip to content

docs(agents): replace the Review record with a Reviewer Brief - #778

Merged
vishal-bala merged 3 commits into
mainfrom
feat/reviewer-brief
Oct 26, 2026
Merged

vishal-bala merged 3 commits into
mainfrom
feat/reviewer-brief

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

The Review record introduced in #767 turned out to be an audit log of the author's review: what was fixed, what was rejected, how many findings were raised and why each lens ran. On the first PRs that carried one, it ran to 190–560 words, and the parts a reviewer acts on were buried in the middle. A reviewer sees the change as it is now and never saw the problems the review fixed, so most of the record cost reading time without changing what they did. This PR replaces it with a Reviewer Brief that points the reviewer at the current code.

Changes

The Reviewer Brief

references/review-record.md becomes references/reviewer-brief.md. The brief has a fixed order:

  • Start here: the one file or function the rest of the change hangs off.
  • Where it could fail: up to three spots in the final diff, each with a file and line.
  • Not verified: what wasn't run or exercised.
  • Beyond CI: evidence CI can't show.
  • Footer: one line naming the reviewed commit, the agent and model, and the lenses that ran.

It deliberately leaves out fixed findings, rejected findings, counts, lens reasons and open questions. A new step in SKILL.md has the agent settle every open question with the author before the PR is opened or marked ready, so the PR reaches the reviewer with none. An agent that can't reach the author leaves the PR as a draft and reports the questions to the author instead. The reference also adds short guidance on writing for a reviewer who never saw the review: lead with the consequence, use names the reviewer can see in the diff, and cut anything the diff or CI already says. Its example is adapted from a real PR's brief.

Six lenses by default

SKILL.md now runs all six lenses by default. The trigger table is kept as guidance for authors who want to save tokens. The diff step also covers uncommitted and untracked work. Fixes for review comments that change behaviour now need a fresh review rather than a one-line note.

Helper scripts

Two steps that agents kept doing by hand are now scripts that use only the Python standard library and gh:

  • scripts/update_brief.py replaces only the brief section of a PR description. It keeps Bugbot's summary, replaces an older ## Review record heading, and copes with Windows line endings, which some PR descriptions use. tests/unit/test_pre_pr_review_scripts.py pins those cases.
  • scripts/open_feedback.py lists unresolved review threads through GraphQL, which is the only place resolution state is exposed. It also lists review summaries and top-level comments.

Smaller changes

  • evals/evals.json holds three prompts for comparing runs with and without the skill.
  • The PR template gains an optional Notes section, where deliberate trade-offs are recorded as decisions with their reasons.
  • AGENTS.md, CONTRIBUTING.md, the PR template and .cursor/BUGBOT.md now describe the Reviewer Brief.

Reviewer Brief

Start here: .agents/skills/pre-pr-review/references/reviewer-brief.md. It defines what agents write for you, and SKILL.md, AGENTS.md, CONTRIBUTING.md, the PR template and BUGBOT.md all point at it.

Where it could fail

  • .agents/skills/pre-pr-review/scripts/update_brief.py:22–25: the section ends at the next ## heading or at Bugbot's summary. A description with a ## line inside a code block below the brief would lose everything from the brief down to that line.
  • .agents/skills/pre-pr-review/SKILL.md:63–69: the new step 6 makes an unattended agent stop at draft whenever it has an unanswered question. That's deliberate, but it can hold back a PR whose only open point is minor.
  • .agents/skills/pre-pr-review/references/reviewer-brief.md:51–52: these two rules decide when a brief still applies after a push. Agents will follow them literally.

Not verified: open_feedback.py on a PR with unresolved threads (none were available); the evals were not run; agents other than Claude Code.

Beyond CI: a separate agent ran the revised skill on the first commit, and a second reviewed the removal of the decisions section. All eleven of their kept findings are fixed. The Windows line-ending test fails against the script without the fix. This brief was written into the description by update_brief.py.

Reviewed the working tree on 8d620ca (fixes in e2a3777) and 5c1e806 (fixes in 62daab7) · Claude Code (claude-opus-5-5) · lenses: correctness, security, design, tests, docs, Redis behaviour


Note

Low Risk
Changes are limited to agent skill docs, contributor guidance, and stdlib gh helpers with unit tests; no RedisVL library runtime behavior changes.

Overview
Replaces the pre-PR Review record (audit log of fixes, rejections, and lens rationale) with a Reviewer Brief aimed at human approvers: where to start, decisions, up to three failure spots, what was not verified, and optional beyond-CI evidence—without replaying problems already fixed in the branch.

The skill workflow in SKILL.md now defaults to all six review lenses, broadens diff scoping to uncommitted/untracked work, and documents update_brief.py (surgical PR description updates, including legacy ## Review record headings and Bugbot blocks) and open_feedback.py (GraphQL-based open review thread listing). references/review-record.md is removed in favor of references/reviewer-brief.md; lens references rename “Not checked” to Not verified and adjust challenger guidance so rejections surface in the brief only when a reviewer might disagree.

AGENTS.md, CONTRIBUTING.md, the PR template, and BUGBOT.md are aligned on the new section name and how to read it. Adds evals/evals.json for skill regression prompts and unit tests for replace_brief() in tests/unit/test_pre_pr_review_scripts.py.

Reviewed by Cursor Bugbot for commit e2a3777. Bugbot is set up for automated code reviews on this repo. Configure here.

The Review record read as an audit log of the author's review: what was
fixed, what was rejected, finding counts and lens reasons. A reviewer
sees the change as it is now, so most of that cost reading time without
changing what they do.

The Reviewer Brief points the reviewer at the current code instead:
where to start reading, what they need to decide, up to three places
the change could fail, what was not verified, and evidence CI cannot
show. A one-line footer names the reviewed commit, the agent and model,
and the lenses that ran. The reference adds short guidance on writing
for a reviewer who never saw the review.

All six lenses now run by default, with the triggers kept as a way to
save tokens. Two helper scripts handle steps agents kept repeating:
update_brief.py replaces only the brief section of a PR description,
keeping Bugbot's summary and coping with Windows line endings, and
open_feedback.py lists unresolved review threads through GraphQL. An
evals file holds three prompts for comparing runs with and without the
skill.
@vishal-bala vishal-bala added the auto:documentation Changes only affect the documentation label Oct 20, 2026
@vishal-bala
vishal-bala marked this pull request as ready for review October 20, 2026 15:29
Drop "Decisions for you" from the Reviewer Brief. The brief is written
for the reviewer, but the questions it held were the author's to
answer, so the section mixed two audiences.

The skill gains a step before the brief is written: take anything the
review could not settle to the author and get an answer before the PR
is opened or marked ready, so it reaches the reviewer with no open
questions. A deliberate trade-off goes in the author's Notes section as
a decision with its reason.
Add an optional Notes section to the PR template, where deliberate
trade-offs are recorded as decisions. Apply the settle-questions step
to every rewrite of the brief, not only the first, and say what an
agent that cannot reach the author does: leave the PR as a draft and
report the questions to the author instead of the PR. List the step in
AGENTS.md's summary, make clear which brief sections are always
present, and add an eval expectation for it.
@vishal-bala
vishal-bala merged commit 31d2d0a into main Oct 26, 2026
58 checks passed
@vishal-bala
vishal-bala deleted the feat/reviewer-brief branch October 26, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:documentation Changes only affect the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant