Skip to content

The merge check judge gives the shell's verdicts without changing checks evaluate #863

Description

@tbhb-agent

Problem

The merge scripts judge a pull request's check rollup with mp_judge_checks and mp_superseded in pull-request.sh, a second implementation beside agentrepo checks evaluate. The two already agree on grouping runs by check type, workflow and name and on treating unstarted runs as the latest. They differ on malformed input, unrecognized states (the shell reads them as ok, checks evaluate rejects them), runs with equal start times, how status contexts are chosen (the shell judges each context as it stands, checks evaluate groups them) and the output and status contract (per-check lines plus an aggregate status). The merge commands need the shell's observable result without changing what checks evaluate returns to its existing consumers. This is under the shell freeze in #588.

Part of #324, within #588. Child 21 in #324's Children list. Replaces the check-judge part of packages/agents/source/skills/merge-pr/shared/scripts/pull-request.sh.

Acceptance criteria

  1. AC1: Given a check rollup, when the merge judge runs, then it prints one line per judged check with a verdict (ok, pending or fail, as pull-request.sh emits), the state it read and the check's name, and its aggregate status is 0 when every check is ok, 1 when any failed, 2 when any is still running and none failed, 3 when the rollup could not be read and 4 when no checks were reported, matching the shell judge on the same fixtures.
  2. AC2: Given superseded runs of one check, cancelled runs replaced by later ones, a cancel with no later run, and queued re-runs without a start time, when it is judged, then the latest run of each workflow and check name stands for its group, an unstarted run counts as the latest, a lone cancel fails, and the count of earlier runs left out equals the shell's count.
  3. AC3: Given two runs of one check with equal start times and differing verdicts, when it is judged in each input order, then each ordering matches the frozen shell's result for that same ordering (max_by(started) keeps the last tied entry, so the two orderings may legitimately differ). Order-independent selection of the worse verdict, as internal/checks/checks.go:267 does for checks evaluate, is not part of this port; it is a Go fix filed as a follow-up after the port.
  4. AC4: Given status contexts, when it is judged, then each context is judged as it stands, and a state outside the shell's known failure and waiting sets reads as ok, as in the frozen judge.
  5. AC5: Given an empty rollup or an unreadable one, when it is judged, then the result is status 4 or 3 respectively, never success.
  6. AC6: Given the existing consumers of agentrepo checks evaluate, when this child is merged, then the output, exit status and tests of that command are unchanged.
  7. AC7: Non-functional: the judge runs against fixtures on macOS, Linux and Windows in CI.

Scope and non-goals

Scope: the merge-caller check judge and superseded count with their fixtures, ported from the check cases in merge-workflow.bats.

Non-goal: changing agentrepo checks evaluate behavior or tightening the shell's treatment of unrecognized states, which would be a Go fix after the port. Non-goal: reading the rollup from GitHub (child 04). Non-goal: editing the shell scripts before child 14.

Approach

Meet AC1 to AC7 with failing cases first: record the shell judge's lines and status on each fixture, then match them. Mechanism, including whether the existing checks code is reused, is the implementer's choice, subject to AC6.

Dependencies

Blocked by: #333.

Prerequisites: #321 (shared readers) and #333.

Size

S. One pull request.

Verify plan

  • AC1, AC2, AC4, AC5: golden comparison of lines, status and superseded count against the shell judge on rollup fixtures, one per case in the criteria.
  • AC3: tied-start fixtures with differing verdicts, each run in both input orders, each order compared with the shell judge's result for that same order; the follow-up issue for order-independent selection is filed and linked in the pull request.
  • AC6: the existing tests of agentrepo checks evaluate run unchanged at the merged head, with the diff showing no change to them.
  • AC7: the CI matrix run.
  • Definition of done: D1 each AC is verified at the merged head, with the commands and results cited in the pull request's Verification. D2 an independent review approves the exact merged head, and every finding has a recorded disposition. D3 required checks pass with no gate loosened. D4 the leak scans the repository specifies are clean, and every post made for this work passes the per-post leak review before it is posted. D5 every commit is signed. D6 the documentation matches the change or the Verification says why there is no impact. D7 every commit has an Assisted-by trailer and the pull request has the provenance:agent-assisted label. D8 the pull request stays within the size limits. D9 follow-ups are filed as issues and listed under Related. D10 the merge closes the issue through a closing reference, the board shows Done, and the final Status: comment names the merge commit.

Risks

A judge that reads an empty or unreadable rollup as green would let a merge through; AC5 holds both as non-success. Changing checks evaluate would alter other consumers; AC6 holds that. Undo: revert; nothing calls the judge until children 05 and later.

Operator authority

None.

Activity

  1. tbhb-agent commented on Oct 27, 2026

    @tbhb-agent
    ContributorAuthor

    Refinement: READY — The merge check judge gives the shell's verdicts without changing checks evaluate
    Model: gpt-6-astra

    Child 21 The merge check judge gives the shell’s verdicts without changing checks evaluate

    Verdict: READY

    Checked: AC3 and its verification now compare each tied-run ordering with the corresponding frozen-shell result. They no longer require order independence. The distinction matches packages/agents/source/skills/merge-pr/shared/scripts/pull-request.sh:88 and the existing Go selection at internal/checks/checks.go:267. Changing checks evaluate remains excluded.

    Gaps: None.

    Split: None needed.

    Definition of done

    The definition of done for this issue (Feature). Tick each item and give its evidence.

    • D1 The author verified each AC on the merged head, and the Verification section of the pull request cites each AC ID with its evidence: the command and its result, or a link.
    • D2 An independent reviewer approved the exact merged head, and the author resolved each finding or filed it as a follow-up.
    • D3 The required checks pass on that head, and any loosened gate has the owner's approval of that exact head.
    • D4 The leak scans come back clean: gitleaks, devtools' check-leaks.sh with the repository's private-strings list, and the leak review of each post.
    • D5 The author signed each commit, the squash included.
    • D6 The docs match the change (README, docs/, CHANGELOG or help text), or the Verification section says "No doc impact" and why.
    • D7 The commits and the squash have Assisted-by trailers, and the pull request has the provenance:agent-assisted label.
    • D8 The pull request stays within the size limits, or the owner approved its exact head over the hard limit.
    • D9 The author files follow-ups and deferred findings as issues with at least a Problem and numbered AC and lists them under the Related section of the pull request. Any work left in the code links its issue.
    • D10 The merge closes the issue through a closing reference and the board shows Done. The final Status: comment names the merge commit.
    • devtools-1 mise run check is green, and so are the coverage floors mise run cover-go-check and mise run cover-py-check, which check doesn't run.
    • devtools-2 A change to skill sources under packages/agents/source/ commits what mise run generate-skills, then mise run skill-tokens, then apm install rewrite, in that order, and check-generated-skills (part of lint) passes.
    • devtools-3 A consumer-visible change (the vendored payload, a skill, a reusable workflow, a task or a pin) has its consumer steps in docs/agent-skills-migration.md or the adoption docs, or the PR's Verification says "No consumer impact" and why.

    Round: 1 of 3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    refinement:readyPassed the Definition of Ready; may be picked up

    Fields

    Priority

    None yet

    Work type

    None yet

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions