Skip to content

Retire the pr workflow Bats suite where Go covers it #923

Open

Description

@tbhb-agent

Problem

tests/skills/pr/ holds the workflow Bats suite: 52 cases in about 699 lines that exercise the pr entry scripts end to end (preflight, review, publication, scaffold, validation) against the frozen shell. Once child 12 forwards those entry points to Go and children 01 to 09 carry Go tests for the same behavior, most of these cases test the shell twice and slow the Bats jobs on Ubuntu and macOS. Each case needs a recorded decision: covered by a named Go test and removed, or kept because it proves something only the real scripts can show.

Part of #588 and #323. Changes only tests/skills/pr/ workflow suite files and the opt-in flag RUN_HEADLESS_REVIEWS. No shell body, library or generated file changes.

Acceptance criteria

  1. AC1: Given the 52 cases before the change, when the change lands, then a table in the pull request lists every case with its disposition: removed with the name of the Go test that covers it and the name of the retained frozen-path case that covers the same shell outcome, or kept with the reason (including "frozen shell outcome with no other coverage: kept until the shell is removed").
  2. AC2: Given each of the five forwarded entry scripts, when the suite runs, then at least one frozen-path case per script remains, running the script with a binary that lacks the behavior, and no case is removed whose only coverage of a supported frozen-shell outcome (a refusal, an exit status, a message, a file written) is itself.
  3. AC3: Given the pre-push hook and a real git push to a bare remote, when the suite runs, then the cases that exercise them are kept.
  4. AC4: Given the forwarding cases for exec, argument pass-through and exit-status forwarding, when the suite runs, then each remains at least once for each forwarded entry script.
  5. AC5: Given RUN_HEADLESS_REVIEWS, when the suite runs with it unset, then the delegation case that uses the deterministic reviewer still runs as today (it is not skipped), and when it is set, then the retained live-review cases also run; the flag is documented where it is read.
  6. AC6: Given the Bats jobs on Ubuntu and macOS, when they run on the merged head, then they pass and the suite's case count is lower than 52 by the number of removed cases.
  7. AC7: Given every removed case, when its behavior is read in the Go tests it names, then the Go test asserts the same outcome (exit status and message), checked by a reviewer for each row of the table.
  8. AC8: Non-functional: no shared helper under tests/skills/pr/ is changed by this pull request.

Scope and non-goals

Scope: the workflow suite under tests/skills/pr/ and the table of dispositions.

Non-goal: the compatibility, section-pipes and writer suites, which child 15 retires. Non-goal: the agent-pr suites and the shared helper, which child 16 handles. Non-goal: changing or deleting any shell. This pull request deletes only Bats cases: no script, library or generated copy is removed, and no owner-tier path in .agents/agent-skills.conf is touched (those go to the single cutover issue under #588). Non-goal: removing cases whose behavior has no Go test, or whose frozen-shell outcome has no other retained coverage; those stay and are listed.

Approach

Meet AC1 to AC8 by building the disposition table first, with each Go test name resolved to an existing test from children 01 to 09, then removing the covered cases in one pull request. Cases with no Go counterpart stay and are listed as follow-ups.

Dependencies

Blocked by: #921.

Prerequisite: children 01 to 09 merged (reached through child 12).

Size

M. One pull request, mostly deletions: about 699 lines of Bats at most are touched, plus the table in the pull request body; under the code line limit.

Verify plan

  • AC1, AC2, AC7: the disposition table in the pull request, checked row by row at review, including the frozen-path coverage column.
  • AC2, AC3, AC4: the remaining case list shows the named cases, and the Bats job passes on both operating systems.
  • AC5: the suite run with the flag unset (asserting the deterministic-reviewer delegation case ran and passed) and set (asserting the live-review cases ran).
  • AC6: the Bats jobs on Ubuntu and macOS on the merged head, with the case counts before and after.
  • AC8: the diff path listing shows no helper file.
  • Definition of done: D1 each AC is verified at the merged head, with the commands and results cited in the pull request's Verification. D2 an independent review approves the exact merged head, and every finding has a recorded disposition. D3 required checks pass with no gate loosened. D4 the leak scans the repository specifies are clean, and every post made for this work passes the per-post leak review before it is posted. D5 every commit is signed. D6 the documentation matches the change or the Verification says why there is no impact. D7 every commit has an Assisted-by trailer and the pull request has the provenance:agent-assisted label. D8 the pull request stays within the size limits. D9 follow-ups are filed as issues and listed under Related. D10 the merge closes the issue through a closing reference, the board shows Done, and the final Status: comment names the merge commit.

Risks

A removed case may have covered a behavior the Go tests miss, which the row-by-row review in AC7 is meant to catch. Undo: revert the change; the cases return and run against the forwarders.

Operator authority

None.

Activity

  1. tbhb-agent commented on Oct 27, 2026

    @tbhb-agent
    ContributorAuthor

    Refinement: READY — Retire the pr workflow Bats suite where Go covers it
    Model: gpt-6-astra

    Child 14 Retire the pr workflow Bats suite where Go covers it

    Verdict: READY

    Checked: Each removed case must now identify retained coverage of the same frozen-shell outcome. Deterministic default review and enabled live review are distinguished correctly. The current workflow baseline is 52 cases and 699 physical lines. Real-push coverage and helper exclusions remain explicit. Evidence: tests/skills/pr/scripts/pr-workflow.bats:133, tests/skills/pr/scripts/pr-workflow.bats:387, tests/skills/pr/scripts/pr-workflow.bats:402.

    Gaps: None.

    Split: M remains plausible. Cases without equivalent retained frozen-path coverage stay.

    Definition of done

    The definition of done for this issue (Chore). Tick each item and give its evidence.

    • D1 The author verified each AC on the merged head, and the Verification section of the pull request cites each AC ID with its evidence: the command and its result, or a link.
    • D2 An independent reviewer approved the exact merged head, and the author resolved each finding or filed it as a follow-up.
    • D3 The required checks pass on that head, and any loosened gate has the owner's approval of that exact head.
    • D4 The leak scans come back clean: gitleaks, devtools' check-leaks.sh with the repository's private-strings list, and the leak review of each post.
    • D5 The author signed each commit, the squash included.
    • D6 The docs match the change (README, docs/, CHANGELOG or help text), or the Verification section says "No doc impact" and why.
    • D7 The commits and the squash have Assisted-by trailers, and the pull request has the provenance:agent-assisted label.
    • D8 The pull request stays within the size limits, or the owner approved its exact head over the hard limit.
    • D9 The author files follow-ups and deferred findings as issues with at least a Problem and numbered AC and lists them under the Related section of the pull request. Any work left in the code links its issue.
    • D10 The merge closes the issue through a closing reference and the board shows Done. The final Status: comment names the merge commit.
    • devtools-1 mise run check is green, and so are the coverage floors mise run cover-go-check and mise run cover-py-check, which check doesn't run.
    • devtools-2 A change to skill sources under packages/agents/source/ commits what mise run generate-skills, then mise run skill-tokens, then apm install rewrite, in that order, and check-generated-skills (part of lint) passes.
    • devtools-3 A consumer-visible change (the vendored payload, a skill, a reusable workflow, a task or a pin) has its consumer steps in docs/agent-skills-migration.md or the adoption docs, or the PR's Verification says "No consumer impact" and why.

    Round: 3 of 3

  2. added theissue type on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    refinement:readyPassed the Definition of Ready; may be picked up

    Type

    Fields

    Priority

    None yet

    Work type

    None yet

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions