Skip to content

Retire the compatibility, section and writer Bats suites where Go covers them #924

Open

Description

@tbhb-agent

Problem

Three smaller Bats suites under tests/skills/pr/ and tests/skills/writers/ also exercise behavior that children 01 to 03 and 11 now cover in Go: the compatibility declaration cases, the section-pipes cases (including the large-section-list regression), and the writer cases that run the description writers. After child 12 they test the shell twice. Only the writer suite's live probes are guarded by RUN_HEADLESS_WRITERS; its manifest permission, context-loading and workflow cases run without it.

Part of #588 and #323. Changes only those suite files. No shell body, shared helper or generated file changes.

Acceptance criteria

  1. AC1: Given the compatibility, section-pipes and writer cases before the change, when the change lands, then a table in the pull request lists every case with its disposition: removed with the name of the Go test that covers it and the name of the retained frozen-path case that covers the same frozen-validator or frozen-writer outcome, or kept with the reason (including "frozen shell outcome with no other coverage: kept until the shell is removed").
  2. AC2: Given the large-section-list regression cases (valid, missing, empty and reordered with 96 extra long sections), when the suite runs, then they are removed only because the Go test named for child 03 asserts the same four outcomes.
  3. AC3: Given RUN_HEADLESS_WRITERS unset, when the writer suite runs, then its live probes are skipped as today while the manifest permission, context-loading and workflow cases still run; given it set, then the retained live probes also run; the flag is documented where it is read.
  4. AC4: Given the frozen validator path, when the suite runs, then at least one compatibility case remains that runs validate-description.sh with a binary lacking the behavior, and no compatibility, section or writer case is removed whose only coverage of a supported frozen-validator or frozen-writer outcome (a finding, an exit status, a message) is itself.
  5. AC5: Given the Bats jobs on Ubuntu and macOS, when they run on the merged head, then they pass.
  6. AC6: Non-functional: the shared helper tests/skills/pr/test_helper.bash is not changed by this pull request.

Scope and non-goals

Scope: the three suites above and the table of dispositions.

Non-goal: the workflow suite (child 14) and the agent-pr suites with the helper trim (child 16). Non-goal: changing any shell. This pull request deletes only Bats cases: no script, library or generated copy is removed, and no owner-tier path in .agents/agent-skills.conf is touched (those go to the single cutover issue under #588). Non-goal: removing cases with no Go counterpart or whose frozen-shell outcome has no other retained coverage.

Approach

Meet AC1 to AC6 by building the disposition table first, resolving each Go test name to an existing test from children 01, 02, 03 and 11, then removing the covered cases in one pull request.

Dependencies

Blocked by: #921, #922.

Prerequisite: children 01 to 03 and 11 merged (reached through children 12 and 13).

Size

S. One pull request, mostly deletions, under the code line limit.

Verify plan

  • AC1, AC2: the disposition table in the pull request, checked row by row at review, including the frozen-path coverage column.
  • AC3: the writer suite run with the flag unset (asserting the permission, context-loading and workflow cases ran and the probes skipped) and set (asserting the probes ran).
  • AC4: the retained case list, the table's frozen-path coverage column and the passing Bats job.
  • AC5: the Bats jobs on Ubuntu and macOS on the merged head.
  • AC6: the diff path listing shows no helper file.
  • Definition of done: D1 each AC is verified at the merged head, with the commands and results cited in the pull request's Verification. D2 an independent review approves the exact merged head, and every finding has a recorded disposition. D3 required checks pass with no gate loosened. D4 the leak scans the repository specifies are clean, and every post made for this work passes the per-post leak review before it is posted. D5 every commit is signed. D6 the documentation matches the change or the Verification says why there is no impact. D7 every commit has an Assisted-by trailer and the pull request has the provenance:agent-assisted label. D8 the pull request stays within the size limits. D9 follow-ups are filed as issues and listed under Related. D10 the merge closes the issue through a closing reference, the board shows Done, and the final Status: comment names the merge commit.

Risks

A removed case may cover an input the Go tests miss; the row-by-row review catches that. Undo: revert the change; the cases return.

Operator authority

None.

Activity

  1. tbhb-agent commented on Oct 27, 2026

    @tbhb-agent
    ContributorAuthor

    Refinement: READY — Retire the compatibility, section and writer Bats suites where Go covers them
    Model: gpt-6-astra

    Child 15 Retire the compatibility, section and writer Bats suites where Go covers them

    Verdict: READY

    Checked: Per-case frozen-path coverage closes the retirement gap. Only live writer probes skip by default; permission, context-loading and workflow cases remain active. The large-section regression and helper boundary are correct. Evidence: tests/skills/pr/scripts/section-pipes.bats:7, tests/skills/writers/permissions.bats:10, tests/skills/writers/permissions.bats:24, tests/skills/writers/permissions.bats:102.

    Gaps: None.

    Split: S remains plausible for eligible deletions.

    Definition of done

    The definition of done for this issue (Chore). Tick each item and give its evidence.

    • D1 The author verified each AC on the merged head, and the Verification section of the pull request cites each AC ID with its evidence: the command and its result, or a link.
    • D2 An independent reviewer approved the exact merged head, and the author resolved each finding or filed it as a follow-up.
    • D3 The required checks pass on that head, and any loosened gate has the owner's approval of that exact head.
    • D4 The leak scans come back clean: gitleaks, devtools' check-leaks.sh with the repository's private-strings list, and the leak review of each post.
    • D5 The author signed each commit, the squash included.
    • D6 The docs match the change (README, docs/, CHANGELOG or help text), or the Verification section says "No doc impact" and why.
    • D7 The commits and the squash have Assisted-by trailers, and the pull request has the provenance:agent-assisted label.
    • D8 The pull request stays within the size limits, or the owner approved its exact head over the hard limit.
    • D9 The author files follow-ups and deferred findings as issues with at least a Problem and numbered AC and lists them under the Related section of the pull request. Any work left in the code links its issue.
    • D10 The merge closes the issue through a closing reference and the board shows Done. The final Status: comment names the merge commit.
    • devtools-1 mise run check is green, and so are the coverage floors mise run cover-go-check and mise run cover-py-check, which check doesn't run.
    • devtools-2 A change to skill sources under packages/agents/source/ commits what mise run generate-skills, then mise run skill-tokens, then apm install rewrite, in that order, and check-generated-skills (part of lint) passes.
    • devtools-3 A consumer-visible change (the vendored payload, a skill, a reusable workflow, a task or a pin) has its consumer steps in docs/agent-skills-migration.md or the adoption docs, or the PR's Verification says "No consumer impact" and why.

    Round: 3 of 3

  2. added theissue type on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    refinement:readyPassed the Definition of Ready; may be picked up

    Type

    Fields

    Priority

    None yet

    Work type

    None yet

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions