Skip to content

ci: run repository gate checks in one Bazel analysis - #4638

Merged
ScriptedAlchemy merged 3 commits into
masterfrom
ci/gates-one-bazel-analysis
Oct 31, 2026
Merged

ScriptedAlchemy merged 3 commits into
masterfrom
ci/gates-one-bazel-analysis

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Why

Repository gates paid a separate Bazel analysis for each Bazel step, and Feature gates paid a whole hosted runner for one more. Baseline run 41016935321 (PR #4626):

Job Step Elapsed Of which analysis/setup Critical path
Repository gates Test repository policies with Bazel workers 71 s 65 s 6 s
Repository gates Check formatting (bazel build //:rustfmt_check) 37 s 35 s 2 s
Feature gates setup-bazel + bazel test //scripts:gen_builds_test 52 s + 85 s 52 s + 83 s 1 s

What

  • Repository gates runs //:rustfmt_check, every //scripts:*_test policy test, gen_builds_test, and release_pr_refresh_test in one bazel test invocation, so one analysis serves actions that now run side by side.
  • Feature gates becomes a pass-through job that requires Repository gates. It keeps the branch-protection check name and can be deleted once protection requires Repository gates directly.

Locally the merged invocation analyzes and passes cold in 125 s (1 target, 8 tests, critical path 22 s).

Before/after (this PR's CI run)

Job Baseline This PR
Repository gates 285 s pending
Feature gates 151 s (own runner, setup-bazel 52 s) pending
Clippy (unchanged) 1022 s pending
Dashboard (unchanged) 321 s pending
Test Linux (fleet) (unchanged) 1775 s pending

Per-step after-timings will be added when the run finishes.

Validation

  • actionlint passes.
  • scripts/check-github-hosted-runners.py and its 11 tests pass.
  • The merged Bazel step passes locally (8/8 tests, //:rustfmt_check built).

🤖 Generated with Claude Code


Devin Review

Measured (run 41122200755 vs baseline run 41016935321)

Baseline This PR
Policy tests + rustfmt + release PR refresh test 72 s + 37 s + separate step 99 s, one Bazel invocation
Repository gates job 285 s 274 s
Feature gates job 151 s (own runner and Bazel setup) 4 s (status check on Repository gates)

That saves about 160 s of runner time per run, and one fewer runner doing Bazel setup. Those jobs already ran in parallel with the longer Linux and Windows jobs, so a run's total time barely changes. The saving shows up in the hosted-runner queue: in this run, Change scope waited 26 min for a runner and Repository gates started 53 min in.

Repository gates paid a separate Bazel analysis for each of its Bazel
steps, and Feature gates paid a whole runner for one more. In run
41016935321 the policy-test step spent 65 s of 71 s loading and
analyzing (critical path 6 s), `bazel build //:rustfmt_check` 35 s of
37 s (critical path 2 s), and Feature gates 52 s of Bazel setup plus
83 s of analysis for a 1 s `gen_builds_test` (151 s job).

One `bazel test` invocation now builds `//:rustfmt_check` and runs every
policy test, including `gen_builds_test` and `release_pr_refresh_test`,
so the actions run side by side under one analysis. Locally the merged
invocation analyzes and passes cold in 125 s (8 tests, critical path
22 s). Expected: Repository gates about 40 s shorter; Feature gates
becomes a pass-through that keeps the protected check name and frees a
hosted runner slot and a 2.3 GB cache restore per run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: c28d7b4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy
ScriptedAlchemy enabled auto-merge (squash) October 30, 2026 19:50

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

A Renovate bump under mockups/ (PR #4633) reran Clippy for 53 minutes
and six Linux shards for 45 to 80 minutes on run 41101852182, although
mockups/ is in .bazelignore and no Bazel action could change. Two
causes: Renovate's actor (29139614) has no fleet identity, so its runs
take the hosted path, and the hosted path has no compiled-action cache
at all: `gh cache list` shows zero setup-bazel disk entries, because a
fleet-enabled master run skips the restore step, which leaves the save
key empty, so master never saved one.

Fixes:
- `vars.TS_BAZEL_RENOVATE_CLIENT_ID` enrolls Renovate's pull requests
  in the fleet alongside the owner and the Devin bot, in every copy of
  the identity expression, so its unchanged actions become remote
  cache hits.
- A `Change scope` job lists a pull request's files once and decides,
  through scripts/ci-change-scope.py, whether any path reaches a Bazel
  input. Only mockups/, docs/, and repository-root Markdown qualify as
  skippable; plugin/** Markdown is globbed by Bazel. Clippy, Test
  Linux, Test Windows, and Dashboard skip when nothing qualifies as
  Bazel input; the Test Linux result job accepts a skipped matrix only
  in that case. Empty, truncated, or unlisted file sets fail closed and
  run everything, and pushes and dispatches never skip.
- Dashboard, the one Bazel job without a fleet identity, saves its disk
  cache on master so hosted runs stop rebuilding the frontend toolchain
  every time.

Validation: actionlint, the hosted-runner policy check, 16 change-scope
unit tests, and the decision script against PR #4633's real file list
(bazel=false) and an empty list (bazel=true).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

Comment on lines +35 to +36
if len(parts) > 1:
return parts[0] in SKIPPABLE_DIRECTORIES

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 New documentation targets bypass CI

When a PR adds docs/BUILD.bazel, path_is_skippable skips the Bazel jobs despite the new package entering //.... Failing targets in that package receive no Linux or Windows validation.

Learn more

The change-scope job decides whether to run Bazel-backed checks from changed filenames. Bazel does not currently have a package under docs/, but adding a BUILD or BUILD.bazel file creates one. The directory-wide allowlist treats that new file as documentation, so the ready-PR Linux and Windows checks skip even though their //... patterns would include the package. The Linux result job also treats its skipped matrix as successful when the scope output is false.

Example: A PR adds docs/BUILD.bazel with a failing sh_test. The classifier reports bazel=false, so the Linux test result succeeds without running the new test.

Recommended fix: Reject docs/BUILD, docs/BUILD.bazel, and other Bazel package/configuration files before applying the docs/ allowlist; add a regression test for new packages under skippable directories.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

#4637 moved the Linux test job's setup-bazel step above the disk cleanup.
Keep that position and carry this branch's Renovate fleet identity into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ScriptedAlchemy added a commit that referenced this pull request Oct 31, 2026
* test(kimi): reject probes of a port kimi web never announced

A port TraceDecay asks `kimi web` for, but that `kimi web` never bound,
can belong to any local process. The new `foreign` fake mode has another
server take that port while Kimi never announces, and asserts Kimi's
bearer token never reaches it. Fails on master with
"GET /api/v1/plugins returned HTTP 404", the CI message from #4638.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(kimi): only probe the endpoint kimi web announces

TraceDecay picked a loopback port by binding port 0 and dropping the
listener, passed it to `kimi web --port`, and polled it with Kimi's
bearer token as soon as `server.token` existed. Until Kimi bound that
port, any local process could hold it, and Kimi itself moves to the next
free port when the requested one is busy. Either way Kimi's token went
to a server that was not Kimi; in CI a parallel test's server answered
404.

Start `kimi web --port 0` so only Kimi holds its port, and probe only
the port and token from a complete `Local:` line of that child's ready
banner, which Kimi prints after it binds. The `server.token` read goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@ScriptedAlchemy
ScriptedAlchemy merged commit a5af5cd into master Oct 31, 2026
28 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant