Skip to content

Run minimum-versions build on pushes to main to warm sccache - #3721

Open
agarg21 wants to merge 1 commit into
NVIDIA:mainfrom
agarg21:ci/min-versions-warm-sccache-main
Open

agarg21 wants to merge 1 commit into
NVIDIA:mainfrom
agarg21:ci/min-versions-warm-sccache-main

Conversation

@agarg21

@agarg21 agarg21 commented Oct 21, 2026

Copy link
Copy Markdown

Description

minimum_versions.yml only runs on pull_request. sccache's GitHub Actions cache backend therefore stores every compiled object in a PR-scoped cache (refs/pull/<N>/merge), which no other PR can read. When we checked, the repo had 0 cache entries on main and ~24.6k PR-scoped ones. As a result every new PR's first run compiles TE from scratch: ~2.6% sccache hit rate and ~60 min median wall time, versus ~10 min per job once that PR's own cache is warm.

Running the workflow on pushes to main fills the base-branch cache scope, which PR runs can restore.

Fixes # (none filed)

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • Add push: branches: [main] to .github/workflows/minimum_versions.yml (3 lines).

Measurement

Measured on a fork (agarg21/TransformerEngine), 3 fresh PRs per arm, all green:

First run of a new PR (median) Job-minutes per run sccache hit rate
Before (no main scope) 63.9 min 116 2.6%
After (main warmed by one push run) 14.7 min 24 96.4%

Fork run IDs: before 39471922855, 39471924582, 39471927842; after 39496253340, 39496257361, 39496261774; warm-up push run 39482052619.

Caveats:

  • Only each PR's first run benefits; later pushes to the same PR already hit that PR's own cache. That is roughly 88 of ~986 PR runs/month.
  • Cost: one extra build per merge to main on GitHub-hosted runners, mostly warm after the first.
  • Revert: remove the 3 lines.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Prepared with AI assistance (Claude) and measured on a fork. Opened as a draft per AGENTS.md until the submitter has reviewed it. Commit is DCO signed-off.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rt4UYn6zqR9uS7DWNGEBYx

The workflow only runs on pull_request, so sccache's GHA backend stores
every object in a PR-scoped cache that other PRs can't read. Running it
on pushes to main populates the base-branch scope, which PR runs can
restore, so a new PR's first build no longer compiles from scratch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rt4UYn6zqR9uS7DWNGEBYx
Signed-off-by: Apoorva Garg <271577+agarg21@users.noreply.github.com>
@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Oct 21, 2026
@agarg21
agarg21 marked this pull request as ready for review October 21, 2026 16:17
@greptile-apps

greptile-apps Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge.

Summary

Adds pushes to main as a trigger for .github/workflows/minimum_versions.yml, so builds can warm the main-branch sccache cache.

  • Pushes to main now run the minimum-version checks.

agarg21 explicitly acknowledged the extra build cost as the cost of warming the main cache.

Reviews (1) · Last reviewed commit: "Run minimum-versions build on pushes to ..." · Reviewed by Greptile

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant