Repository navigation
Make a reproducible clean build part of done: vendoring, declared tests, no host installs, nothing gitignored - #52
Open
anakanemison wants to merge 2 commits into
Conversation
The definition of done was parity of the action graph. Parity is unaffected by whether a TU compiles: the diff compares commands, and the command for a TU that fails compares as well as one that succeeds. So an agent could converge, truthfully report 0 discrepancies, and hand back a workspace where `bazel build //...` fails. Measured in any2bazel-lab: across ten larger projects, the skill-guided migration failed `bazel build //...` on three, and the unguided one on none. On Kodi the agent converged over 1,789 TUs while ten TUs could not compile, reasoning that FFmpeg and two other libraries were missing from the host and that "the parity diff is unaffected". The unguided run in the same environment vendored FFmpeg and SWIG through rules_foreign_cc and built crossguid and libdvdcss from pinned sources. - The "what converged does not cover" list gains the build itself, and the Repeat step says: repeat until converged, then, once, `bazel build //...` from a fresh output base must succeed, and so must `bazel test` over the tests the migration declares, with the files they read declared as data. Any target left out is named and explained. - "Create initial BUILD.bazel files" says a missing host dependency can be vendored (a pinned module or http_archive, or rules_foreign_cc), that a TU left uncompiled for a missing dependency is a gap in the migration, and not to install the dependency into the host instead (make install, /usr/local, /opt), since the build then passes only on the machine that was prepared. - The report includes the clean-build result. In the lab's later runs with this change: - The next Kodi migration built every target from a fresh output base for the first time, by vendoring FFmpeg, SWIG and four smaller libraries. Its ported kodi-test, though, aborted under `bazel test` with its data files undeclared, which is what the test clause addresses. - With the test clause, a later Kodi migration ran kodi-test under `bazel test`, and 1,686 of the 1,689 reference-passing tests passed. That agent had compiled FFmpeg 9 into its container's /usr/local instead of vendoring it, so on a fresh machine with the distribution's FFmpeg 6.1 the tree does not compile. The host-install clause was written after that run and has not yet been tested. Lab-Proposal: P23 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n's files Before reporting, the agent now checks `git status --short --ignored` for files it wrote that the project's .gitignore hides, and renames them or adds an exception. A file git ignores is left out of the migration anyone commits, and nothing in the parity loop or the build notices: both read the working tree. Found in any2bazel-lab: QGIS's `/build*` rule matched an agent's `build_defs/`, dropping from a plain `git add` the ten .bzl and .py files every BUILD file loads. Blender's .gitignore hid an unguided run's `.bazelrc`, YugabyteDB's its four `bazel/thirdparty` files, and GDAL's an agent's `cpl_config.h`. In each case the build passed in the agent's tree and fails from a commit of it. In the lab's six runs with this change, no migration had a file the project's .gitignore hides, and the QGIS agent ran the check and kept its rules out of `/build*`. Lab-Proposal: P27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 17 of 23 of a stacked series (tracking issue: #35). Based on
U16-parity-artifacts-location, the previous part: review and merge that first.Kind: guidance. Lab proposal: P23, P27.
What the migration runs showed
Three of ten large migrations failed
bazel build //...(kodi and mysqld among the missing targets) where the paired runs without the skill built, xbmc's by vendoring. With this guidance, xbmc built every target by vendoring, then passed 1,686 of 1,689 reference tests underbazel test(runs 088, 095); one run installed FFmpeg into its container instead, which the second revision rules out; and three of 79 kept migrations had Bazel files the project's own.gitignorehid.The change (2 commits)
Make a clean, reproducible build part of done
The definition of done was parity of the action graph. Parity is unaffected
by whether a TU compiles: the diff compares commands, and the command for a
TU that fails compares as well as one that succeeds. So an agent could
converge, truthfully report 0 discrepancies, and hand back a workspace where
bazel build //...fails.Measured in any2bazel-lab: across ten larger projects, the skill-guided
migration failed
bazel build //...on three, and the unguided one on none.On Kodi the agent converged over 1,789 TUs while ten TUs could not compile,
reasoning that FFmpeg and two other libraries were missing from the host and
that "the parity diff is unaffected". The unguided run in the same
environment vendored FFmpeg and SWIG through rules_foreign_cc and built
crossguid and libdvdcss from pinned sources.
Repeat step says: repeat until converged, then, once,
bazel build //...from a fresh output base must succeed, and so must
bazel testover thetests the migration declares, with the files they read declared as data.
Any target left out is named and explained.
vendored (a pinned module or http_archive, or rules_foreign_cc), that a TU
left uncompiled for a missing dependency is a gap in the migration, and
not to install the dependency into the host instead (make install,
/usr/local, /opt), since the build then passes only on the machine that
was prepared.
In the lab's later runs with this change:
the first time, by vendoring FFmpeg, SWIG and four smaller libraries. Its
ported kodi-test, though, aborted under
bazel testwith its data filesundeclared, which is what the test clause addresses.
bazel test, and 1,686 of the 1,689 reference-passing tests passed. Thatagent had compiled FFmpeg 9 into its container's /usr/local instead of
vendoring it, so on a fresh machine with the distribution's FFmpeg 6.1 the
tree does not compile. The host-install clause was written after that run
and has not yet been tested.
Say to check that the project's .gitignore does not hide the migration's files
Before reporting, the agent now checks
git status --short --ignoredfor filesit wrote that the project's .gitignore hides, and renames them or adds an
exception. A file git ignores is left out of the migration anyone commits, and
nothing in the parity loop or the build notices: both read the working tree.
Found in any2bazel-lab: QGIS's
/build*rule matched an agent'sbuild_defs/, dropping from a plaingit addthe ten .bzl and .py filesevery BUILD file loads. Blender's .gitignore hid an unguided run's
.bazelrc, YugabyteDB's its fourbazel/thirdpartyfiles, and GDAL's anagent's
cpl_config.h. In each case the build passed in the agent's treeand fails from a commit of it.
In the lab's six runs with this change, no migration had a file the
project's .gitignore hides, and the QGIS agent ran the check and kept its
rules out of
/build*.Testing
pytest tests/at this commit: 196 passed (130 onmain).🤖 Generated with Claude Code