Skip to content

Make a reproducible clean build part of done: vendoring, declared tests, no host installs, nothing gitignored - #52

Open
anakanemison wants to merge 2 commits into
mschaller/u16-parity-artifacts-locationfrom
mschaller/u17-done-means-reproducible
Open

anakanemison wants to merge 2 commits into
mschaller/u16-parity-artifacts-locationfrom
mschaller/u17-done-means-reproducible

Conversation

@anakanemison

Copy link
Copy Markdown

Part 17 of 23 of a stacked series (tracking issue: #35). Based on U16-parity-artifacts-location, the previous part: review and merge that first.

Kind: guidance. Lab proposal: P23, P27.

What the migration runs showed

Three of ten large migrations failed bazel build //... (kodi and mysqld among the missing targets) where the paired runs without the skill built, xbmc's by vendoring. With this guidance, xbmc built every target by vendoring, then passed 1,686 of 1,689 reference tests under bazel test (runs 088, 095); one run installed FFmpeg into its container instead, which the second revision rules out; and three of 79 kept migrations had Bazel files the project's own .gitignore hid.

The change (2 commits)

Make a clean, reproducible build part of done

The definition of done was parity of the action graph. Parity is unaffected
by whether a TU compiles: the diff compares commands, and the command for a
TU that fails compares as well as one that succeeds. So an agent could
converge, truthfully report 0 discrepancies, and hand back a workspace where
bazel build //... fails.

Measured in any2bazel-lab: across ten larger projects, the skill-guided
migration failed bazel build //... on three, and the unguided one on none.
On Kodi the agent converged over 1,789 TUs while ten TUs could not compile,
reasoning that FFmpeg and two other libraries were missing from the host and
that "the parity diff is unaffected". The unguided run in the same
environment vendored FFmpeg and SWIG through rules_foreign_cc and built
crossguid and libdvdcss from pinned sources.

  • The "what converged does not cover" list gains the build itself, and the
    Repeat step says: repeat until converged, then, once, bazel build //...
    from a fresh output base must succeed, and so must bazel test over the
    tests the migration declares, with the files they read declared as data.
    Any target left out is named and explained.
  • "Create initial BUILD.bazel files" says a missing host dependency can be
    vendored (a pinned module or http_archive, or rules_foreign_cc), that a TU
    left uncompiled for a missing dependency is a gap in the migration, and
    not to install the dependency into the host instead (make install,
    /usr/local, /opt), since the build then passes only on the machine that
    was prepared.
  • The report includes the clean-build result.

In the lab's later runs with this change:

  • The next Kodi migration built every target from a fresh output base for
    the first time, by vendoring FFmpeg, SWIG and four smaller libraries. Its
    ported kodi-test, though, aborted under bazel test with its data files
    undeclared, which is what the test clause addresses.
  • With the test clause, a later Kodi migration ran kodi-test under
    bazel test, and 1,686 of the 1,689 reference-passing tests passed. That
    agent had compiled FFmpeg 9 into its container's /usr/local instead of
    vendoring it, so on a fresh machine with the distribution's FFmpeg 6.1 the
    tree does not compile. The host-install clause was written after that run
    and has not yet been tested.

Say to check that the project's .gitignore does not hide the migration's files

Before reporting, the agent now checks git status --short --ignored for files
it wrote that the project's .gitignore hides, and renames them or adds an
exception. A file git ignores is left out of the migration anyone commits, and
nothing in the parity loop or the build notices: both read the working tree.

Found in any2bazel-lab: QGIS's /build* rule matched an agent's
build_defs/, dropping from a plain git add the ten .bzl and .py files
every BUILD file loads. Blender's .gitignore hid an unguided run's
.bazelrc, YugabyteDB's its four bazel/thirdparty files, and GDAL's an
agent's cpl_config.h. In each case the build passed in the agent's tree
and fails from a commit of it.

In the lab's six runs with this change, no migration had a file the
project's .gitignore hides, and the QGIS agent ran the check and kept its
rules out of /build*.

Testing

pytest tests/ at this commit: 196 passed (130 on main).

🤖 Generated with Claude Code

anakanemison and others added 2 commits October 18, 2026 14:19
The definition of done was parity of the action graph. Parity is unaffected
by whether a TU compiles: the diff compares commands, and the command for a
TU that fails compares as well as one that succeeds. So an agent could
converge, truthfully report 0 discrepancies, and hand back a workspace where
`bazel build //...` fails.

Measured in any2bazel-lab: across ten larger projects, the skill-guided
migration failed `bazel build //...` on three, and the unguided one on none.
On Kodi the agent converged over 1,789 TUs while ten TUs could not compile,
reasoning that FFmpeg and two other libraries were missing from the host and
that "the parity diff is unaffected". The unguided run in the same
environment vendored FFmpeg and SWIG through rules_foreign_cc and built
crossguid and libdvdcss from pinned sources.

- The "what converged does not cover" list gains the build itself, and the
  Repeat step says: repeat until converged, then, once, `bazel build //...`
  from a fresh output base must succeed, and so must `bazel test` over the
  tests the migration declares, with the files they read declared as data.
  Any target left out is named and explained.
- "Create initial BUILD.bazel files" says a missing host dependency can be
  vendored (a pinned module or http_archive, or rules_foreign_cc), that a TU
  left uncompiled for a missing dependency is a gap in the migration, and
  not to install the dependency into the host instead (make install,
  /usr/local, /opt), since the build then passes only on the machine that
  was prepared.
- The report includes the clean-build result.

In the lab's later runs with this change:
- The next Kodi migration built every target from a fresh output base for
  the first time, by vendoring FFmpeg, SWIG and four smaller libraries. Its
  ported kodi-test, though, aborted under `bazel test` with its data files
  undeclared, which is what the test clause addresses.
- With the test clause, a later Kodi migration ran kodi-test under
  `bazel test`, and 1,686 of the 1,689 reference-passing tests passed. That
  agent had compiled FFmpeg 9 into its container's /usr/local instead of
  vendoring it, so on a fresh machine with the distribution's FFmpeg 6.1 the
  tree does not compile. The host-install clause was written after that run
  and has not yet been tested.

Lab-Proposal: P23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n's files

Before reporting, the agent now checks `git status --short --ignored` for files
it wrote that the project's .gitignore hides, and renames them or adds an
exception. A file git ignores is left out of the migration anyone commits, and
nothing in the parity loop or the build notices: both read the working tree.

Found in any2bazel-lab: QGIS's `/build*` rule matched an agent's
`build_defs/`, dropping from a plain `git add` the ten .bzl and .py files
every BUILD file loads. Blender's .gitignore hid an unguided run's
`.bazelrc`, YugabyteDB's its four `bazel/thirdparty` files, and GDAL's an
agent's `cpl_config.h`. In each case the build passed in the agent's tree
and fails from a commit of it.

In the lab's six runs with this change, no migration had a file the
project's .gitignore hides, and the QGIS agent ran the check and kept its
rules out of `/build*`.

Lab-Proposal: P27

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant