[anakanemison]

Part 5 of 23 of a stacked series (tracking issue: #35). Based on U04-deterministic-variants, the previous part: review and merge that first.

Kind: report. Lab proposal: P16, P17, F56.

What the migration runs showed

The differ discarded -O* and -g* and compared flags one way only, so 8 of 9 pono migrations (with or without the skill) built at -O2 against an -O3 reference, none of 7 lensfun migrations kept -g, and four judged migrations turned assertions off (NDEBUG), with no difference reported. With the report, pono kept -O3, lensfun and openttd kept -g, and cmake added neither -O2 nor NDEBUG (the lab's later runs).

The change (4 commits)

Report Bazel-only flags beside the verdict

The flag comparison is an asymmetric subset: a CMake flag missing on the
Bazel side is an error, and a flag only Bazel has is tolerated. That is the
right policy, since Bazel's toolchain injects flags no CMake build has, but
nothing in the output said so. "0 errors" reads as "both builds compile the
same way" when it means "no CMake flag is missing on the Bazel side", and a
flag the Bazel side added appeared nowhere unless that TU also had a real
discrepancy.

Measured in any2bazel-lab: one migration's Bazel side carried 15 distinct
compiler warning flags against CMake's one, including ten -Wno-error= demotions the agent
added to get past -Werror, and the diff reported 0 errors and 0 warnings
with an empty ignore block. Those demotions were justified; an unjustified
-w would have looked the same.

diff_models takes an optional DiffNotes collector. For every compared TU
pair (the chosen variant pair) it counts the compile flags only Bazel has,
after the ignore lists and minus the toolchain defaults already hidden from
bazel_only; for every aligned executable, it does the same for link flags.
summarize() reports them as "bazel_only_flags" (flag -> compared TU pairs,
or executables) with a note that they are not compared; triage.py prints a
one-line summary with the top entries; SKILL.md says so where diff.json is
described. Severity and converged are unchanged.

Not done: raising warning-control flags the Bazel side adds (-w, -Wno-,
-Wno-error=) to WARN. That changes what the tool counts, so it is left for
the maintainers to decide.

In the lab's later graphicsgems run with this change, the agent's report
named its Bazel-only flags; the earlier run's report had not.

Report each side's optimization and debug level in the diff summary

canonicalize.py drops -O* and -g* from both sides before comparing. That is
the right rule for a per-TU diff, since the level is a build-wide setting,
but nothing else looked at it, so the tool reported parity for builds
compiled at a different optimization level than the reference, and SKILL.md
told the agent the flags were "never your concern".

Measured in any2bazel-lab across its first 35 judged migrations, with and
without the skill: seven are of pono, whose reference compiles all 94 TUs at
-O3, and six of those seven compile at -O2, each reported as 0 errors.
lensfun's reference sets -g on all 40 TUs, and every migration lost it. On
MySQL Server one migration flattened the reference's 201 -O3 TUs to -O2.

summarize() now adds "levels": per side and per language (C, CXX, OBJC,
OBJCXX, by source extension), histograms of the optimization level, the
debug level and the debug format each compared compile action carries. The
last flag of each kind wins, and "(none)" means absent. Level and format are
kept apart because they vary independently: -g3 -gdwarf-4 and
-g1 -gdwarf-4 share a format, and only -g3 emits macro information. A
-gdwarf-N with no level flag counts as -g, since it enables debug info.
"differs" is set when, for a language both sides compile, any of the three
differs. It reads the raw argv the models already hold, over exactly the
targets the diff compares (participating roles, minus exclude_targets on
the name the diff compares under, minus Bazel header-processing actions),
so no new extraction is needed. It never affects errors or converged.
triage.py prints the levels, and SKILL.md says where to look instead of
calling the flags no concern.

In the lab's later runs with this series, pono kept -O3 (8 of 9 earlier
pono migrations were at -O2), lensfun and openttd kept -g (no earlier
lensfun migration had), and xbmc kept its reference's -O3.

Apply -U in flag canonicalization: the last -D/-U wins

canonicalize_flags collected every -D into the define set and passed -U
through as an ordinary flag, so an undefine never removed anything. A
Bazel build that keeps assertions on with build --copt=-UNDEBUG after
-c opt's -DNDEBUG was reported as defining NDEBUG (a WARN on every TU),
and one that turned assertions back on against a reference compiled with
-DNDEBUG converged with 0 errors. Later commits in this series report
NDEBUG state and recommend exactly that -UNDEBUG, so it has to be read
correctly.

• canonicalize_flags applies -U NAME / -UNAME as the compiler does: it
  removes every NAME define collected so far, and a later -D defines it
  again. A -D likewise replaces every earlier definition of its name, so
  each side ends with the last word per macro: -DF(x)=1 -DF(x)=2 and
  -DF(x)=1 -UF -DF(x)=2 both leave F(x)=2 and compare equal. -UNAME is
  also kept as a comparable flag iff the -U is the last word on NAME (a
  later -D clears it), so whether it survives never depends on what either
  side's toolchain passed before it. Bazel's -c opt preamble
  (-D_FORTIFY_SOURCE=1 -DNDEBUG) followed by a project's
  -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 now compares equal to the
  reference (before this change it produced a discrepancy, in either
  direction). A last-word -U still compares a -U of a compiler-predefined
  macro (-U_FORTIFY_SOURCE alone). A define is matched by its macro name,
  the identifier before any ( or =, so -UF cancels a function-like
  -DF(x)=x, as gcc -E shows.
• A define with no value means value 1 to the compiler, so -DX is
  spelled X=1 (and -DF(x) is F(x)=1); -DX -DX=1 and -DX=1 -DX
  compare equal. Define values in diff.json now show that spelling.
• extract_cmake.py synthesizes each compile argv in the order CMake's
  generators emit it ($DEFINES $INCLUDES $FLAGS, as in CMake 3.28's
  Makefile and Ninja output) instead of flags first, so a -UNDEBUG in
  CMAKE_CXX_FLAGS lands after, and cancels, an NDEBUG compile definition,
  as it does in the real command. It does not cancel Release's -DNDEBUG,
  which comes later in the same flags fragment; only a compile definition
  is cancelled. Include order changes too: structured include directories
  now precede any -I from the flags fragments. That has no verdict impact,
  since includes are compared for presence, not order.

Checked against the preprocessor: a differential test generates fixed-seed
random -D/-U sequences over object-like and function-like names (joined
and split forms, with and without values, repeats) and requires two
sequences to canonicalize to the same define set exactly when cc -E
expands a probe source identically; it is skipped when no C compiler is
installed. A larger run of the same check (3,000 sequences, 4.5 million
pairs, three seeds) found no disagreement in either direction. One class
is documented instead of fixed: a function-like macro that differs only by
parameter names (F(x)=x vs F(y)=y) preprocesses the same but reports a
define difference.

Trade-off: a reference compiled with -DNDEBUG -UNDEBUG against a Bazel
side with neither now shows a cmake-only -UNDEBUG flag error. It is
benign (both sides compile without NDEBUG); copy the flag into copts, or
add it to ignore.flags (ignore.defines does not reach it: it is a flag).

Tests cover valueless defines, last-word -U in both orders, redefinition
with and without an intervening -U, function-like macros in both orders,
the predefined-macro case, the fortify override against Bazel's opt
preamble in both directions, both NDEBUG directions through the diff, and
the CMake argv order.

Report Bazel-only defines and NDEBUG state beside the verdict

Defines are compared both ways, but a define only the Bazel side has is a
WARN on each TU, and no summary line showed it. The one that matters most is
NDEBUG: Bazel's -c opt adds -DNDEBUG, so a migration that pins
--compilation_mode=opt against a reference that keeps assertions turns them
off everywhere and still converges.

Measured in any2bazel-lab: four judged migrations compile with NDEBUG
against a reference that passes none, and nothing in the differ's verdict
or summary said so. Three migrations of CMake's own sources got it from
-c opt, on every compile action; a GDAL migration passed -DNDEBUG itself,
on 1,534 of 1,534 actions, at no optimization level.

• bazel_only_flags gains "defines": each define only the Bazel side has,
  after ignore.defines, with the compared TU pairs it appears on. Its note
  says these are a WARN, not an error; triage prints the count and the top
  entries.
• A new "ndebug" block sits next to "levels". Per side, it counts the
  compared compile actions that leave NDEBUG defined. With the diff's
  notes, "bazel_added" and "bazel_missing" count the compared TU pairs
  where only Bazel, or only CMake, defines it, and "differs" is true when
  either count is nonzero; a Bazel-only TU that is never compared cannot
  flip it. Both read the canonicalized defines, which apply -U in order (the
  previous commit), so a build --copt=-UNDEBUG after -c opt counts as
  assertions on. The counts ignore ignore.defines on purpose, so an accepted
  suppression cannot hide assertion state. triage.py prints one NDEBUG line
  after the levels.
• SKILL.md mentions ndebug under "Diff against reference build". The
  guidance on opt's -DNDEBUG is in the build-type commit.

Severity and converged are unchanged: a Bazel-only NDEBUG is still the
existing WARN. Whether it should be an error, since it changes what the
built program does, is a decision for the maintainers.

In the lab's later CMake runs with this series, none added NDEBUG against
the reference's none (three runs; two earlier runs had added it).

Testing

pytest tests/ at this commit: 178 passed (130 on main).

🤖 Generated with Claude Code