Repository navigation
Align conda-recipe-cf and its CI with the conda-forge feedstock - #287
antonwolfy wants to merge 3 commits into
Conversation
Make the conda-forge-only CI build mkl_umath the way conda-forge/mkl_umath-feedstock does, so problems the feedstock would hit show up upstream first: - Pass --error-overlinking to conda build (the feedstock sets error_overlinking: true) and allowlist $RPATH/mkl_umath_loops.dll on Windows, as the feedstock does. - Build against the newest NumPy per Python version (2.4 for 3.11, 2.5 for 3.12+), matching the feedstock, and drop the unused numpy 1.26.4 pin from conda_build_config.yaml. - Take run dependencies from run_exports instead of pin_compatible, stop ignoring the numpy and llvm-openmp run_exports, add intel-cmplr-lib-rt to host, and ignore the dpcpp run_exports on Linux. - Use gcc/gxx 15 on Linux, as in conda-forge's global pinning.
jharlow-intel
left a comment
There was a problem hiding this comment.
Thanks, this looks good! The build and requirements sections now match the feedstock line for line, and turning on --error-overlinking with the same DLL allowlist means a feedstock break should show up here first. A couple of small things inline.
| - python | ||
| - mkl-devel | ||
| - intel-cmplr-lib-rt # [not osx] | ||
| - numpy |
There was a problem hiding this comment.
With pin_compatible gone, the numpy run dep now comes from numpy's run_exports, so the 3.12+ builds against 2.5 accept numpy >=1.25,<3 instead of >=2.3,<3. That's what the feedstock ships too, but the test jobs only install numpy>=2, which resolves to the newest, so nothing here exercises the low end. Worth adding one test leg pinned to the oldest numpy we claim (1.26.4 per pyproject)? A feedstock-only breakage at that end would then show up here as well.
There was a problem hiding this comment.
Good catch, the low end turned out to be broken, not just untested. USE_NUMPY_2 is set at compile time from the NumPy headers (mkl_umath_loops.c.src), so a build against NumPy 2 carries NumPy 2's complex sign and the int64 ldexp loops. I installed the published feedstock 0.6.0 build with NumPy 1.26.4 (py3.12) and 1.23.5 (py3.11) on Linux, and the complex sign tests fail: np.sign(3+4j) gives 0.6+0.8j instead of NumPy 1's 1+0j. On Windows, NumPy 1.x's ldexp only has int/long (32-bit) loops, so the patcher won't find a match for the int64 ones and patch_numpy_umath() raises. So instead of a NumPy 1 test job I added numpy >=2 to run: in 1111c31; together with the run_exports that gives >=2,<3. NumPy 2.0.2 passes the whole suite locally, so I didn't extend the matrix. The feedstock needs the same fix, I'll follow up there.
| @@ -1,11 +1,11 @@ | |||
| numpy: | |||
| - '1.26.4' | |||
| c_compiler: # [linux] | |||
There was a problem hiding this comment.
Nit: since the matrix always passes --numpy, dropping the 1.26.4 pin is fine for CI. But a plain local conda build conda-recipe-cf now falls back to conda-build's built-in numpy default (1.26), which gives a numpy-1-only package and isn't what the feedstock does. Could we put numpy: ['2'] here, matching conda-forge's global pinning? I believe the --numpy flag in CI would still take precedence.
There was a problem hiding this comment.
Agreed, done in 1111c31, same as conda-forge's global pinning. Checked with conda render: without --numpy, py3.12 now builds against 2.5 instead of 1.26.4, py3.13 resolves at all, and --numpy 2.4 still takes precedence.
mkl_umath decides between its NumPy 1 and NumPy 2 loops at compile time (USE_NUMPY_2 in mkl_umath_loops.c.src). numpy's run_exports alone allow NumPy >=1.25,<3 for a build against NumPy 2.5, but with NumPy 1.x such a build returns NumPy 2 results for complex sign, and on Windows NumPy 1.x has no int64 ldexp loops for the patcher to replace. Add numpy >=2 to the run requirements; run_exports still provide the <3 upper bound. Set numpy: '2' in conda_build_config.yaml, as in conda-forge's global pinning. CI overrides it with --numpy, but a plain local conda build otherwise falls back to conda-build's built-in NumPy 1.26 default, which yields a NumPy-1-only package and cannot be solved for Python >= 3.13.
This PR makes the conda-forge-only CI (
conda-package-cf.yml+conda-recipe-cf/) buildmkl_umaththe same way conda-forge/mkl_umath-feedstock does, so problems the feedstock would hit show up here first.Changes
Fail on overlinking
--error-overlinkingtoconda buildon Linux and Windows (the feedstock setsconda_build: error_overlinking: true).missing_dso_whitelist: $RPATH/mkl_umath_loops.dll # [win], as in the feedstock. That DLL is the only overlinking hit across all 14 build jobs of the latestmainrun (every Windows job; Linux is clean).Build against the same NumPy as the feedstock
2.4for py3.11 and2.5for py3.12–3.15 including 3.14t (was2.3). py3.10 stays on2.2.numpy: '1.26.4'pin inconda_build_config.yamlwithnumpy: '2', as in conda-forge's global pinning. CI overrides it with--numpy; without a pin, a localconda build conda-recipe-cffalls back to conda-build's built-in NumPy 1.26 default, which gives a NumPy-1-only package and cannot be solved for py >= 3.13.Run dependencies from run_exports, as in the feedstock
pin_compatible('mkl' / 'numpy' / 'intel-cmplr-lib-rt')run lines and stop ignoring thenumpyandllvm-openmprun_exports.intel-cmplr-lib-rt # [not osx]to host. Previously it reached the build env only as a dependency of another package, so its run dependency had no version.dpcppcompiler run_exports on Linux, so the package no longer depends ondpcpp-cpp-rtthere.numpy >=2to run. mkl_umath picks its NumPy 1 or NumPy 2 loops at compile time (USE_NUMPY_2inmkl_umath_loops.c.src), so a build against NumPy 2 must not run on NumPy 1.x: complexsignthen returns NumPy 2 results, and on Windows NumPy 1.x has noint64ldexploops for the patcher to replace. numpy's run_exports alone would allow>=1.25,<3.The numpy run dependency becomes
numpy >=2plusnumpy >=1.25,<3from run_exports (>=1.23,<3on py3.11), so in effect>=2,<3instead of>=2.3.5,<3. Judging by the feedstock's build logs,mklandintel-cmplr-lib-rtare pinned>=2026.x,<2027andllvm-openmp >=23.1.3.Compilers
c_compiler_version/cxx_compiler_version15(wascxx_compiler_version: 14only), as in conda-forge's global pinning.Notes
numpy >=1.25,<3); thenumpy >=2run requirement needs to go there as well.pytest --pyargs mkl_umath, as the feedstock does) is covered separately by Run the whole test suite in the conda recipes #283.