Skip to content

REL: ship array API test xfails in wheels - #249

Open
nightcityblade wants to merge 1 commit into
data-apis:mainfrom
nightcityblade:rel/issue-239
Open

nightcityblade wants to merge 1 commit into
data-apis:mainfrom
nightcityblade:rel/issue-239

Conversation

@nightcityblade

Copy link
Copy Markdown

Closes #239.

Summary

Include array-api-tests-xfails.txt in built wheels so downstream test suites can retrieve the matching expectations through importlib.metadata.files("array_api_strict").

Validation

  • Full test suite: 770 passed
  • ruff check .: passed
  • validate-pyproject pyproject.toml: passed
  • Built both wheel and sdist and ran twine check --strict: passed
  • Installed the built wheel into a clean environment and verified its shipped xfail file byte-for-byte against the source file

AI generation disclosure

OpenAI Codex was used to implement the packaging configuration and run the validation steps. The configuration change was AI-assisted and verified against the contents of a locally built and installed wheel.

@nightcityblade

Copy link
Copy Markdown
Author

CI follow-up: the only failure is the randomized Array API conformance test array_api_tests/test_linalg.py::test_trace in the Ubuntu / Python 3.10 / NumPy 1.26 job while testing API version 2025.12. Hypothesis generated a uint32 case where the test reported Array(5) != Array(1).

This PR changes only setuptools package-data metadata in pyproject.toml; it does not modify trace or any runtime implementation. The same job passed its 2023.12 and 2024.12 runs, the equivalent Python 3.10 / NumPy 1.26 jobs passed on macOS and Windows, every other matrix job passed, and this exact Ubuntu matrix has passed on recent upstream runs.

I attempted to rerun the failed job, but GitHub requires repository admin permission. Could a maintainer please rerun it? I don't believe a code change in this PR would be appropriate for this unrelated property-test failure.

@betatim

betatim commented Oct 14, 2026

Copy link
Copy Markdown
Member

Please also package the other xfail files.

@nightcityblade

Copy link
Copy Markdown
Author

Done — the wheel and sdist now include array-api-tests-macos-xfails.txt and array-api-tests-windows-xfails.txt alongside the shared xfail file. I rebuilt both distributions, verified all three files are present in each artifact, ran twine check --strict, and ran Ruff; all passed.

@betatim

betatim commented Oct 15, 2026

Copy link
Copy Markdown
Member

Please move the xfail files inside array_api_strict/ instead of using ../ in package-data. As written they install to the top level of site-packages. After this you will also have to update the CI workflow, docs link, and array-api-tests workflow accordingly (we use the files there so we need to coordinate if we move the files). array-api-tests' .github/workflows/test.yml:37 hardcodes /tmp/array-api-strict/array-api-tests-xfails.txt. Either coordinate a change there or keep the root path working during a transition phase.

It would also be good to add a test that checks that using importlib.resources.files("array_api_strict") / "array-api-tests-xfails.txt" to load the file works.

What do you think @ev-br ?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REL: ship array-api-tests-xfails.txt in wheels

2 participants