Skip to content

CircleCI: build and test platform wheels with release artifact bundles - #218

Open
eresende-nuodb wants to merge 3 commits into
masterfrom
ci/platform-wheel-artifacts
Open

eresende-nuodb wants to merge 3 commits into
masterfrom
ci/platform-wheel-artifacts

Conversation

@eresende-nuodb

@eresende-nuodb eresende-nuodb commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implement the complete public build/test/artifact stage in CircleCI only. Publishing remains an internal responsibility; no PyPI publishing credentials are used by these jobs. The earlier GitHub Actions workflow has been removed.

  • Build one source archive and share it with all 25 release-wheel builds and the database-regression wheel build.
  • Cover CPython 3.9–3.13 on Linux x86-64/ARM64, macOS Intel/Apple Silicon, and Windows x86-64.
  • Use native Linux/Windows/Apple Silicon executors; cross-build Intel macOS wheels and run their tests under Rosetta, with an explicit execution check and no test skips.
  • Test installed wheels outside the checkout and verify the no-compiler source-install fallback.
  • Gate final bundle assembly on all five wheel jobs, database regressions, and the existing performance tests in the same CircleCI workflow.
  • Store one release-bundle.tar.gz artifact containing the 26 distributions, checksums, and manifest with CircleCI pipeline/workflow/job identities.
  • Validate versions, ABIs/platforms, required source files, and compiled-extension presence. Keep parallel workspace outputs in distinct directories.
  • Enable version-tag builds on all upstream jobs and document the internal artifact-collection handoff.

Only the existing database/performance jobs use common-config; source/wheel/bundle jobs do not use a secret context. The organization must have access to Linux ARM, m4pro.medium, and windows.medium executors.

Understanding the wheel filenames and build matrix

The cp39 tag means CPython 3.9. CPython is the standard Python interpreter most users install. Cython is the build-time tool that translates our extension sources into C, which is then compiled into a native module. The Cython version is not encoded in the wheel filename.

A wheel filename follows this pattern:

package-version-python_tag-abi_tag-platform_tag.whl

For example, pynuodb-5.0.0-cp39-cp39-win_amd64.whl means:

Filename component Meaning
pynuodb Package name
5.0.0 Driver version
First cp39 CPython 3.9 interpreter
Second cp39 CPython 3.9 binary interface (ABI)—the interface the compiled extension must match
win_amd64 Windows on x86-64 CPUs (both Intel and AMD)

The Python tags in this PR are cp39, cp310, cp311, cp312, and cp313, meaning CPython 3.9 through 3.13 respectively.

Why 25 wheels?

Unlike a pure-Python package, this Cython extension contains compiled native code. For the build configuration used here, each wheel must match both the Python interpreter/ABI and the operating system/CPU architecture. A CPython 3.9 wheel is not interchangeable with a CPython 3.13 wheel, and a Windows x86-64 binary cannot serve a Linux ARM64 system.

We therefore build 5 Python versions × 5 OS/CPU targets = 25 wheels:

  • Linux x86-64
  • Linux ARM64
  • macOS Intel (x86-64)
  • macOS Apple Silicon (ARM64)
  • Windows x86-64

Linux uses the manylinux2014 compatibility standard, so we do not need a separate wheel for every glibc-based Linux distribution. Each wheel is installed and tested for its Python/architecture combination, with Intel macOS tests running under Rosetta.

Users still run pip install pynuodb. All wheels belong to the same package/version on PyPI; pip automatically selects the compatible one. Installing a matching wheel requires neither Cython nor a C compiler. The additional source archive provides an installation path when no matching wheel is available, including the existing optional pure-Python fallback.

Documentation

The documentation-only follow-up commit 47839da includes:

  • README.rst: optional acceleration, automatic wheel selection, source-install fallback and dependency requirements, and the distinction between the wheel test matrix and Linux database regressions. It explicitly distinguishes CI candidates from published PyPI releases and uses python -m pip for installation.
  • CONTRIBUTING.rst: durable wheel-filename/ABI examples and the reason for 25 wheels, alongside the existing build, test, bundle, and publishing-handoff documentation.
  • Both RST files render without warnings. The updated source archive passes twine check --strict, its package description contains the updated README, and all 36 CI-helper tests pass.

The documentation follow-up has its own CircleCI check run (see the PR Checks tab). The completed workflow and independent artifact inspection below remain explicitly scoped to implementation commit c839de1.

Validation

  • 36 CI-helper tests passed, covering release validation, CircleCI provenance, matrix coverage, dependency gates, tag filters, and workspace paths.
  • CircleCI CLI 1.11.0 accepted the configuration; compiled shell commands passed syntax checks.
  • Built and validated the updated source archive; twine check --strict passed.
  • Previously verified 45 installed-wheel tests on each of CPython 3.9, 3.11, and 3.13 on macOS, plus the no-compiler fallback.
  • The implementation workflow for c839de1 passed all nine jobs; its downloaded release bundle was independently inspected as detailed below. The later 47839da commit changes documentation only.

CircleCI validation completed (implementation commit c839de1)

All 9 CircleCI jobs passed for commit c839de10ab88ba70ed98ab8f033d53cebe4d376c in pipeline 274, workflow 3b4f0f4f-a094-4c95-b32e-0c5bce459002 (approximately 11 minutes).

Downloaded artifact inspection

Inspected the actual release-bundle.tar.gz artifact from assembly job 347, not a local rebuild. No issues were found in the following checks:

  • Completeness and integrity: exactly 25 wheels (five per OS/CPU target) and one source archive, all version 5.0.0. All 26 distribution sizes and SHA-256 checksums match the manifest; SHA256SUMS also verifies successfully.
  • Provenance: the manifest's repository, commit, branch, pipeline ID/number, workflow ID, and assembly-job identity match CircleCI's live records and the inspected implementation commit c839de1.
  • Wheel contents: ZIP integrity, internal RECORD hashes/sizes, package/dependency metadata, non-pure-Python wheel flags, and Python/ABI/platform tags all check out. Every wheel contains a compiled accelerator whose actual binary format and CPU architecture match its target (ELF on Linux, Mach-O on macOS, PE on Windows).
  • Source consistency: 48 tracked source/config/license files in the source archive match the inspected commit byte-for-byte. The archive includes generated C, Cython sources, headers, and test sources. The Python modules in all wheels match the source archive.
  • Distribution metadata: twine check --strict passes for all 26 downloaded distributions.
  • Additional local runtime verification: installed the downloaded macOS ARM64 wheels into fresh CPython 3.9 and 3.13 environments, without rebuilding. All 45 database-independent tests passed in each environment (90 passes total), with the compiled accelerator loaded. Other platforms' runtime validation was performed by the successful CircleCI wheel jobs, not rerun locally.

Outer archive SHA-256:

c7b671b42710d353d85f26b225dcf1ffd5ce1d5eaafa72a9d116e7a068e83a58

No distributions are uploaded to PyPI by this PR. Successful branch/PR test artifacts are not automatically approved releases.

@eresende-nuodb eresende-nuodb changed the title Build and test platform wheels with release artifact bundles CircleCI: build and test platform wheels with release artifact bundles Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants