Repository navigation
ci(release): publish to PyPI from an annotated tag - #2
Draft
dimitri-yatsenko wants to merge 5 commits into
Draft
dimitri-yatsenko wants to merge 5 commits into
dimitri-yatsenko wants to merge 5 commits into
Conversation
A release is now an annotated tag vX.Y.Z on main whose message is the release notes. Pushing the tag runs the CD workflow, which checks that the tag is annotated and on main, runs the CI workflow as the test gate, checks that the built files carry the tag's version, publishes them, and creates the GitHub release with the tag's message as its notes. The tag push replaces the "release published" trigger. The workflow file keeps its name, cd.yml, and the publish job keeps the pypi environment, because PyPI trusted publishing is bound to both. Pull requests and pushes to main still build the package, and a release run is never cancelled by a newer push. CI can now be called from CD. Its concurrency group is named "CI" explicitly: in a called run, github.workflow is the caller's name, and the shared group would make GitHub cancel the run. .github/scripts/check_release_tag.sh is the same script datajoint-python uses. CONTRIBUTING describes how to cut a release.
The publish step no longer sets repository-url, so a release tag publishes to pypi.org. Trusted publishing is unchanged: the same workflow file and the same pypi environment.
The codec calls self._extract_context(key, context) and self._codec_config(key, context), which datajoint adds in 2.3.4. Under 2.3.0 to 2.3.3, which datajoint>=2.3 allowed, _extract_context takes only the key and _codec_config does not exist, so every encode and decode would fail. The README and the docs index state the requirement. uv.lock already locked datajoint 2.3.4; only its recorded specifier changes.
DataJoint 2.3.4 and later pass context to every codec that declares it: schema, table, field and config on insert, config on fetch. The codec now reads them there and nowhere else. It no longer calls _extract_context and _codec_config, which fall back to the underscore keys in key (_schema, _table, _field, _config) when context lacks a value; DataJoint 2.4 removes those keys. The primary key still comes from key, without its underscore entries. Called without a context, or with one that lacks a value it needs, the codec raises a DataJointError that names the missing values and says that DataJoint 2.3.4 and later pass them, before it reads or writes any file. New tests call encode and decode without a context, with key filled the way DataJoint 2.3 fills it, and expect that error. The round trips through a real table check that DataJoint 2.3.4 passes the context on insert and on fetch.
The job set up "3.x", which now resolves to Python 3.15. Its pylint session installs the package and so datajoint, whose dependency deepdiff resolves to 9.1, which needs cachebox 5. cachebox 5 has no Python 3.15 build and fails to compile, so the job fails. It runs on 3.14 until datajoint 2.3.5, which supports Python 3.15 and keeps deepdiff below 9 there, is on PyPI. No other job resolves to 3.15: the test matrix names its versions, and the build action in CD defaults to 3.14.
dimitri-yatsenko
requested review from
MilagrosMarin,
lum-agilyti and
ttngu207
October 20, 2026 23:14
dimitri-yatsenko
marked this pull request as draft
October 21, 2026 16:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Releases of dj-plotly-codecs become annotated tags, the same process datajoint/datajoint-python#1588 adopts: a release is a tag
vX.Y.Zonmainwhose message is the release notes, and pushing it builds, publishes to PyPI, and creates the GitHub release from that message. The package has no release yet, so this lands before the first one. The PR also makes the package ready for that release: it publishes to pypi.org, requires the DataJoint version its codec needs, and fixes the failing Format job.Changes
Each item is one commit.
Publish a release from its annotated tag
cd.ymlruns on a push of avX.Y.Ztag instead of on a published GitHub release. Pull requests and pushes tomainstill build the package.checkjob runs.github/scripts/check_release_tag.sh, the script datajoint-python uses, unchanged. It refuses a tag that is notvX.Y.Z, a lightweight tag (and prints thegit tag -acommand that replaces it), and a tag whose commit is not onmain.ci.ymlcan now be called fromcd.yml, and publishing waits for its lint and test jobs.releasejob creates the GitHub release from the tag's message, with the wheel and sdist attached.mainor a pull request still replaces a build in progress, but a release run is never cancelled.ci.ymlnames its concurrency groupCIexplicitly. In a run called from CD,github.workflowisCD, and a group shared with the caller makes GitHub cancel the run.CONTRIBUTING.mdhas a new "Releasing" section.Publish to PyPI, not TestPyPI: the publish step drops
repository-url: https://test.pypi.org/legacy/.Require datajoint 2.3.4: the codec calls
self._extract_context(key, context)andself._codec_config(key, context), which DataJoint adds in 2.3.4. Under 2.3.0 to 2.3.3, whichdatajoint>=2.3allowed, every encode and decode would fail. The README states the requirement, anduv.lockchanges only in its recorded specifier.Read the codec context only: DataJoint 2.3.4 and later pass
contextto every codec that declares it:schema,table,field, andconfigon insert, andconfigon fetch. The codec now reads them there and nowhere else, with no fallback to the underscore keys inkey, which DataJoint 2.4 removes (datajoint/datajoint-python#1557). Without a context, it raises aDataJointErrorthat says DataJoint 2.3.4 and later pass one.Run the Format job on Python 3.14:
"3.x"now resolves to 3.15. The job's pylint session installs datajoint, which pulls in deepdiff 9.1, and deepdiff 9.1 needs cachebox 5, which has no Python 3.15 build. The pin stays until datajoint 2.3.5 is on PyPI. No other job resolves to 3.15.Why
cd.ymland thepypienvironment keep their namesA PyPI trusted publisher names the repository, the workflow file, and the environment. If either the file or the environment were renamed, PyPI would refuse the upload until its publisher was changed to match.
Before the first release
Pending trusted publisher on PyPI, under the DataJoint organization: project
dj-plotly-codecs, repositorydatajoint/dj-plotly-codecs, workflowcd.yml, environmentpypi. The project is created, owned by the organization, on the first upload.Optionally, protect the
pypienvironment (Settings, Environments): add required reviewers, or limit deployments to tags matchingv*. The environment does not exist yet. Without this step, GitHub creates it, unprotected, on the first release.Merge this PR, then tag
mainwith the release notes innotes.mdand push the tag:The notes start with one sentence that says what the release is about. After a blank line come the changes, grouped by area, each naming its pull request.
--cleanup=verbatimkeeps Markdown headings, which git would otherwise drop as comments. The version comes from the tag through hatch-vcs, so nothing in the repository is edited.Testing
cd.ymlandci.yml, and so do the repository's pre-commit hooks.check_release_tag.shis byte-identical to datajoint-python's. On a scratch repository withmain, it accepts an annotated tag onmainand refuses a lightweight tag, a tag offmain, and a tag that is notvX.Y.Z. It accepts a tag with aRetroactive:line but does not publish it.v0.1.0producesdj_plotly_codecs-0.1.0.tar.gzanddj_plotly_codecs-0.1.0-py3-none-any.whl, which the version check accepts. It refuses them for av0.1.1tag.pytestwith datajoint 2.3.4 and MySQL 8 in Docker: 30 passed, with warnings treated as errors, so no deprecation warnings either. The round trips through a real table exercise the context on insert and on fetch, and new tests check the error when there is no context.