Skip to content

ci(release): publish the built files to TestPyPI and PyPI - #784

Merged
masquare merged 1 commit into
ci/release-open-prfrom
ci/release-publish
Nov 18, 2026
Merged

masquare merged 1 commit into
ci/release-open-prfrom
ci/release-publish

Conversation

@masquare

@masquare masquare commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Stack: automated per-package releases. Review the PRs in this order. Each PR shows only its own layer.

  1. ci(release): generate release notes per package #781 Release notes
  2. ci(release): open a release PR and tag the release when it merges #782 Open a release PR and tag the release when it merges
  3. ci(release): publish the built files to TestPyPI and PyPI #784 Publish to TestPyPI and PyPI
  4. ci(release): open the PR for the next development version #785 Open the next development version PR

main keeps working after each merge. Nothing changes how packages are published until #784 deletes publish.yml.

What this PR does

The release job now keeps the built files as a workflow artifact. A TestPyPI job and then a PyPI job upload those files. Each job waits for approval in the pypi environment. The upload jobs build nothing, so the files on PyPI are the files that the GitHub release holds. This PR deletes publish.yml.

Where to look

  • Both jobs use trusted publishing (OIDC). Their only permission is id-token: write, and they do not check out the repository.
  • publish-pypi starts only after publish-testpypi succeeds.
  • uv publish --check-url skips files that the index already has, so a re-run after a partial upload sends the rest.

Setup before you merge

On both PyPI and TestPyPI, for both timenet and timenet-connectors, set the trusted publisher to owner OpenTSLM, repository TimeNet, workflow release.yml, environment pypi. Change an existing publisher from publish.yml, or add a pending publisher for a new project. The pypi environment already has its reviewers.

Testing

The TestPyPI upload ran for real in a private sandbox repository, with dummy projects. The PyPI job ran there with its upload skipped, because the sandbox has no PyPI project. The first real PyPI upload happens with the first release.

@masquare
masquare marked this pull request as ready for review November 13, 2026 15:32
@masquare
masquare requested a review from karhu November 13, 2026 15:33
@masquare
masquare changed the base branch from ci/release-tag to ci/release-open-pr November 13, 2026 15:39
@masquare
masquare changed the base branch from ci/release-open-pr to ci/release-notes November 13, 2026 18:33
@masquare
masquare changed the base branch from ci/release-notes to ci/release-open-pr November 13, 2026 18:33
@masquare
masquare added this pull request to stack #787 November 13, 2026 18:48
The release job keeps the built files as a workflow artifact. A TestPyPI job and then a PyPI job upload those files from the pypi environment, each after approval, and build nothing. This replaces publish.yml, so the trusted publishers on PyPI and TestPyPI must point at release.yml before this merges.
@masquare
masquare merged commit c482db9 into main Nov 18, 2026
4 checks passed
@masquare
masquare deleted the ci/release-publish branch November 18, 2026 22:11
masquare added a commit that referenced this pull request Nov 18, 2026
**Stack: automated per-package releases.** Review the PRs in this order.
Each PR shows only its own layer.

1. #781 Release notes
2. #782 Open a release PR and tag the release when it merges
3. #784 Publish to TestPyPI and PyPI
4. #785 Open the next development version PR

`main` keeps working after each merge. Nothing changes how packages are
published until #784 deletes `publish.yml`.

## What this PR does

Adds the two pieces that write release notes. `cliff.toml` configures
git-cliff for the timenet notes. `connector_notes.py` writes the
timenet-connectors notes. They list the connectors that were added,
changed or removed, and the changes to the shared connector code. Core
changes that only touched connector files on the way are left out,
because the timenet notes list them.

## Where to look

- `cliff.toml`: the commit groups, and the pre-1.0 bump rule (a breaking
change or a feature raises the minor version).
- `connector_notes.py`: a connector is a `datasets/<org>/<name>/` folder
with a `dataset.yaml`. It is matched by `dataset_id`, so a moved folder
is a change and a new id is one removal and one addition. A connector PR
is a `feat`, a `fix` or a breaking change that did not touch
`packages/timenet`.

## Not in this PR

No workflow calls these files yet. PR 2 does.

## Example output

<details>
<summary>Notes for a release of both packages</summary>

```markdown
## timenet 0.21.0

[![timenet 0.21.0](https://img.shields.io/badge/timenet-0.21.0-blue?logo=pypi&logoColor=white)](https://pypi.org/project/timenet/0.21.0/)

### Features

- **test-mean:** Sandbox timenet change 8 (#925)

## timenet-connectors 0.21.0

[![timenet-connectors 0.21.0](https://img.shields.io/badge/timenet--connectors-0.21.0-blue?logo=pypi&logoColor=white)](https://pypi.org/project/timenet-connectors/0.21.0/)

### Connector framework

- **test-mean:** Sandbox connectors change 8 (#926)
```

</details>

## Testing

The whole stack ran end to end in a private sandbox repository. This PR
passes ruff, ty and the pre-commit hooks on its own.

---------

Co-authored-by: masquare <11543761+masquare@users.noreply.github.com>
masquare added a commit that referenced this pull request Nov 18, 2026
**Stack: automated per-package releases.** Review the PRs in this order.
Each PR shows only its own layer.

1. #781 Release notes
2. #782 Open a release PR and tag the release when it merges
3. #784 Publish to TestPyPI and PyPI
4. #785 Open the next development version PR

`main` keeps working after each merge. Nothing changes how packages are
published until #784 deletes `publish.yml`.

## What this PR does
Adds the `Release` workflow. A maintainer runs it on `main` and picks
`timenet`, `timenet-connectors` or `both`. It opens a release PR that
sets the version of each package and holds the release notes in the PR
body, between `release-notes` markers. The notes can be edited in the PR
before it is merged.

When the release PR is merged to `main`, the same workflow builds the
wheel and the sdist of each released package, smoke tests them, creates
the tag, and creates the GitHub release with the files attached. The
release notes are the text between the markers in the merged PR.

## Where to look
- `authorize` job: only admins and maintainers can run it.
- `prepare` job: it makes a GitHub App token, so CI runs on the release
PR. `GITHUB_TOKEN` cannot do that.
- `release.py prepare`: the version comes from git-cliff or from the
input. It fails if the tag already exists. A release of both packages
raises the timenet floor of timenet-connectors. A release of one package
warns when the other package has changes that it leaves out.
- Trigger of the release job: a push to `main` whose commit message
starts with `chore(release):`. A squash merge uses the PR title, so do
not edit the title when you merge.
- `released`: finds the packages whose version the PR changed.
- `smoke`: imports each file in an isolated environment and makes sure
that it records this commit.
- `github-release`: reads the notes from the merged PR and creates the
tag and release. A re-run creates only what is missing.
- Permissions: `contents: write` and `pull-requests: read`, for the
release job only.

## Not in this PR
Uploading to TestPyPI and PyPI. `publish.yml` still does that. #784
replaces it.

## Setup
The GitHub App, the `RELEASE_APP_ID` variable and the
`RELEASE_APP_PRIVATE_KEY` secret are set on this repository already.

## Testing
Run end to end in a private sandbox repository, with a real merge and a
hand edit of the notes in the PR. The script passes ruff and ty, and the
workflow passes actionlint.

---------

Co-authored-by: masquare <11543761+masquare@users.noreply.github.com>
masquare added a commit that referenced this pull request Nov 18, 2026
**Stack: automated per-package releases.** Review the PRs in this order.
Each PR shows only its own layer.

1. #781 Release notes
2. #782 Open a release PR and tag the release when it merges
3. #784 Publish to TestPyPI and PyPI
4. #785 Open the next development version PR

`main` keeps working after each merge. Nothing changes how packages are
published until #784 deletes `publish.yml`.

## What this PR does

Right after the release job, a separate job opens a PR that moves each
released package to its next development version, for example
`0.13.0.dev0`. Then builds from `main` do not look like a release. The
PR has auto-merge on, so it merges when it has the approvals that `main`
needs.

## Where to look

- A package moves on only when `main` still has the released version. A
re-run after the merge changes nothing.
- The job is separate from the upload jobs, so the App key never runs
next to the PyPI token.
- It does not wait for PyPI. The tag already fixes the released version,
also when an upload fails.

## Setup

Turn on the repository setting "Allow auto-merge". It is off now.
Without it, the job warns and the PR waits for a manual merge.

## Testing

Run end to end in a private sandbox repository. The PR merged by itself
29 seconds after an approval.

Co-authored-by: masquare <11543761+masquare@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants