Repository navigation
Conversation
Unit tests mock the API client, so they check the CLI against the author's reading of the API, not the API itself. A new CI check fails PRs that touch src/ unless the description's Artifacts section holds real output (a code block) or a screenshot. The skip-artifacts label opts out for changes with nothing to run. The PR template and CONTRIBUTING.md explain what to include.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unit tests mock the API client, so they check the CLI against the author's reading of the API, not the API itself. This adds a CI check that a PR changing
src/shows the change working against a real API in an## Artifactssection of its description.Running the open PRs against a live API turned up bugs their unit tests had passed. These included attachment uploads that failed on every call and a
statistical-journals listcrash on real rows. This check makes that step part of every PR.Changes
scripts/check_artifacts.py: fails when the PR touchessrc/(the same rule as the changelog check, reusingneeds_note) and the description has no## Artifactssection, or the section has no evidence. Evidence means a non-empty fenced code block, an image, a video, or a GitHub upload link. Template comments are ignored, so the untouched template and text like "N/A" fail..github/workflows/artifacts.yml: runs the check onpull_request, includingedited, so adding artifacts to the description re-runs it without a new push. The body is passed throughenv, not interpolated into the script. Askip-artifactslabel opts a PR out.tests/test_check_artifacts.py.Test plan
uv run pytest): 305 passed.uv run ruff check src/ tests/ scripts/anduv run ruff format --check src/ tests/ scripts/)src/; see Artifacts.Artifacts
Run against the live descriptions of #39–#42, the new template, and some failing cases. A throwaway commit touching
src/made each one count as a user-facing change:Without the throwaway commit (this branch only changes
.github/,scripts/,tests/and docs):Not exercised: the workflow itself on GitHub. This PR doesn't touch
src/, so the job reports the exemption rather than checking a body.Needs a maintainer
skip-artifactslabel.The check confirms evidence is present, not that it's convincing; that's still the reviewer's call.