Skip to content

fix(ci): remove the workflow-level permissions cap that kills fork PR runs - #257

Merged
shockalotti merged 1 commit into
mainfrom
fix/fork-pr-workflow-permissions
Nov 17, 2026
Merged

shockalotti merged 1 commit into
mainfrom
fix/fork-pr-workflow-permissions

Conversation

@shockalotti

Copy link
Copy Markdown
Contributor

Docker publish and NS8 module publish never start on pull requests from forks. The run is created with zero jobs, so there is no log, no annotation, and --log-failed returns nothing. The only trace is a check suite concluding startup_failure or failure.

Cause

#231 added a workflow-level permissions: contents: read to both files, for least privilege. That is the only change to either file between the last green fork run and the first red one.

  • Last green fork PR: 2026-11-06 18:55
  • First red: 2026-11-11 20:29
  • Merged between: #232, #231, #233, #245

#247 and #252 merged after the first failure, so they were never candidates. #233 only rewrote uses: pins. #231 is the one, and it changed exactly one thing per file.

What it is not, which is why this took a week

It is not "workflows that declare packages: write are refused on forks". At the last green fork run (main 98b5799) the job-level block was already there:

build: {contents: read, packages: write}
top-level: absent

Fork PRs passed with that for months. The declaration is necessary for the break but it is not what changed, so toning it down would not have been the fix.

It is not "a job escalating above a workflow-level cap" either. Audit has a top-level read-all, and its scorecard job escalates to security-events: write and id-token: write. Audit ran SUCCESS with 3 jobs on fork head fe75122, the same head where these two workflows produced zero jobs.

So the failing shape is narrower: a workflow-level cap plus a job asking for packages write.

The signature, and why it proves expansion

On head fe75122:

workflow result jobs
CLA Assistant success 1
CodeQL success 2
CI success 1
Audit success 3
Docker publish failure 0
NS8 module publish failure 0

The decisive detail is ns8-module.yml: its validate job needs only contents: read, and its build job is the one asking for packages: write. Yet validate never ran either, because the whole workflow produced zero jobs.

So the run is rejected before any job exists. That is a workflow-expansion failure, not a build failure wearing a strange exit code, which is why there is no log to read.

The fix

Remove the workflow-level block from both files, and leave a comment in its place saying there is deliberately none and why, so nobody re-adds it as a tidy-up. The reasoning belongs in the file, not only in a PR description that nobody will find in two months.

Least privilege is not weakened. The repository default is already read (default_workflow_permissions=read), so a job without an explicit block still gets read, and every job in both files declares its own permissions anyway. What is lost is redundancy, not a guarantee.

Not done on purpose: moving the packages: write work into a separate off-PR job. That is the better end state if the PR path and publish path ever need to differ more than they do today, but it is a larger diff solving a problem this does not.

Diff

Comments, plus the removal of two lines in each file. No other workflow, no other permission, no pin changes.

Unproven, stated plainly

There is no GitHub documentation citation here for packages write behaviour on forks under an explicit cap. The evidence is the A/B above, a perfect correlation across every fork head, and a counterexample that narrows the rule.

If fork runs still produce zero jobs after this, the rule is wider and this is not the whole story. The test is to re-run both workflows on the live fork PR #256 and check whether jobs > 0.

… runs

Docker publish and NS8 module publish never start on pull requests from forks.
The run is created with ZERO jobs, so there is no log and no annotation, and the
only visible trace is a check suite concluding startup_failure or failure.

CAUSE. #231 added a workflow-level `permissions: contents: read` to both files
for least privilege. That is the only change to either file between the last
green fork run and the first red one, and removing it restores them.

WHAT IT IS NOT, which is why this took a week.

It is not "workflows that declare packages: write are refused on forks". At the
last green fork run (main 98b5799, 2026-11-06) the job-level

    build: {contents: read, packages: write}

was already present and fork PRs passed for months. The declaration is necessary
for the break but it is not what changed, so toning it down would not have been
the fix.

It is not "a job escalating above a workflow-level cap" either. Audit has a
top-level `read-all` and its scorecard job escalates to security-events: write
and id-token: write, and Audit ran SUCCESS with 3 jobs on fork head fe75122, the
same head where these two workflows produced zero jobs. So the failing shape is
narrower: a workflow-level cap plus a job asking for PACKAGES write.

THE SIGNATURE, and why it proves the failure is at expansion.

On head fe75122:
    CLA Assistant        success   jobs=1
    CodeQL               success   jobs=2
    CI                   success   jobs=1
    Audit                success   jobs=3
    Docker publish       failure   jobs=0
    NS8 module publish   failure   jobs=0

The decisive detail is ns8-module.yml: its `validate` job needs only
contents: read, and its `build` job is the one asking for packages: write. Yet
`validate` never ran either, because the whole workflow produced zero jobs. So
the run is rejected before any job exists. That is a workflow-expansion failure,
not a build failure wearing a strange exit code.

THE FIX. Remove the workflow-level block from both files and leave a comment in
its place saying there is deliberately none, and why, so nobody re-adds it as a
tidy-up.

Least privilege is not weakened. The repository default is already `read`
(default_workflow_permissions=read), so a job without an explicit block still
gets read, and every job in both files declares its own permissions anyway. What
is lost is redundancy, not a guarantee.

Deliberately not done: moving the packages: write work into a separate off-PR
job. That is the better end state if the PR and publish paths ever need to differ
more than they do, but it is a larger diff solving a problem this does not.

UNPROVEN, stated plainly. There is no GitHub documentation citation here for
packages-write behaviour on forks under an explicit cap. The evidence is the A/B
above, a perfect correlation across every fork head, and a counterexample that
narrows the rule. If fork runs still produce zero jobs after this, the rule is
wider and this is not the whole story.

Diff is comments plus the removal of two lines in each file. No other workflow or
permission was touched.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — the removal itself is token-neutral, but the two comment blocks record a diagnosis the evidence does not support yet.

Approach: removing the workflow-level block is safe here — every job in both files already declares its own permissions, and a job block replaces rather than inherits, so no token changes on any event — but that also means GitHub's documented fork rule (write downgraded to read, applied last) predicts no effect at all, so this is a workaround for an undocumented cause, not a demonstrated fix.

Reviewed changes

  • Workflow-level cap removed from docker-publish.yml — the permissions: {contents: read} block from #231 is gone, leaving each job's own block to govern its token.
  • Workflow-level cap removed from ns8-module.yml — same removal, and the comment carries the useful observation that validate never ran either, because a rejected run has no jobs at all.
  • Both removals are replaced by 16- and 18-line comment blocks — recording the #231 attribution, the zero-job signature, the packages: write combination, and a "do not re-add" instruction.

ℹ️ The fix can't be verified by re-running the fork PR's existing runs

The comments state the fork runs were restored, but nothing here demonstrates that yet — and the obvious check will not produce the evidence. A re-run reuses the original merge-ref commit, so PR #256's existing failing runs will keep executing main's pre-fix copy of both workflows after this merges. The demonstration requires a fresh commit on the fork branch (a push, or a close/reopen) once this is on main, and then a non-empty jobs list on the new runs.

Also, so it doesn't send you chasing it: the one red check on this PR (ns8-module → validate) is a read: connection reset by peer fetching a GHCR blob in the pre-existing registry-backed manifest selftest, not this diff — validate declares contents: read and never touches packages.

Technical details
# Unverified claims in the replacement comments

## Affected sites
- `.github/workflows/docker-publish.yml:40-41` — "Removing it restored them" (past tense, nothing restored yet)
- `.github/workflows/ns8-module.yml:41-42` — the same claim

## Required outcome
- The comments distinguish observation from inference: zero-job runs with no log, starting when the cap landed, is the observation; the mechanism is undetermined.
- When a fork run with `jobs > 0` exists, cite that run and then say "restored".

## Suggested approach
- Reword both to: "observed on fork pull requests: zero jobs, no log, `startup_failure`/`failure`. The mechanism is not documented by GitHub; if fork runs still fail after this, the rule is wider than the cap."
- Keep the pointer to the PR for the A/B, which is already there and is the right way to carry the evidence.

## Open questions for the human
- Was either workflow ever executed from a fork with the block absent (a fork push of this branch, say)? If so, linking that run would settle the whole question and make the comment accurate as written.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash | 𝕏

Comment on lines +39 to +48
# One was added in #231 for least privilege. It broke every run from a fork
# pull request: the run failed during workflow expansion, so it was created with
# ZERO jobs and produced no log and no annotation. Removing it restored them.
#
# The combination is what matters, not either half. The build job below asks for
# `packages: write` to push to GHCR, and a fork pull request can never be granted
# a write token. On its own that is harmless, because GitHub downgrades it and
# the job runs. Underneath a workflow-level cap, the same request was observed to
# make GitHub reject the run before any job existed. Evidence, the full A/B and
# the counterexample that narrows this are in the PR that removed it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two claims in this block will outlive the PR and neither holds as written. Removing it restored them is past tense about something not yet observed: main still carries the block, and no fork run has exercised this branch. And the packages-specific narrowing does not survive the counterexample it cites — audit.yml is contents: read, not read-all as the PR body says, and its only write-scoped job (Scorecard, security-events: write + id-token: write) is gated if: github.ref == 'refs/heads/main' and was skipped on the fork head. Set that aside and the evidence fits the broader cap + any job-level write scope shape, which is what whoever re-adds a cap next to a pull-requests: write job needs to know.

Technical details
# Overstated diagnosis in the replacement comments

## Affected sites
- `.github/workflows/docker-publish.yml:40-48` — "Removing it restored them" plus the "combination is what matters" paragraph naming `packages: write`
- `.github/workflows/ns8-module.yml:40-51` — the same two claims, mirrored
- `.github/workflows/audit.yml:23-24` — top-level block is `contents: read`, not `read-all`
- `.github/workflows/audit.yml:33-37` — Scorecard is the only job with a write scope and is `main`-gated; run `43449040419` on fork head `fe75122` shows SBOM ok, claims guard ok, Scorecard skipped

## Required outcome
- The comments separate what was observed (zero-job runs, no log, no annotation, coincident with the cap landing) from what is inferred (the mechanism), and do not assert a restore that has not happened.
- The generalisation is stated at the width the evidence supports: with the Audit counterexample removed, nothing distinguishes `packages` from any other write scope, so the shape to avoid is a workflow-level cap plus a job-level write on a workflow that runs on fork PRs.
- Keep the "do not re-add this as a tidy-up" warning — it is the durable value in this block.

## Suggested approach
- Observation-only wording, e.g. "Fork pull requests produced zero-job runs (`startup_failure`, no log, no annotation) once this cap was added; the mechanism is undocumented, so if fork runs still fail, the rule is wider than this."
- If the intended end state is worth recording, note it here: a PR-path workflow with no write scopes at all (publish split off), which is what #231's least-privilege intent actually needed.
- Drop `Removing it restored them` until a fork run with a non-empty job list exists, then cite it.

@shockalotti
shockalotti merged commit 6b8e6de into main Nov 17, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant