Skip to content

fix(ns8): normalise the repository argument, and test that it cannot drift - #154

Merged
shockalotti merged 2 commits into
mainfrom
fix/ns8-repo-arg-normalisation
Oct 16, 2026
Merged

shockalotti merged 2 commits into
mainfrom
fix/ns8-repo-arg-normalisation

Conversation

@shockalotti

Copy link
Copy Markdown
Contributor

Your apology was not needed and the correction was right. Not dropping row 4 was the only reason this got found at all.

The mechanism, measured

Three spellings resolve, one does not, and I can now say why:

calnode/ns8-calnode              200
Calnode/ns8-calnode              200
ghcr.io/calnode/ns8-calnode      200
ghcr.io/Calnode/ns8-calnode      404   <- what the workflow passes

With no extra leading segment the repository is matched case-insensitively. With one extra leading segment, that segment is tolerated whatever it is, and the segments after it are matched case-sensitively:

/v2/ghcr.io/calnode/ns8-calnode       -> 200
/v2/quay.io/calnode/ns8-calnode       -> 200
/v2/zzz.example/calnode/ns8-calnode   -> 200
/v2/GHCR.IO/calnode/ns8-calnode       -> 200
/v2/ghcr.io/Calnode/ns8-calnode       -> 404
/v2/quay.io/Calnode/ns8-calnode       -> 404
/v2/zzz.example/Calnode/ns8-calnode   -> 404
/v2/calnode/Calnode                   -> 404

Those 404s are GitHub's edge router, not the registry. The body is 404 page not found in text/plain. A genuinely absent repository returns JSON with MANIFEST_UNKNOWN. That is the discriminator, and it is what tells us nothing about the image or the label is wrong.

So: the failure is routing, and it only manifests when the caller supplies the redundant host segment. The script builds https://ghcr.io/v2/{repo}/... itself, so a ghcr.io/ prefix does not belong in that path at all; ghcr merely tolerates it, which is exactly why it went unnoticed until github.repository_owner handed us a capital C.

This also corrects me twice over

d5cc8ee lowercased the repo for the token scope, and I read that as half a fix for a case problem that does not exist. The manifest endpoint is case-insensitive. The real fault is the redundant host prefix, which only becomes fatal once the owner is capitalised. Your curl reached the right conclusion for the wrong reason and mine reached the wrong one for an unproved reason. Neither of us ran the four forms before asserting a mechanism, which is the actual lesson.

The fix

One normaliser in token(), get() and get_blob(): strip an optional leading ghcr.io, then lowercase. All three, so they cannot drift apart again, which is how the half-fix happened in the first place.

One place I pushed back on your suggested fix, narrowly. Stripping "an optional leading registry host" generically would be wrong here: a dotted first segment is a legal image name, so my.org/team/image must not become team/image. I strip only the host this script actually talks to. That case is tested explicitly.

Four spellings, one outcome:

calnode/ns8-calnode              exit=0
Calnode/ns8-calnode              exit=0
ghcr.io/calnode/ns8-calnode      exit=0
ghcr.io/Calnode/ns8-calnode      exit=0

Enforced, not spot-checked

packaging/ns8/test_assert_multiarc.py, wired into the validate job so it runs on every push. A regression fails immediately instead of waiting for a release.

The four-way equivalence is one invariant over four inputs, alongside the dotted-owner case, idempotence, case-insensitive host matching, and an assertion that all three fetch paths call the normaliser. The live registry round-trip lives in the same file but is opt-in behind NS8_REGISTRY_TESTS=1, so CI does not need ghcr to be up in order to accept a source change. Locally they pass against the real registry: 9 tests, 11.3s, all four spellings resolving to the same digest.

Re-verified, unchanged from #151

module-only exit 0, --pair --expect-app-tag 0.11.1 exit 0, dead --app-repo still exit 2, missing app image still bounded failure in 2.2s, mismatched pair still exit 1, gate logic still verify.

Your point about blocked being rc=0

Noted and I agree it is right, but it deserves to be a conscious choice rather than an accident. blocked currently concludes the job green while saying ::error::. When ns8-module.yml is green this workflow will go green, which is what we want, but a future reader could easily read a green tick as "the release was verified" when the real meaning is "there was nothing to verify". If you want, the honest version is a step summary or a job name that carries the outcome. I have not done it because it is a presentation change and this PR is a fix, but flag it rather than let it pass silently.

No tag pushed, nothing merged from me. Worktree /tmp/ns8-case-fix still in place.

…drift

Main has been red on every push because ns8-module.yml passes
`ghcr.io/${{ github.repository_owner }}/ns8-calnode` and that exact string
cannot be fetched, while the image it names is present and multi-arch.

THE MECHANISM, measured rather than assumed. Three spellings resolve and one
does not:

    calnode/ns8-calnode            200
    Calnode/ns8-calnode            200
    ghcr.io/calnode/ns8-calnode    200
    ghcr.io/Calnode/ns8-calnode    404   <- what the workflow passes

With no extra leading segment the repository is matched case-insensitively, so
capital C is fine. With ONE extra leading segment that segment is tolerated
whatever it is, and the segments after it are matched case-sensitively:

    /v2/ghcr.io/calnode/ns8-calnode   -> 200
    /v2/quay.io/calnode/ns8-calnode   -> 200
    /v2/zzz.example/calnode/ns8-calnode -> 200
    /v2/ghcr.io/Calnode/ns8-calnode   -> 404
    /v2/quay.io/Calnode/ns8-calnode   -> 404
    /v2/zzz.example/Calnode/ns8-calnode -> 404

Those 404s come from GitHub's edge router, not from the registry. The body is
`404 page not found` in text/plain, whereas a genuinely absent repository
returns JSON with MANIFEST_UNKNOWN. So nothing about the image or the label is
wrong; the failure is routing, and it only appears when the caller supplies the
redundant host segment.

That also corrects a conclusion I got wrong twice this week. d5cc8ee lowercased
the repo for the token scope and I read it as half a fix for a case problem that
did not exist. The manifest endpoint is case-insensitive. The actual fault is the
redundant host prefix, and it only becomes fatal once the owner is capitalised,
which is what github.repository_owner hands us verbatim.

THE FIX. One normaliser applied in token(), get() and get_blob(): strip an
optional leading ghcr.io, then lowercase. Applied at all three so they cannot
drift apart, which is how the earlier half-fix happened.

Only the host this script talks to is stripped. An arbitrary dotted first
segment is left alone, because a dotted owner is a legal image name and
`my.org/team/image` must survive intact; rewriting it to `team/image` would be a
far worse failure than the 404 being prevented. That case is tested.

Four spellings now produce one outcome:

    calnode/ns8-calnode              exit=0
    Calnode/ns8-calnode              exit=0
    ghcr.io/calnode/ns8-calnode      exit=0
    ghcr.io/Calnode/ns8-calnode      exit=0

ENFORCED, NOT SPOT-CHECKED, as asked. packaging/ns8/test_assert_multiarc.py runs
in the validate job on every push, so a regression fails immediately instead of
waiting for a release. The four-way equivalence is asserted as one invariant
over four inputs, alongside the dotted-owner case, idempotence, case-insensitive
host matching and that all three fetch paths call the normaliser. The live
registry round-trip tests are in the same file but opt-in behind
NS8_REGISTRY_TESTS=1, so CI does not depend on ghcr being up to accept a source
change. Locally they pass against the real registry: 9 tests, 11.3s, all four
spellings resolving to the same digest.

Re-verified, unchanged from #151: module-only exit 0, --pair --expect-app-tag
0.11.1 exit 0, dead --app-repo still rejected with exit 2, missing app image
still fails bounded in 2.2s, mismatched pair still exit 1, gate logic still
verify.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The fix is right and I confirmed it end to end — but this commit also adds a
zero-byte file named 404 at the repo root, which is a leftover from the curl
probing described in the description. Please delete it before merging.

Reviewed changes

  • A normalize_repo() normaliser in packaging/ns8/assert-multiarc.py that strips an optional leading ghcr.io/ segment and then lowercases, replacing the bare repo.lower() that token() used to do.
  • Applied at all three fetch paths (token(), get(), get_blob()), so the URL and the token scope are built from the same string and cannot drift apart again.
  • packaging/ns8/test_assert_multiarc.py (new, 121 lines): seven offline tests covering the four-way spelling equivalence, case-insensitive host matching, the dotted-owner passthrough, idempotence, whitespace, plus a source-level assertion that all three fetch paths call the normaliser.
  • Opt-in registry round-trip in the same file behind NS8_REGISTRY_TESTS=1, so CI does not need ghcr reachable to accept a source change.
  • ns8-module.yml now runs py_compile plus unittest discover in the validate job, so a regression fails on every push rather than at release time.

I re-ran both suites. Offline: 9 tests, 7 pass, 2 skipped, 0.001s. Opt-in: all four spellings pass check_tag against the real registry. release-verify.yml:173 passes the same MODULE_REPO shape, so it is covered by the same fix.

⚠️ A zero-byte file named 404 is committed at the repo root

80f9be7 adds an empty 404 at the repository root (0 bytes, 0 insertions). Nothing references it, and test_assert_multiarc.py does not read it — it is the shell redirect from the curl -o 404 probing in the description. It has no functional effect, but it is an unexplained artifact and it should not become part of history.

Technical details
# Remove the stray `404` file

## Affected sites
- `404` (repo root) — added empty by commit 80f9be7, confirmed 0 bytes and
  referenced by nothing in the tree or the new test.

## Required outcome
- `404` is not present at the repository root after this PR merges. The removal
  can ride in a follow-up commit on this branch; it does not need a rebase.

## Note
- `py_compile test_assert_multiarc.py` in the validate job already pins the
  test filename, so the "discover matches nothing and silently passes" risk is
  covered — no extra guard is needed for that.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using space-bunny-free (free) | 𝕏

def test_every_fetch_path_normalises(self):
"""get(), get_blob() and token() must all agree, not just one of them."""
for name in ("get", "get_blob"):
src = getattr(am, name).__doc__ or ""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src is assigned and then deleted without ever being read, and inspect is reached through an inline __import__("inspect") twice when the module already imports four other stdlib modules at the top. This is the one test that enforces "the three paths cannot drift", so it should read as the deliberate check it is. Add import inspect to the import block and delete the dead pair.

Technical details
# Tidy test_every_fetch_path_normalises

## Affected sites
- packaging/ns8/test_assert_multiarc.py:17-20 — import block, needs `import inspect`
- packaging/ns8/test_assert_multiarc.py:76-78 — `src = ...` / `del src` dead pair
- packaging/ns8/test_assert_multiarc.py:79 — second `__import__("inspect")`

## Required outcome
- `inspect` is imported at module level, in the existing stdlib group.
- The loop body is the single `assertIn` against `inspect.getsource(...)`.
- No assigned-then-deleted local remains.

## Do not change
- The assertion itself. `inspect.getsource` is the right tool here: the
  invariant is "every fetch path normalises", which is a property of the source
  text, and a behavioural test would need the network the offline suite
  deliberately avoids.
Suggested change
src = getattr(am, name).__doc__ or ""
self.assertIn("normalize_repo", inspect.getsource(getattr(am, name)))

Committed by accident in 80f9be7. It does not exist on main and there is no
reason a file of that name should ever exist in this repository.

No .gitignore rule for it, deliberately. An ignore rule would hide the whole
class of mistake, which is shell output or a redirect landing in the repository
root under whatever name it happened to be given. A file called 404 appearing at
the root is worth seeing, not worth muting.
@shockalotti
shockalotti merged commit b5b2f10 into main Oct 16, 2026
7 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant