Skip to content

packaging(ns8): poll for the route instead of checking it once - #145

Merged
shockalotti merged 2 commits into
mainfrom
packaging/ns8-robot-route-wait
Oct 15, 2026
Merged

shockalotti merged 2 commits into
mainfrom
packaging/ns8-robot-route-wait

Conversation

@shockalotti

Copy link
Copy Markdown
Contributor

Found by running the robot suite against a real cluster for the first time, after #144 wired it up.

Check if ns8-calnode works as expected curled the host route immediately after configure-module returned. But Traefik registers the route asynchronously a few seconds later, so the request arrives before the route exists:

curl: (22) The requested URL returned error: 404

It reproduced on two consecutive runs and on a freshly built cluster, so it was not a one-off. The module is fine: measured on the cluster, the route answers 302 within about 5s of configure and /admin/ returns 200. The test was asserting on Traefik's registration latency.

The check is now wrapped in Wait Until Keyword Succeeds 60x 5s. Polling rather than sleeping a fixed amount keeps the suite fast when the route is already up, and still tolerates a slow node.

Verification

Suite now passes against a live single-node NS8 cluster (Rocky 9.8, NS8 from ns8-core@ns8-stable), run the way the official run-ns8-tests runner does it:

robot -v NODE_ADDR:... -v IMAGE_URL:ghcr.io/calnode/ns8-calnode:edge \
  -v SSH_KEYFILE:... --skiponfailure unstable --exclude ui tests/
Check if ns8-calnode is installed correctly    | PASS
Check if ns8-calnode can be configured         | PASS
Check if ns8-calnode works as expected         | PASS
Check if ns8-calnode is removed correctly      | PASS
4 tests, 4 passed, 0 failed

The [ui] test is still the only one that has never executed; it needs RUN_UI_TESTS=true and the Playwright image.

Found by running the suite against a real cluster for the first time. The
'works as expected' test curled the host route immediately after
configure-module returned, but Traefik registers the route asynchronously a
few seconds later, so the request hit Traefik before the route existed:

    curl: (22) The requested URL returned error: 404

It reproduced on two consecutive runs and on a freshly built cluster, so it
was not a one-off. Measured on the cluster: the route is live and answering
302 within about 5s of configure, and /admin/ returns 200, so the module
itself is fine. The test was asserting on Traefik's registration latency.

Wrap the check in Wait Until Keyword Succeeds (60x5s). Polling rather than a
fixed sleep keeps the suite fast when the route is already up, and still
tolerates a slow node.

Suite now passes 4/4 against a live single-node NS8 9 cluster.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

One file, packaging/ns8/tests/10__calnode.robot, one commit — the route check that used to run once right after configure-module is now polled.

  • The route responds — new user keyword holding the unchanged curl (-f, explicit Host: ${TEST_HOST}, return_rc=True return_stdout=False) plus Should Be Equal As Integers ${rc} 0, so the single attempt is reusable and the retry has something to call.
  • Check if ns8-calnode works as expected — now Wait Until Keyword Succeeds 60x 5s The route responds, which returns as soon as Traefik has registered the route instead of asserting on its registration latency.
  • Comment — records why the wait exists (asynchronous Traefik route registration) and why polling beats a fixed sleep, so this doesn't get "cleaned up" back into a one-shot check.

A few things I checked before signing off, all of which hold: SSHLibrary.Execute Command returns a bare rc rather than a tuple when only return_rc=True is set, so the assertion reads correctly; the retry/interval form matches what NS8 core's own suites do (Wait Until Keyword Succeeds 10s 1s … in core/tests/10__cluster_sanity/); and the runner imposes no robot-level timeout, so the ~300 s budget sits comfortably inside the 60 min per-leg job timeout. The keyword lives in the same suite that uses it, and inherits the Suite Setup Connect to the Node connection from __init__.robot.

Worth stating explicitly since it's the trade-off you bought: a genuinely broken route now fails after five minutes of polling rather than instantly. That's the right call for a flake, and the final assertion message still names the curl rc, so the diagnosis survives.

The curl: (22) 404 reproduction on two consecutive runs against a freshly built cluster is exactly the evidence that makes this a fix rather than a guess — thanks for including it.

Pullfrog  | View workflow run | Using space-bunny-free (free) | 𝕏

The suite has now actually executed against a single-node NS8 cluster
(Rocky 9.8, ns8-core@ns8-stable): 4/4 without the browser and 5/5 with
RUN_UI_TESTS=true. Corrects the earlier claim that it had never been run.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ Docs-only delta this round — one README paragraph, no test or code behaviour changed. Two small accuracy fixes inline.

Reviewed changes

Only 3dfbbeb on top of the previously reviewed 644f057.

  • Live-run claim in the module README — the closing paragraph of packaging/ns8/README.md no longer says the robot suite "has still never been executed against a live cluster"; it now records a real run (4/4 without the browser, 5/5 with RUN_UI_TESTS=true, single-node ns8-core@ns8-stable on Rocky 9.8) and points at Testing as the place where the route-registration race is described.
  • Test code untouched — 10__calnode.robot is byte-identical to what the last review signed off, Wait Until Keyword Succeeds 60x 5s The route responds included. The earlier findings still hold, including the accepted trade-off that a genuinely broken route now takes ~5 min to fail.

The replacement paragraph is the right call in spirit: it drops a stale "every live check above was done by hand" disclaimer that the run has now overtaken. Two things in it don't yet line up with the rest of the PR.

  Pullfrog  | View workflow run | Using space-bunny-free (free) | 𝕏

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using space-bunny-free (free) | 𝕏

Comment thread packaging/ns8/README.md
has still never been executed against a live cluster. Every live check above
was done by hand. See [Testing](#testing).
The `tests/` robot suite has now been run against a live cluster: 4/4 without
the browser and 5/5 with `RUN_UI_TESTS=true`, on a single-node NS8 built from

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 5/5 with RUN_UI_TESTS=true claim is contradicted by the PR's own Verification section, which still ends with "The [ui] test is still the only one that has never executed; it needs RUN_UI_TESTS=true and the Playwright image." One of the two is stale — worth resolving in whichever is wrong, since the line this paragraph replaces existed precisely because an unverified claim about what had actually been run was a problem.

Comment thread packaging/ns8/README.md
The `tests/` robot suite has now been run against a live cluster: 4/4 without
the browser and 5/5 with `RUN_UI_TESTS=true`, on a single-node NS8 built from
`ns8-core@ns8-stable` on Rocky 9.8. Running it is what turned up the
route-registration race described in [Testing](#testing).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pointer says the race is described in Testing, but that section only covers which suite file does what, how to run locally, and the CI situation — it never mentions asynchronous route registration. Either add the one-line "the route check polls because Traefik registers the route after configure-module returns" note there, or drop the link so it isn't a dead reference. While here: this paragraph also lands as the last block of Backup and restore, which is unrelated to backups.

@shockalotti
shockalotti merged commit 437e937 into main Oct 15, 2026
7 checks passed
@shockalotti
shockalotti deleted the packaging/ns8-robot-route-wait branch October 15, 2026 05:42
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant