Skip to content

fix(container): [#2855] stop the tracker with timeout --foreground - #2857

Merged
josecelano merged 3 commits into
torrust:developfrom
josecelano-bot:2855-container-regression-timeout-sigint
Nov 13, 2026
Merged

josecelano merged 3 commits into
torrust:developfrom
josecelano-bot:2855-container-regression-timeout-sigint

Conversation

@josecelano-bot

@josecelano-bot josecelano-bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2855. Intended merge order: #2856 (spec-only, folder-style issue specification and evidence for #2855) first, then this PR.

Fixes the Container workflow's "Run Persistence Transition Regression" step, which has been failing on develop and on unrelated PRs since 2026-11-12 (first bad run 43184363486, last good run 43059027926).

Root cause

run_tracker in test-mounted-no-persistence-configuration.sh runs timeout --signal=INT --kill-after=3s 10s docker run ... and only accepts exit status 0 or 124. On the self-hosted CI runner (torrust-runner-01, Ubuntu 26.04.1), timeout is provided by rust-coreutils (the uutils implementation), which was upgraded 0.8.0 -> 0.10.0 on 2026-11-12 06:36 UTC. Without --foreground, that timeout does not forward SIGINT to the docker run client; the tracker never receives the signal, and --kill-after=3s SIGKILLs the client instead, giving exit status 137. The tracker itself shuts down correctly on SIGINT and SIGTERM in every case tested (docker kill --signal=INT/TERM, GNU timeout with or without --foreground, uutils timeout --foreground).

Fix

Add --foreground to the timeout invocation, and report the unexpected exit status on failure so a future regression is diagnosable from the CI log alone.

Verification

Ran the script locally against a release image built from develop at the merge of #2841, with both timeout implementations:

  • uutils timeout 0.10.0 (as on the runner): before the fix, exit 137 and no shutdown log line (reproducing the CI failure); after the fix, exit 0 and both regression markers printed.
  • GNU timeout 9.7: exit 0 and both markers printed, before and after.

linter all passes.

Files touched

  • contrib/dev-tools/containers/tests/test-mounted-no-persistence-configuration.sh: adds --foreground to the timeout invocation and reports the unexpected exit status on failure.
  • project-words.txt: adds uutils.

Full root-cause analysis and evidence: #2855 and the spec in #2856.

🤖 Generated with Claude Code

…ound

uutils' timeout (the default on Ubuntu 26.04/26.04.1 via rust-coreutils) does
not forward SIGINT to the docker run client unless --foreground is set. The
self-hosted CI runner's rust-coreutils was upgraded 0.8.0 -> 0.10.0 on
2026-11-12, which regressed the Container workflow's Run Persistence
Transition Regression step: the client was SIGKILLed by --kill-after instead
of shutting the tracker down gracefully, giving exit status 137 instead of
the accepted 0 or 124.

Also report the unexpected exit status on failure, to make the next failure
diagnosable without re-running under -x.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
da2ce7
da2ce7 previously approved these changes Nov 13, 2026

@da2ce7 da2ce7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 08e4d2bffd9f320275cc39123f00fbdaf7335087. Recomputed from the bytes at this head, against the merge base ec1f58ec9; git merge-tree against develop ec1f58ec9 writes a conflict-free tree identical to the head's tree.

Process: a bug fix, governed by .github/skills/dev/debugging/fix-bug/SKILL.md plus the commit-changes and open-pull-request skills in .github/skills/dev/git-workflow/. The maintained regression test is the existing container-workflow step on the self-hosted runner; the issue-local evidence fix-bug requires is not in the tree (F1).

The change against its claim: run_tracker adds --foreground to timeout and prints any status other than 0 or 124 to stderr; the final test is unchanged, so set -e still stops the script on such a status. The runner facts in the description match docs/issues/closed/2323-1840-hetzner-self-hosted-ci-runner/runner-server-setup.md (Ubuntu 26.04.1, rust-coreutils 0.8.0 installed, 0.10.0 pending), and no input of the failing step changed on develop after 2026-11-04 apart from a publish-job action bump; the uutils behaviour itself cannot be recomputed offline. project-words.txt adds uutils, used at script line 32, in LC_ALL=C sort -u order. The rationale comment has three precision issues (F2 to F4).

Consistency: no Markdown, link, index or path change, so nothing to resolve or register.

Hygiene: one commit, 08e4d2bff fix(container): [#2855] stop the tracker with timeout --foreground; author equals committer and the body matches the diff; one attribution footer line (no rule in the tree on it), and the description carries one attribution footer line too. git diff --check is clean, the skill's PR-branch commit-id check prints nothing, and the closing-keyword scan matches only Closes #2855 in the description. The title lacks the issue reference (F5).

Findings: F1 Minor, bug-fix evidence absent while the issue closes on merge. F2 Minor, the comment names the container where the client is killed. F3 Nit, uutils behaviour without its version. F4 Nit, "no-op" holds here only. F5 Nit, title lacks [#2855]; description omits project-words.txt. None blocks.

Checked, no finding: exit-status handling under set -euo pipefail for 0, 124, 137 and other statuses; GNU timeout signals the command directly with or without --foreground; begin_shutdown adds no fixed delay that would use up the 3-second kill window; no review threads and no audit record exist.

Checks

On the loop's compute node at this head, against the merge base ec1f58ec9 (receipt server-gates-kranich-823): the repository's pre-commit gate and lint roster exit 0 and the tree left clean. No Rust package changes, so no package tests apply; the container script itself was not run here, and the uutils behaviour on the runner cannot be recomputed offline. F2 checked against the bytes at posting: the added comment says "the container never receives the signal and is SIGKILLed by --kill-after", while timeout signals its own child, the docker run client. There are no review threads. GitHub checks at this head at posting time: "Package Coverage (${{ matrix.package }}): completed/skipped; Package Coverage Regression: completed/success; Docker E2E: completed/skipped; Unit (nightly): in_progress/pending; Layer Boundary Bans: in_progress/pending; Shipped Features: completed/success; Documented Clippy Allows: queued/pending; Unit (stable): queued/pending; Workspace Packaging: completed/success; Test (Docker) (release): queued/pending; Docs Lint: completed/success; Discover Package Coverage: completed/success; Generate Coverage Report: in_progress/pending; Build on windows-latest (nightly): completed/success; Build on windows-latest (stable): completed/success; Build on ubuntu-latest (nightly): completed/success; Build on macos-latest (nightly): completed/success; Build on ubuntu-latest (stable): completed/success; Build on macos-latest (stable): completed/success". At posting, develop is ec1f58ec9.

Comment thread project-words.txt Resolved
@da2ce7

da2ce7 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

ACK 08e4d2b — the regression script's timeout call gains --foreground with an unexpected-status message, the added dictionary word is used and sorted, and the branch merges cleanly onto ec1f58ec9

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues
What changed in this PR

Fixes the container CI regression where timeout (uutils via rust-coreutils) fails to forward SIGINT to docker run, causing the persistence regression step to exit with 137 instead of shutting down cleanly.

Changes:

  • Add --foreground to the timeout invocation so SIGINT is forwarded correctly to docker run on uutils timeout.
  • Emit a clearer diagnostic when run_tracker exits with an unexpected status.
  • Add uutils to the repository word list.
File Description
project-words.txt Adds uutils to keep spellcheck/linting green for the new documentation text.
contrib/​dev-tools/​containers/​tests/​test-mounted-no-persistence-configuration.sh Uses timeout --foreground to ensure SIGINT propagation and improves logging for unexpected exit codes.

Review effort: Lite

@josecelano

Copy link
Copy Markdown
Member

ACK 08e4d2b

josecelano-bot added a commit to josecelano-bot/torrust-tracker that referenced this pull request Nov 13, 2026
…F8-F14

Re-ran V5 on the self-hosted runner with a full container listing, an
exited-count check, and image/mount identification (31 containers, all
this script's, oldest predating the last green run); softened Impact's
leftover-container claim to match; named the fixing commit (PR torrust#2857)
in V3 and marked T3/T4/AC1-AC3 as validated on that branch; replaced
the V1 diff with a complete, reapplyable patch covering both real
edits; fixed a progress-log timestamp that postdated its own commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@josecelano-bot josecelano-bot changed the title fix(container): stop the tracker with timeout --foreground fix(container): [#2855] stop the tracker with timeout --foreground Nov 13, 2026
…s F2-F4,F7

Rewords the comment above the timeout invocation: names the client,
not the container, as what --kill-after SIGKILLs (F2); names the
uutils version the regression was observed on, 0.10.0 (F3); and
states that --foreground has the same effect under GNU timeout only
for this single-command invocation, not in general (F4). Consolidates
the duplicated 0-or-124 exit-code check into one case statement that
both prints the diagnostic and fails, replacing the obsolescent
test ... -o ... form (F7).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Creates docs/pr-reviews/pr-2857-review/PR-REVIEW.md (started late):
records da2ce7's five findings and Copilot's two, their dispositions,
and the reply/resolution each thread got.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@da2ce7 da2ce7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 38d7631fbd8c82103c818bcc6977466239612c32. Recomputed from the bytes at this head, against the merge base ec1f58ec9; git merge-tree against develop ec1f58ec9 writes a conflict-free tree identical to the head's tree.

Process: since 08e4d2bff, a186f9e8e answers the earlier review and the two automated comments, and 38d7631fb adds the audit record .github/skills/dev/pr-reviews/process-pr-review/SKILL.md requires, logging its late start as the skill asks. Its frontmatter matches the template; it drops the template's last section (F7).

The change against its claim: a186f9e8e rewords the comment above timeout and folds the if and the final test into one case. With timeout stubbed to exit 0, 124, 137, 1 and 125 under set -euo pipefail, this head and 08e4d2bff behave identically. F1 is resolved (the description states that the spec PR merges first), F2 to F4 in the comment, F5 in the title and the description.

Audit record: the validator's logic, run offline against the capture with the branch's commits standing in for the forge's commit list, reports 7 rows, 1 log entry and 0 failures. Every thread has one reply on its own thread, a row and a resolution. Two rows pair a no-change disposition with the wrong thread state (F6).

Consistency: no link, index or path change outside the new record, whose source and reply URLs resolve to this pull request's threads.

Hygiene: three commits, author equals committer on each; each carries one attribution footer line (no rule in the tree on it), and the description carries one attribution footer line too. git diff --check is clean, the skill's PR-branch commit-id check prints nothing, and the closing-keyword scan matches only Closes #2855 in the description.

Findings: F6 Minor, two audit rows pair NO_ACTION with RESOLVED. F7 Nit, the record omits the template's ## Completion Rules. Both are record edits; none blocks.

Checked, no finding: the exit behaviour of the new case; the declined feature-gate suggestion (the script needs bash and already used GNU-style long options); BusyBox and SIGKILLs against existing tree usage; the record's frontmatter and Processing Log.

Checks

On the loop's compute node at this head, against the merge base ec1f58ec9 (receipt server-gates-kranich-846): the repository's pre-commit gate and lint roster exit 0 and the tree left clean; the repository's audit validator on pr-2857-review exit 0 (7 rows, 1 log entry, no failures). No Rust package changes, so no package tests apply; the container script itself was not run here. F6 and F7 checked against the bytes at posting: two rows of PR-REVIEW.md pair NO_ACTION with RESOLVED, and the record has no ## Completion Rules heading. Read on the forge: all 7 review threads are answered and resolved. GitHub checks at this head at posting time: "". At posting, develop is ec1f58ec9.


| Finding ID | Review finding reference | Author class | Severity | Category | Relationship | Disposition | Thread state |
| ---------- | ------------------------ | ------------ | -------- | -------- | ------------ | ----------- | ------------ |
| F1 | review-finding:pr-2857-f1 | Human | Minor | maintainability | ORIGINAL | NO_ACTION | RESOLVED |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor][F6] The rows for F1 and for the BusyBox comment record NO_ACTION with Thread state RESOLVED, a pairing the skill does not allow.

.github/skills/dev/pr-reviews/process-pr-review/SKILL.md (lines 95-101) and this record's Status Values (lines 35-38) use FIXED/RESOLVED when a change fixed the concern, and NO_ACTION/SUPERSEDED, with the reply Superseded by <FindingId>: <reason>., only for a no-change thread. F1 was repaired by one of the two fixes it offered, the merge order stated in the description, so it is FIXED/RESOLVED like F5, also a description edit; its Solution argues against splitting the PR, which the finding never asked. The row at line 58 is the no-change case, NO_ACTION/SUPERSEDED. Non-blocking: a record edit. This F6 and F7 continue the earlier review's numbering; the record's F6 and F7 are the automated comments, so these take the next ordinals.


## Processing Log

- 2026-11-13 15:44 UTC - Started this record late, after `da2ce7`'s review `5889979674` (5 findings, F1 to F5) and `copilot-pull-request-reviewer`'s review `5890022304` (2 findings, F6 and F7) were already fixed/declined and replied to. F2, F3, F4 and F7 fixed in commit `fix(container): [#2855] address PR #2857 review findings F2-F4,F7`; F5 fixed via `gh-bot pr edit` (title and description); F1 and F6 declined with reasoning. Replied on all seven threads, then resolved them.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit][F7] The record ends at the Processing Log and omits the template's ## Completion Rules section.

The skill (lines 46-47) creates the record from docs/templates/PR-REVIEW-TEMPLATE.md, whose last section (template line 120) is ## Completion Rules; the records for #2716, #2811 and #2841 keep it. Copying it in restores the rule that F6 depends on.

@da2ce7

da2ce7 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

ACK 38d7631 — the timeout --foreground fix and its single exit-status case behave correctly for every status class, F1 to F5 are resolved, the audit record passes the validator's logic offline with a reply, row and resolution for all seven threads, and the branch merges cleanly onto ec1f58ec9

@josecelano

Copy link
Copy Markdown
Member

ACK 38d7631

@josecelano
josecelano merged commit 24230dd into torrust:develop Nov 13, 2026
22 checks passed
josecelano-bot added a commit to josecelano-bot/torrust-tracker that referenced this pull request Nov 13, 2026
…F19-F27

Round 3: da2ce7's APPROVED review 5893913692 (F19-F24, reviewer labels
F15-F20) and copilot-pull-request-reviewer's review 5894073993 (F25-F27,
reviewer labels F19-F21).

- Added the skill-link marker (F19).
- Cited the first commit by subject instead of id, and corrected the
  Ownership note's reply-count attribution (F20).
- Recaptured V5 on torrust-runner-01 with a fresh, internally-consistent
  listing: 34 containers, 0 exited, elision counts that add up (F21, F26).
- Corrected the V1 patch's second hunk header and the `cp` location prose
  (F22).
- Recorded the real V5 capture time throughout, and the PR torrust#2857 merge
  (F23, F27).
- Ran `linter all`, checked the criterion, and added the audit record to
  the PR description's Files touched list (F24).
- Reclassified F18 as RE_RAISE_OF:F13 (F25).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
josecelano-bot added a commit to josecelano-bot/torrust-tracker that referenced this pull request Nov 13, 2026
Round 5: copilot-pull-request-reviewer's review 5899193311 (F32-F33,
reviewer label F28 for both), submitted against the round-3 head before
round 4's fix was pushed.

- Moved the issue folder from docs/issues/open/ to docs/issues/closed/
  now that PR torrust#2857 merged and closed torrust#2855; set status: done, and
  updated spec-path and the evidence file's issue-spec reference (F32).
- F33 duplicates round 4's F28 (pin V1 to the pre-fix script), already
  fixed before this review landed; no further action needed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
coverage — 38d7631f Deployed Nov 13, 2026 by josecelano-bot via Generate Coverage Report #3154
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Container regression script fails when timeout is uutils (SIGINT not forwarded)

4 participants