diff --git a/docs/node-image-prefetch.md b/docs/node-image-prefetch.md index 08dda35..cc4cbc7 100644 --- a/docs/node-image-prefetch.md +++ b/docs/node-image-prefetch.md @@ -75,7 +75,9 @@ readiness 조회로 무조건 보내지 않습니다. 실제 요청은 release· `artifact_sha256`은 원본 JSON 파일 hash이고 `image_list.sha256`은 줄 단위 목록의 checksum입니다. #70의 `image_set_id`는 architecture와 정렬·중복 제거한 전체 이미지 참조 집합에서 Runtime이 계산하는 별도 식별자입니다. 서로 대체하지 않습니다. -이번 PR은 제안 단계의 API 요청 생성·hash 계산·HTTP 호출을 추가하지 않습니다. +목록 생성·취합기는 준비 API 요청이나 HTTP 호출을 추가하지 않습니다. 별도 +[readiness 대조 도구](node-readiness-audit.md)는 기존 bundle과 저장된 조회 응답을 +검수하기 위해 image_set_id만 재계산하며, 준비 요청·조회·노드 작업은 실행하지 않습니다. ### 현재 준비 상태는 파일 목록이 아님 diff --git a/docs/node-readiness-audit.md b/docs/node-readiness-audit.md new file mode 100644 index 0000000..2d2caf0 --- /dev/null +++ b/docs/node-readiness-audit.md @@ -0,0 +1,91 @@ +# 노드 이미지 준비 응답 대조 + +`scripts/audit_node_image_readiness.py`는 기존 publish bundle과 **저장된 Runtime +readiness 원본 JSON**을 오프라인에서 대조합니다. 준비 API·노드·GHCR·Backend를 +호출하거나 이미지를 다운로드하지 않습니다. 정보 조회·준비 worker·Scheduler·릴리스 +활성화 구현을 대체하지 않으며, 기존 생성 fallback이나 준비 선호 정책을 변경하지 않습니다. + +## 입력과 범위 + +- 공식 성공 발행에서 가져온 `artifact-v2.json`: schema2.0/2.1의 일반 Linux K3s 문제. +- Runtime `POST /v1/image-readiness/query`가 실제 반환한 JSON. operation의 SUCCEEDED + 이력이나 카톡 요약문으로 대신하지 않습니다. +- 운영 담당자가 확인한 **예상 target ID 전체 목록**. 노드명으로 추측하지 않습니다. +- 판정 기준 시각. 생략하면 실행 PC의 현재 UTC이며, 과거 증거 재현에는 `--as-of`를 씁니다. + +기존 `generate_node_image_plan.py`를 재사용해 main 출처 형식·revision·검사 증거· +전체 컨테이너의 고정 GHCR 참조를 검사합니다. KOTH·VM·DOCKER는 제외합니다. +이는 **입력 자료의 계약 검사**입니다. GitHub run/artifact의 진위·출처나 Backend의 +현재 active release를 독립 조회한 것이 아니므로 기존 출처 검증을 먼저 거쳐야 합니다. +schema2.1의 env·비밀값 참조도 보고서에 복사하지 않습니다. + +## 검사 내용 + +1. 공개 컨테이너뿐 아니라 DB 등 보조 컨테이너를 포함한 전체 이미지 집합을 사용합니다. +2. architecture와 전체 참조 집합으로 Runtime canonical `image_set_id`를 재계산해 + 응답과 비교합니다. JSON 파일 hash·줄 단위 이미지 목록 checksum과는 다릅니다. +3. 예상 target 누락은 UNKNOWN입니다. 중복·예상 밖 target이나 모순된 개수/타입은 + 잘못된 응답으로 거절합니다. +4. `observed_at <= 판정 시각 < valid_until`을 확인하고 RFC3339Nano를 나노초까지 + 비교합니다. 미래 관측·만료·시각 누락/오류를 준비 완료로 표시하지 않습니다. +5. 최대 관측 나이는 기본60초이며 현재 Scheduler 기본값과 같습니다. 운영 설정이 + 다르면 승인된 값으로 `--max-observation-age-seconds`를 지정합니다. 이 옵션은 + 실제 Runtime·Scheduler 설정이나 증거의 시각을 바꾸지 않습니다. +6. 만료·미관측 target의 개수를 현재 준비된 노드 합계에 포함하지 않습니다. + +`ready=true`는 적합한 같은 노드 **한 대 이상**에 전체 이미지가 있다는 힌트입니다. +`ready_node_count=1, eligible_node_count=2`는 준비 선호에 사용할 수 있어도 +전체 노드 준비 완료가 아닙니다. 이 경우 `PARTIAL`로 기록합니다. + +명세·identity 기준: +[Runtime B안 계약](https://github.com/MSG-CTF/secure-provisioner/blob/619423a35876d3e823df17b812cb0b1539001c13/docs/api/image-preparation-b-contract.md), +[canonical 구현](https://github.com/MSG-CTF/secure-provisioner/blob/619423a35876d3e823df17b812cb0b1539001c13/internal/imageprep/model.go#L265-L275), +[Scheduler 조회 시각 기준](https://github.com/MSG-CTF/instance-scheduler/blob/692be04aabab0addfbce251c81c1da16b8c2c1e8/docs/image-readiness.md#L91-L111). +공식 query/response 예제는 `tests/fixtures/runtime-readiness/`에 원문으로 보관합니다. + +## 실행 + +```bash +python3 -B -m scripts.audit_node_image_readiness \ + --artifact /absolute/path/to/artifact-v2.json \ + --readiness-response /absolute/path/to/runtime-readiness.json \ + --target-id target-a \ + --target-id target-b \ + --output-dir /absolute/path/to/new-private-audit +``` + +과거 증거를 재현할 때만 `--as-of 2026-10-24T00:00:30Z`처럼 실제 기준 시각을 지정합니다. +보고서는 생성 시각과 판정 기준 시각을 따로 기록합니다. 과거 기준의 성공 결과를 현재 +준비 상태로 재사용하지 않습니다. 입력 시각을 수정하거나 만료를 연장하지 않습니다. + +새 출력 디렉터리에 `readiness-audit.json` 하나만 만듭니다. 디렉터리는0700, +보고서는0600이며 기존 출력은 덮어쓰지 않습니다. target ID 등 운영 정보가 있으므로 +공개 저장소·공개 Actions artifact에 원본 보고서를 업로드하지 않습니다. +표준 출력에는 상태·집계만 쓰고 입력 원문·env·token·오류 응답 본문은 출력하지 않습니다. +파일 크기는 각1MiB, 이미지32개·target50개까지입니다. + +## 판정과 종료 코드 + +| 상태 | 의미 | 종료 코드 | +|---|---|---:| +| ALL_ELIGIBLE_READY | 모든 요청 target에서 ready_node_count=eligible_node_count>0이며 관측이 fresh | 0 | +| PARTIAL | 일부 target/적합 노드만 fresh READY | 1 | +| NOT_READY | 모든 요청 target이 fresh 관측됐으나 준비된 노드 없음 | 1 | +| UNKNOWN | 누락·미관측·만료·시각 오류 등으로 전체 준비를 판단할 수 없음 | 1 | +| 잘못된 입력/응답·I/O 오류 | 이미지 집합·architecture·target·개수·파일 등이 계약과 다름 | 2 | + +종료1은 **대조 증거가 전체 완료를 보장하지 못한다는 뜻**입니다. 생성 API를 차단하거나 +준비 요청·재시도를 실행하지 않습니다. 기존 발행 workflow의 필수 gate로 연결하지 않았으며 +검사 실패로 정상 bundle을 Backend Poller가 수집하지 못하게 만들지도 않습니다. + +## 증명하지 않는 것 + +- API의 실제 호출·응답 인증, Backend 현재 active, 노드 UID/boot generation의 독립 검증 +- Backend 준비 operation 성공, 실제 Pod 생성·HTTP 접속·imageID, cold pull 소요 시간 +- 새 노드 자동 준비, 운영 버전 전환·기존 인스턴스 hotfix 교체·롤백 성공 +- 요청 목록 밖의 모든 운영 노드 준비와 물리 노드 UID별 중복 없는 총수 + +보고서의 개수는 fresh target 응답에 있는 수의 합입니다. 유일한 물리 노드 수로 +취급하지 않습니다. `image_set_id`도 서명이나 실행 승인 증명이 아닙니다. +`readinessCache=miss`는 Scheduler 조회 응답 캐시의 상태이므로 이 도구의 이미지 +준비 판정이나 cold pull 증거로 사용하지 않습니다. diff --git a/scripts/README.md b/scripts/README.md index 5fbeb63..0cb563f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -9,6 +9,14 @@ Provisioner API 요청으로 변환합니다. 생성 Operation을 조회한 뒤 ## 파일 +### `audit_node_image_readiness.py` + +기존 발행 bundle과 저장된 Runtime readiness JSON을 오프라인에서 대조합니다. +전체 이미지·architecture·예상 target·관측 시각/만료를 검사하고, ready1/2 같은 +부분 준비와 전체 적합 노드 준비를 구분합니다. 별도 HTTP 호출·다운로드·활성화는 +하지 않으며 운영 정보를 담은 보고서는0700/0600의 새 비공개 경로에만 기록합니다. +[판정 범위와 실행 방법](../docs/node-readiness-audit.md)을 참고합니다. + ### `collect_node_image_plans.py` 현재 main과 성공한 실제 bundle을 대조해 일반 Linux K3s 문제의 전체 다운로드 diff --git a/scripts/audit_node_image_readiness.py b/scripts/audit_node_image_readiness.py new file mode 100644 index 0000000..ea0c02c --- /dev/null +++ b/scripts/audit_node_image_readiness.py @@ -0,0 +1,273 @@ +#!/usr/bin/env python3 +"""Compare a publish bundle with a saved Runtime readiness response, offline.""" + +import argparse +from datetime import datetime, timezone +import hashlib +import json +import os +from pathlib import Path +import re +import sys + +if __package__: + from .generate_node_image_plan import generate_plan, MAX_ARTIFACT_BYTES +else: + from generate_node_image_plan import generate_plan, MAX_ARTIFACT_BYTES + + +MAX_TARGETS = 50 +MAX_IMAGES = 32 +MAX_COUNT = 2**31 - 1 +EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc) +RFC3339 = re.compile( + r"^([0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2})" + r"(?:\.([0-9]{1,9}))?(Z|[+-][0-9]{2}:[0-9]{2})$" +) + + +def _time_ns(value): + if not isinstance(value, str): + raise ValueError("timestamp must be RFC3339") + matched = RFC3339.fullmatch(value) + if matched is None or matched[3] == "-00:00": + raise ValueError("timestamp must have a known offset") + zone = matched[3] + if zone != "Z" and (int(zone[1:3]) > 23 or int(zone[4:6]) > 59): + raise ValueError("invalid timestamp offset") + base = datetime.fromisoformat(matched[1] + ("+00:00" if zone == "Z" else zone)) + delta = base.astimezone(timezone.utc) - EPOCH + # Keep Runtime RFC3339Nano boundaries instead of truncating to microseconds. + return ((delta.days * 86400 + delta.seconds) * 10**9 + + int((matched[2] or "").ljust(9, "0"))) + + +def calculate_image_set_id(architecture, images): + canonical = { + "schema_version": 1, + "architecture": architecture, + "images": sorted(set(images)), + } + raw = json.dumps(canonical, separators=(",", ":"), ensure_ascii=True).encode() + return "sha256:" + hashlib.sha256(raw).hexdigest() + + +def _validate_targets(target_ids): + if not isinstance(target_ids, list) or not 1 <= len(target_ids) <= MAX_TARGETS: + raise ValueError("expected targets are required") + seen = set() + for target in target_ids: + if (not isinstance(target, str) or not 1 <= len(target) <= 128 + or target.strip() != target + or any(ord(character) < 32 or ord(character) == 127 for character in target) + or target in seen): + raise ValueError("invalid or duplicate expected target") + seen.add(target) + return seen + + +def _observation(entry, now_ns, max_age_seconds): + result = { + "target_id": entry["target_id"], + "state": "UNKNOWN", + "reason": "NO_OBSERVATION", + "ready_node_count_as_reported": entry["ready_node_count"], + "eligible_node_count_as_reported": entry["eligible_node_count"], + "observed_at": None, + "valid_until": None, + } + observed, until = entry.get("observed_at"), entry.get("valid_until") + if observed is None or until is None: + return result + try: + observed_ns, until_ns = _time_ns(observed), _time_ns(until) + except (ValueError, OverflowError): + result["reason"] = "INVALID_TIMESTAMPS" + return result + result.update(observed_at=observed, valid_until=until) + if until_ns <= observed_ns: + result["reason"] = "INVALID_TIME_WINDOW" + elif observed_ns > now_ns: + result["reason"] = "FUTURE_OBSERVATION" + elif now_ns >= until_ns: + result["reason"] = "EXPIRED" + elif now_ns - observed_ns >= max_age_seconds * 10**9: + result["reason"] = "OBSERVATION_TOO_OLD" + elif not entry["ready"]: + result.update(state="NOT_READY", reason="NO_READY_NODE_AS_REPORTED") + elif entry["ready_node_count"] < entry["eligible_node_count"]: + result.update(state="SOME_NODES_READY", reason="PARTIAL_NODE_COVERAGE") + else: + result.update(state="ALL_NODES_READY", reason="FRESH_AS_REPORTED") + return result + + +def audit_readiness(artifact, artifact_sha256, response, target_ids, *, as_of, + max_age_seconds=60): + expected_targets = _validate_targets(target_ids) + if type(max_age_seconds) is not int or not 1 <= max_age_seconds <= 86400: + raise ValueError("invalid maximum observation age") + now_ns = _time_ns(as_of) + generated = generate_plan(artifact, artifact_sha256) + if generated is None: + raise ValueError("only general Linux Kubernetes bundles are supported") + plan, _ = generated + images = [container["image"] for container in plan["containers"]] + if len(images) > MAX_IMAGES: + raise ValueError("the Runtime image limit was exceeded") + identity = calculate_image_set_id(plan["architecture"], images) + if (not isinstance(response, dict) + or response.get("architecture") != plan["architecture"] + or response.get("image_set_id") != identity): + raise ValueError("response image set or architecture does not match") + targets = response.get("targets") + if not isinstance(targets, list) or len(targets) > MAX_TARGETS: + raise ValueError("a readiness target array is required") + by_id = {} + for entry in targets: + if not isinstance(entry, dict): + raise ValueError("invalid readiness target") + target = entry.get("target_id") + if (not isinstance(target, str) or target not in expected_targets + or target in by_id): + raise ValueError("unexpected or duplicate response target") + ready, count, eligible = ( + entry.get("ready"), entry.get("ready_node_count"), + entry.get("eligible_node_count"), + ) + if (type(ready) is not bool or type(count) is not int or type(eligible) is not int + or not 0 <= count <= eligible <= MAX_COUNT or ready != (count > 0)): + raise ValueError("invalid readiness counts") + by_id[target] = entry + + audited = [] + for target in target_ids: + if target not in by_id: + audited.append({ + "target_id": target, "state": "UNKNOWN", "reason": "MISSING_TARGET", + "ready_node_count_as_reported": None, + "eligible_node_count_as_reported": None, + "observed_at": None, "valid_until": None, + }) + else: + audited.append(_observation(by_id[target], now_ns, max_age_seconds)) + fresh = [entry for entry in audited if entry["state"] != "UNKNOWN"] + positive = [entry for entry in fresh if entry["state"] != "NOT_READY"] + all_nodes = all(entry["state"] == "ALL_NODES_READY" for entry in audited) + if all_nodes: + status = "ALL_ELIGIBLE_READY" + elif positive: + status = "PARTIAL" + elif len(fresh) != len(audited): + status = "UNKNOWN" + else: + status = "NOT_READY" + return { + "schema_version": "1.0", + "scope": "SUPPLIED_BUNDLE_REQUESTED_TARGETS_ONLY", + "status": status, + "as_of": as_of, + "max_observation_age_seconds": max_age_seconds, + "bundle": { + "challenge_slug": plan["challenge_slug"], + "registry_revision": plan["registry_revision"], + "architecture": plan["architecture"], + "artifact_sha256": plan["artifact_sha256"], + "container_count": len(images), + "image_set_id": identity, + }, + "all_requested_targets_have_ready_hint": len(positive) == len(audited), + "all_reported_eligible_nodes_ready": all_nodes, + "counts": { + "requested_targets": len(audited), "fresh_targets": len(fresh), + "unknown_targets": len(audited) - len(fresh), + "ready_nodes_in_fresh_reports": sum( + entry["ready_node_count_as_reported"] for entry in fresh), + "eligible_nodes_in_fresh_reports": sum( + entry["eligible_node_count_as_reported"] for entry in fresh), + }, + "targets": audited, + "limits": { + "live_api_queried": False, + "backend_active_verified": False, + "node_identity_independently_verified": False, + "operation_success_proven": False, + "pod_execution_proven": False, + "cold_pull_proven": False, + "rollout_proven": False, + "all_operating_nodes_verified": False, + }, + } + + +def _unique_object(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError("duplicate JSON key") + result[key] = value + return result + + +def _invalid_constant(_value): + raise ValueError("nonfinite JSON number") + + +def _read_json(path): + path = Path(path) + if path.is_symlink() or not path.is_file() or path.stat().st_size > MAX_ARTIFACT_BYTES: + raise ValueError("invalid input file") + with path.open("rb") as stream: + raw = stream.read(MAX_ARTIFACT_BYTES + 1) + if len(raw) > MAX_ARTIFACT_BYTES: + raise ValueError("input size limit exceeded") + return raw, json.loads(raw, object_pairs_hook=_unique_object, + parse_constant=_invalid_constant) + + +def write_audit(artifact_path, response_path, target_ids, output_dir, *, as_of=None, + max_age_seconds=60): + raw, artifact = _read_json(artifact_path) + _, response = _read_json(response_path) + generated_at = datetime.now(timezone.utc).isoformat() + report = audit_readiness( + artifact, hashlib.sha256(raw).hexdigest(), response, target_ids, + as_of=generated_at if as_of is None else as_of, + max_age_seconds=max_age_seconds, + ) + report["generated_at"] = generated_at + report["time_basis"] = "EXPLICIT_AS_OF" if as_of is not None else "SYSTEM_UTC_NOW" + output_dir = Path(output_dir) + output_dir.mkdir(mode=0o700, parents=True, exist_ok=False) + descriptor = os.open(output_dir / "readiness-audit.json", + os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + with os.fdopen(descriptor, "w", encoding="utf-8") as stream: + json.dump(report, stream, ensure_ascii=True, indent=2, allow_nan=False) + stream.write("\n") + return report + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--artifact", required=True, type=Path) + parser.add_argument("--readiness-response", required=True, type=Path) + parser.add_argument("--target-id", required=True, action="append") + parser.add_argument("--output-dir", required=True, type=Path) + parser.add_argument("--as-of", help="RFC3339 audit time; omitted for current UTC") + parser.add_argument("--max-observation-age-seconds", type=int, default=60) + args = parser.parse_args() + try: + report = write_audit( + args.artifact, args.readiness_response, args.target_id, args.output_dir, + as_of=args.as_of, max_age_seconds=args.max_observation_age_seconds, + ) + except (ValueError, OSError, TypeError, RecursionError, OverflowError): + print("Readiness audit failed: check inputs, time and output path.", file=sys.stderr) + return 2 + print(json.dumps({"status": report["status"], "counts": report["counts"], + "live_api_queried": False})) + return 0 if report["all_reported_eligible_nodes_ready"] else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/README.md b/tests/README.md index f57397c..732825b 100644 --- a/tests/README.md +++ b/tests/README.md @@ -5,6 +5,13 @@ ## 테스트 구성 +### `test_audit_node_image_readiness.py` + +Runtime 공식 예제의 canonical image_set_id, 보조 이미지 포함·대상/architecture +불일치, 부분 준비·누락·미관측·만료·나노초 경계와 반복 조회 시각 보존을 검사합니다. +출력 비공개 권한·덮어쓰기 금지·비밀값 비포함·CLI 종료 코드도 확인합니다. +실제 API·노드 다운로드·Pod·전환을 실행하는 테스트가 아닙니다. + ### `test_collect_node_image_plans.py` 전체 대상과 아키텍처별 취합, 신규·변경·유지 비교, 누락·중복·출처·이전 목록 diff --git a/tests/fixtures/info-valid/prob/for_organizer/web/Dockerfile b/tests/fixtures/info-valid/prob/for_organizer/web/Dockerfile index 6f1bfee..c03e51b 100644 --- a/tests/fixtures/info-valid/prob/for_organizer/web/Dockerfile +++ b/tests/fixtures/info-valid/prob/for_organizer/web/Dockerfile @@ -1 +1,5 @@ FROM nginxinc/nginx-unprivileged:1.29-alpine + +USER root +RUN apk upgrade --no-cache brotli-libs +USER 101 diff --git a/tests/fixtures/runtime-readiness/README.md b/tests/fixtures/runtime-readiness/README.md new file mode 100644 index 0000000..1f30cf3 --- /dev/null +++ b/tests/fixtures/runtime-readiness/README.md @@ -0,0 +1,10 @@ +# Runtime readiness 계약 예제 + +다음 공식 예제를 원문 그대로 보관합니다. + +- [query.json](https://github.com/MSG-CTF/secure-provisioner/blob/619423a35876d3e823df17b812cb0b1539001c13/docs/api/examples/image-preparation/query.json) +- [response.json](https://github.com/MSG-CTF/secure-provisioner/blob/619423a35876d3e823df17b812cb0b1539001c13/docs/api/examples/image-preparation/response.json) + +이는 실제 운영 결과가 아닌 계약용 가상 target·이미지입니다. 타임스탬프를 현재로 +바꾸지 않고 2026-10-24T00:00:30Z 기준으로 재현합니다. 공식 canonical image_set_id, +target-a의1/2 부분 준비, target-b의 미준비와 unknown-target의 응답 누락을 검사합니다. diff --git a/tests/fixtures/runtime-readiness/query.json b/tests/fixtures/runtime-readiness/query.json new file mode 100644 index 0000000..81bbcb2 --- /dev/null +++ b/tests/fixtures/runtime-readiness/query.json @@ -0,0 +1,12 @@ +{ + "architecture": "AMD64", + "images": [ + "ghcr.io/msg-ctf/challenges/example/web@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "ghcr.io/msg-ctf/challenges/example/db@sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + ], + "target_ids": [ + "target-a", + "target-b", + "unknown-target" + ] +} diff --git a/tests/fixtures/runtime-readiness/response.json b/tests/fixtures/runtime-readiness/response.json new file mode 100644 index 0000000..0b3db1c --- /dev/null +++ b/tests/fixtures/runtime-readiness/response.json @@ -0,0 +1,22 @@ +{ + "image_set_id": "sha256:a96c344635195e2a40715531e3d420180ab28ce838ab88130364ad51bbb7c4c1", + "architecture": "AMD64", + "targets": [ + { + "target_id": "target-a", + "ready": true, + "ready_node_count": 1, + "eligible_node_count": 2, + "observed_at": "2026-10-24T00:00:00Z", + "valid_until": "2026-10-24T00:01:00Z" + }, + { + "target_id": "target-b", + "ready": false, + "ready_node_count": 0, + "eligible_node_count": 2, + "observed_at": "2026-10-24T00:00:00Z", + "valid_until": "2026-10-24T00:01:00Z" + } + ] +} diff --git a/tests/test_audit_node_image_readiness.py b/tests/test_audit_node_image_readiness.py new file mode 100644 index 0000000..1d867bf --- /dev/null +++ b/tests/test_audit_node_image_readiness.py @@ -0,0 +1,433 @@ +import copy +import hashlib +import json +from pathlib import Path +import stat +import subprocess +import sys +import tempfile +import unittest + +from scripts.audit_node_image_readiness import ( + audit_readiness, calculate_image_set_id, write_audit, +) +from tests.test_generate_node_image_plan import sample_artifact + + +ROOT = Path(__file__).resolve().parents[1] +AS_OF = "2026-10-30T00:00:30Z" +ARTIFACT_SHA = "c" * 64 + + +def sample_response(artifact=None): + artifact = artifact or sample_artifact() + images = [item["image"] for item in artifact["workload"]["containers"]] + return { + "image_set_id": calculate_image_set_id(artifact["architecture"], images), + "architecture": artifact["architecture"], + "targets": [{ + "target_id": "target-a", "ready": True, + "ready_node_count": 1, "eligible_node_count": 1, + "observed_at": "2026-10-30T00:00:00Z", + "valid_until": "2026-10-30T00:02:00Z", + }], + } + + +def audit(response=None, artifact=None, targets=None, **kwargs): + artifact = artifact or sample_artifact() + return audit_readiness( + artifact, ARTIFACT_SHA, response if response is not None else sample_response(artifact), + ["target-a"] if targets is None else targets, + as_of=kwargs.pop("as_of", AS_OF), **kwargs, + ) + + +class NodeReadinessAuditTests(unittest.TestCase): + def test_canonical_identity_matches_runtime_published_example(self): + images = [ + "ghcr.io/msg-ctf/challenges/example/web@sha256:" + "a" * 64, + "ghcr.io/msg-ctf/challenges/example/db@sha256:" + "b" * 64, + ] + expected = "sha256:a96c344635195e2a40715531e3d420180ab28ce838ab88130364ad51bbb7c4c1" + self.assertEqual(calculate_image_set_id("AMD64", images), expected) + self.assertEqual(calculate_image_set_id("AMD64", images[::-1] + images), expected) + self.assertNotEqual(calculate_image_set_id("ARM64", images), expected) + + def test_runtime_example_has_partial_coverage_and_a_missing_target(self): + fixtures = ROOT / "tests/fixtures/runtime-readiness" + query = json.loads((fixtures / "query.json").read_text()) + response = json.loads((fixtures / "response.json").read_text()) + artifact = sample_artifact() + artifact["challenge_slug"] = "example" + for container in artifact["workload"]["containers"]: + container["image"] = container["image"].replace("/web-notebook/", "/example/") + for entry in artifact["evidence"]["containers"]: + entry["published_image"] = entry["published_image"].replace("/web-notebook/", "/example/") + self.assertEqual( + calculate_image_set_id(query["architecture"], query["images"]), + response["image_set_id"], + ) + report = audit_readiness( + artifact, ARTIFACT_SHA, response, query["target_ids"], + as_of="2026-10-24T00:00:30Z", + ) + self.assertEqual(report["status"], "PARTIAL") + self.assertEqual([entry["state"] for entry in report["targets"]], + ["SOME_NODES_READY", "NOT_READY", "UNKNOWN"]) + self.assertFalse(report["all_reported_eligible_nodes_ready"]) + + expired = audit_readiness( + artifact, ARTIFACT_SHA, response, query["target_ids"], + as_of="2026-10-30T00:00:00Z", + ) + self.assertEqual(expired["status"], "UNKNOWN") + self.assertEqual(expired["counts"]["fresh_targets"], 0) + + def test_complete_fresh_image_set_and_no_input_mutation(self): + artifact, response = sample_artifact(), sample_response() + originals = copy.deepcopy((artifact, response)) + report = audit(response, artifact) + self.assertEqual((artifact, response), originals) + self.assertEqual(report["status"], "ALL_ELIGIBLE_READY") + self.assertTrue(report["all_reported_eligible_nodes_ready"]) + self.assertEqual(report["bundle"]["container_count"], 2) + self.assertEqual(report["bundle"]["artifact_sha256"], ARTIFACT_SHA) + self.assertTrue(report["all_requested_targets_have_ready_hint"]) + self.assertTrue(all(value is False for value in report["limits"].values())) + + def test_true_ready_with_partial_node_coverage_is_not_complete(self): + response = sample_response() + response["targets"][0]["eligible_node_count"] = 2 + report = audit(response) + self.assertEqual(report["status"], "PARTIAL") + self.assertTrue(report["all_requested_targets_have_ready_hint"]) + self.assertFalse(report["all_reported_eligible_nodes_ready"]) + self.assertEqual(report["counts"]["ready_nodes_in_fresh_reports"], 1) + self.assertEqual(report["counts"]["eligible_nodes_in_fresh_reports"], 2) + + def test_missing_target_is_not_assumed_ready(self): + report = audit(targets=["target-a", "target-b"]) + self.assertEqual(report["status"], "PARTIAL") + self.assertEqual(report["targets"][1]["reason"], "MISSING_TARGET") + self.assertFalse(report["all_requested_targets_have_ready_hint"]) + + def test_all_missing_targets_are_unknown_not_zero_real_nodes(self): + response = sample_response() + response["targets"] = [] + report = audit(response) + self.assertEqual(report["status"], "UNKNOWN") + self.assertEqual(report["counts"]["unknown_targets"], 1) + self.assertIsNone(report["targets"][0]["eligible_node_count_as_reported"]) + + def test_multiple_targets_are_checked_in_requested_order(self): + response = sample_response() + other = dict(response["targets"][0], target_id="target-b") + response["targets"].insert(0, other) + report = audit(response, targets=["target-a", "target-b"]) + self.assertEqual([item["target_id"] for item in report["targets"]], ["target-a", "target-b"]) + self.assertEqual(report["status"], "ALL_ELIGIBLE_READY") + self.assertEqual(report["counts"]["ready_nodes_in_fresh_reports"], 2) + + def test_unobserved_and_absent_timestamps_are_unknown(self): + for change in ( + {"ready": False, "ready_node_count": 0, "eligible_node_count": 0, + "observed_at": None, "valid_until": None}, + {"observed_at": None}, {"valid_until": None}, + ): + with self.subTest(change=change): + response = sample_response() + response["targets"][0].update(change) + report = audit(response) + self.assertEqual(report["status"], "UNKNOWN") + self.assertEqual(report["counts"]["fresh_targets"], 0) + + def test_expiry_boundary_is_not_ready_and_counts_are_not_fresh(self): + response = sample_response() + response["targets"][0]["valid_until"] = AS_OF + report = audit(response) + self.assertEqual(report["targets"][0]["reason"], "EXPIRED") + self.assertEqual(report["counts"]["ready_nodes_in_fresh_reports"], 0) + self.assertFalse(report["all_reported_eligible_nodes_ready"]) + + def test_repeated_reads_do_not_extend_original_observation(self): + response = sample_response() + before = copy.deepcopy(response) + fresh = audit(response) + stale = audit(response, as_of="2026-10-30T00:02:00Z") + self.assertEqual(response, before) + self.assertEqual(fresh["targets"][0]["valid_until"], + stale["targets"][0]["valid_until"]) + self.assertEqual(fresh["targets"][0]["observed_at"], + stale["targets"][0]["observed_at"]) + self.assertEqual(stale["status"], "UNKNOWN") + + def test_old_observation_is_unknown_even_with_later_runtime_expiry(self): + response = sample_response() + report = audit(response, as_of="2026-10-30T00:01:00Z") + self.assertEqual(report["targets"][0]["reason"], "OBSERVATION_TOO_OLD") + self.assertEqual(audit(response, as_of="2026-10-30T00:01:00Z", + max_age_seconds=120)["status"], "ALL_ELIGIBLE_READY") + + def test_future_observation_and_invalid_time_window(self): + for observed, until, reason in ( + ("2026-10-30T00:00:31Z", "2026-10-30T00:02:00Z", "FUTURE_OBSERVATION"), + ("2026-10-30T00:00:00Z", "2026-10-30T00:00:00Z", "INVALID_TIME_WINDOW"), + ): + with self.subTest(reason=reason): + response = sample_response() + response["targets"][0].update(observed_at=observed, valid_until=until) + self.assertEqual(audit(response)["targets"][0]["reason"], reason) + + def test_rfc3339_nanoseconds_do_not_truncate_future_or_expiry(self): + response = sample_response() + response["targets"][0]["observed_at"] = "2026-10-30T00:00:30.000000001Z" + self.assertEqual(audit(response)["targets"][0]["reason"], "FUTURE_OBSERVATION") + response["targets"][0].update( + observed_at="2026-10-30T00:00:00Z", + valid_until="2026-10-30T00:00:30.000000001Z", + ) + self.assertEqual(audit(response)["status"], "ALL_ELIGIBLE_READY") + + def test_timezone_offsets_represent_the_same_instant(self): + response = sample_response() + response["targets"][0].update( + observed_at="2026-10-30T09:00:00+09:00", + valid_until="2026-10-30T09:02:00+09:00", + ) + self.assertEqual(audit(response)["status"], "ALL_ELIGIBLE_READY") + + def test_invalid_timestamp_is_not_echoed(self): + for value in ("secret-value", "2026-10-30T00:00:00", 1, + "2026-10-30T00:00:00-00:00", "2026-10-30T00:00:00+09:60", + "2026-02-30T00:00:00Z", "2026-10-30T00:00:00.\u0661Z"): + with self.subTest(value=value): + response = sample_response() + response["targets"][0]["observed_at"] = value + report = audit(response) + self.assertEqual(report["targets"][0]["reason"], "INVALID_TIMESTAMPS") + self.assertNotIn("secret-value", json.dumps(report)) + + def test_fresh_negative_observation_is_not_ready(self): + response = sample_response() + response["targets"][0].update(ready=False, ready_node_count=0) + self.assertEqual(audit(response)["status"], "NOT_READY") + response["targets"][0]["eligible_node_count"] = 0 + self.assertEqual(audit(response)["status"], "NOT_READY") + + def test_wrong_architecture_and_image_set_are_rejected(self): + for change in ({"architecture": "ARM64"}, {"image_set_id": "sha256:" + "f" * 64}): + with self.subTest(change=change), self.assertRaises(ValueError): + audit(dict(sample_response(), **change)) + + def test_missing_helper_or_new_digest_cannot_reuse_old_response(self): + for images in ( + [sample_artifact()["workload"]["containers"][0]["image"]], + [item["image"].replace("a" * 64, "d" * 64) + for item in sample_artifact()["workload"]["containers"]], + ): + with self.subTest(images=images), self.assertRaises(ValueError): + response = sample_response() + response["image_set_id"] = calculate_image_set_id("AMD64", images) + audit(response) + + def test_unexpected_and_duplicate_response_targets_are_rejected(self): + for targets in ( + [dict(sample_response()["targets"][0], target_id="different-target")], + sample_response()["targets"] * 2, + [None], + ): + with self.subTest(targets=targets), self.assertRaises(ValueError): + audit(dict(sample_response(), targets=targets)) + + def test_invalid_counts_and_boolean_types_are_rejected(self): + changes = ( + {"ready": "true"}, {"ready": 1}, {"ready_node_count": True}, + {"eligible_node_count": True}, {"ready_node_count": -1}, + {"eligible_node_count": -1}, {"ready_node_count": 2}, + {"ready_node_count": 0}, {"ready": False}, + {"eligible_node_count": 2**31}, {"ready_node_count": 1.0}, + ) + for change in changes: + with self.subTest(change=change), self.assertRaises(ValueError): + response = sample_response() + response["targets"][0].update(change) + audit(response) + + def test_operation_history_is_not_a_current_readiness_response(self): + response = sample_response() + response.pop("targets") + response.update(status="SUCCEEDED", nodes=[{"status": "READY"}]) + with self.assertRaises(ValueError): + audit(response) + + def test_unsupported_bundle_or_failed_evidence_is_rejected(self): + for change in ( + {"category": "koth"}, {"runtime_type": "VM"}, {"scan_result": "FAIL"}, + {"registry_revision": 9}, {"source_ref": "refs/heads/feature/example"}, + ): + with self.subTest(change=change), self.assertRaises(ValueError): + artifact = dict(sample_artifact(), **change) + audit(artifact=artifact) + + def test_image_and_target_runtime_limits_are_enforced(self): + for targets in ([], ["target-a"] * 2, ["target-" + str(i) for i in range(51)], + [" target-a"], ["target-a\nvalue"], [1], ["x" * 129]): + with self.subTest(targets=targets), self.assertRaises(ValueError): + audit(targets=targets) + response = dict(sample_response(), targets=[sample_response()["targets"][0]] * 51) + with self.assertRaises(ValueError): + audit(response) + artifact = sample_artifact() + container = artifact["workload"]["containers"][0] + artifact["workload"]["containers"] = [ + dict(container, name=f"web-{i}", image=container["image"].replace("/web@", f"/web-{i}@")) + for i in range(33) + ] + artifact["evidence"]["containers"] = [ + {"name": item["name"], "published_image": item["image"], + "vulnerability_scan": "PASS", "secret_scan": "PASS"} + for item in artifact["workload"]["containers"] + ] + with self.assertRaises(ValueError): + audit(artifact=artifact) + + def test_invalid_age_and_audit_time_are_rejected(self): + for value in (0, -1, True, 1.0, 86401): + with self.subTest(value=value), self.assertRaises(ValueError): + audit(max_age_seconds=value) + with self.assertRaises(ValueError): + audit(as_of="not-a-timestamp") + + def test_execution_values_and_response_cache_metadata_are_not_copied(self): + artifact = sample_artifact() + artifact["schema_version"] = "2.1" + artifact["workload"]["containers"][0].update( + env={"PASSWORD": "private-value"}, secret_env={"FLAG": "secret-reference"}) + response = dict(sample_response(artifact), readinessCache="miss", + token="do-not-copy", operation_status="SUCCEEDED") + response["targets"][0]["private"] = "hidden-body" + report = audit(response, artifact) + self.assertEqual(report["status"], "ALL_ELIGIBLE_READY") + for value in ("private-value", "secret-reference", "do-not-copy", "hidden-body", "PASSWORD"): + self.assertNotIn(value, json.dumps(report)) + self.assertFalse(report["limits"]["operation_success_proven"]) + self.assertFalse(report["limits"]["cold_pull_proven"]) + + def input_files(self, root, response=None): + artifact_path, response_path = root / "artifact.json", root / "response.json" + artifact_path.write_text(json.dumps(sample_artifact()) + "\n") + response_path.write_text(json.dumps(response or sample_response())) + return artifact_path, response_path + + def test_private_output_keeps_exact_bundle_hash_and_time_basis(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + output = root / "private-report" + report = write_audit(artifact, response, ["target-a"], output, as_of=AS_OF) + self.assertEqual(report["bundle"]["artifact_sha256"], + hashlib.sha256(artifact.read_bytes()).hexdigest()) + self.assertEqual(report["time_basis"], "EXPLICIT_AS_OF") + self.assertEqual({p.name for p in output.iterdir()}, {"readiness-audit.json"}) + self.assertEqual(stat.S_IMODE(output.stat().st_mode), 0o700) + self.assertEqual(stat.S_IMODE((output / "readiness-audit.json").stat().st_mode), 0o600) + self.assertIn("generated_at", report) + + def test_default_audit_time_is_explicitly_marked_as_system_utc(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + report = write_audit(artifact, response, ["target-a"], root / "out") + self.assertEqual(report["time_basis"], "SYSTEM_UTC_NOW") + self.assertEqual(report["as_of"], report["generated_at"]) + + def test_empty_explicit_time_is_not_silently_replaced_with_current_time(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + with self.assertRaises(ValueError): + write_audit(artifact, response, ["target-a"], root / "out", as_of="") + self.assertFalse((root / "out").exists()) + + def test_existing_output_directory_is_not_overwritten(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + output = root / "report" + write_audit(artifact, response, ["target-a"], output, as_of=AS_OF) + previous = (output / "readiness-audit.json").read_bytes() + with self.assertRaises(FileExistsError): + write_audit(artifact, response, ["target-a"], output, as_of=AS_OF) + self.assertEqual((output / "readiness-audit.json").read_bytes(), previous) + + def test_invalid_input_does_not_create_output_and_rejects_symlinks(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + response.write_text("{}") + with self.assertRaises(ValueError): + write_audit(artifact, response, ["target-a"], root / "out", as_of=AS_OF) + self.assertFalse((root / "out").exists()) + response.write_text(json.dumps(sample_response())) + link = root / "linked.json" + link.symlink_to(artifact) + with self.assertRaises(ValueError): + write_audit(link, response, ["target-a"], root / "out", as_of=AS_OF) + response.write_bytes(b"x" * (1024 * 1024 + 1)) + with self.assertRaises(ValueError): + write_audit(artifact, response, ["target-a"], root / "out", as_of=AS_OF) + + def test_ambiguous_or_nonfinite_json_is_rejected(self): + for contents in ('{"private":"hidden","private":"hidden-again"}', + '{"private":NaN}', '{"private":Infinity}'): + with self.subTest(contents=contents), tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + artifact, response = self.input_files(root) + response.write_text(contents) + with self.assertRaises(ValueError): + write_audit(artifact, response, ["target-a"], root / "out", as_of=AS_OF) + self.assertFalse((root / "out").exists()) + + def run_cli(self, root, response): + artifact_path, response_path = self.input_files(root, response) + return subprocess.run( + [sys.executable, "-B", "-m", "scripts.audit_node_image_readiness", + "--artifact", str(artifact_path), "--readiness-response", str(response_path), + "--target-id", "target-a", "--output-dir", str(root / "out"), + "--as-of", AS_OF], + cwd=ROOT, capture_output=True, text=True, timeout=10, + ) + + def test_cli_exit_codes_distinguish_complete_unknown_and_invalid(self): + for response, expected in ( + (sample_response(), 0), + (dict(sample_response(), targets=[]), 1), + (dict(sample_response(), image_set_id="sha256:" + "f" * 64), 2), + ): + with self.subTest(expected=expected), tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + result = self.run_cli(root, response) + self.assertEqual(result.returncode, expected, result.stderr) + self.assertEqual((root / "out").exists(), expected != 2) + self.assertNotIn("target-a", result.stdout + result.stderr) + self.assertNotIn("Traceback", result.stderr) + + def test_direct_script_cli_also_imports(self): + result = subprocess.run( + [sys.executable, "-B", str(ROOT / "scripts/audit_node_image_readiness.py"), "--help"], + cwd=ROOT, capture_output=True, text=True, timeout=10, + ) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertIn("--readiness-response", result.stdout) + + def test_cli_errors_do_not_print_private_response(self): + with tempfile.TemporaryDirectory() as temporary: + result = self.run_cli(Path(temporary), {"private": "do-not-print-this"}) + self.assertEqual(result.returncode, 2) + self.assertNotIn("do-not-print-this", result.stdout + result.stderr) + self.assertNotIn("Traceback", result.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_workflow_contracts.py b/tests/test_workflow_contracts.py index 6362cdb..85b458c 100644 --- a/tests/test_workflow_contracts.py +++ b/tests/test_workflow_contracts.py @@ -506,6 +506,16 @@ def test_pipeline_self_test_calls_reusable_supply_chain(self): workflow["jobs"][job_name].get("if", ""), ) + def test_self_test_server_fixture_updates_brotli_and_keeps_non_root_user(self): + path = ROOT / "tests/fixtures/info-valid/prob/for_organizer/web/Dockerfile" + instructions = [line.strip() for line in path.read_text().splitlines() if line.strip()] + self.assertEqual(instructions, [ + "FROM nginxinc/nginx-unprivileged:1.29-alpine", + "USER root", + "RUN apk upgrade --no-cache brotli-libs", + "USER 101", + ]) + def test_caller_example_separates_branch_validation_from_main_publish(self): path = ROOT / "docs/challenge-caller-example.yml" workflow = load_workflow(path)