Skip to content

Certify the Microcosm UK national and local-area pair with policyengine-uk 2.132.6 - #685

Merged
juaristi22 merged 26 commits into
mainfrom
uk-553-certify-microcosm-pair
Oct 21, 2026
Merged

juaristi22 merged 26 commits into
mainfrom
uk-553-certify-microcosm-pair

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #553

This PR makes Microcosm UK 2024-25 the UK default dataset, in place of the Enhanced FRS 2024-25. Constituency and local-authority regions use the Microcosm local-area release. It uses the tooling from #615.

This is an interim certification. Both Microcosm cuts were built with policyengine-uk 2.132.6, so this PR pins that version instead of a newer one. A Microcosm pair built on policyengine-uk 2.134.9 or later is planned within about a week, and it will replace this pair. "Known gaps until the next pair" below lists what that pair is expected to change.

What changes

  • Default dataset:
    • Before: enhanced_frs_2024_25 from policyengine-uk-data 1.58.1.
    • After: microcosm_uk_2024_25, national cut microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71.
  • Area regions: region_datasets.constituency and region_datasets.local_authority route to microcosm_uk_2024_25_dense, local-area cut microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe (regional_release_kind: local_area).
    • The default UK region registry now has 1,016 regions: the UK, 4 countries, 650 constituencies and 361 local authorities.
    • Bundle installation still downloads only the national file.
  • Pins:
    • policyengine-uk 2.130.1 → 2.132.6 and policyengine-core 3.33.0 → 3.33.3. These are the versions both cuts were built with (compatibility_basis: built_with_model_package).
    • The UK certificate follows the bundle's core pin (core_binding: bundle_pin), so the pin can move with the US line. Both Microcosm releases claim 3.33.3. If the pin moves outside a release's claims, certification and the strict check warn rather than refuse. The skill's new "The core" section has the rule.
    • uv.lock was re-resolved with uv 0.12.14, the version CI uses. Only these two packages move.
  • Enhanced FRS: enhanced_frs_2024_25, frs_2024_25 and their _tiny variants stay loadable by name, through dataset_overlays.uk entries at 1.58.1.
    • The enhanced FRS weight matrices are no longer in the bundle.
    • Area regions do not use them: they row-filter constituency_code_oa and la_code_oa on the local-area file, which carries both columns.

How it was made

python scripts/bundle.py certify-data --country uk --data-producer populace \
  --manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71/releases/microcosm-uk-2024-25-national/release_manifest.json" \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe/releases/microcosm-uk-2024-25-dense/release_manifest.json" \
  --model-version 2.132.6 --core-version 3.33.3
  • Rebased on main at 5f97905 (6.9.0). The certification was re-run there, so the 6.9.0 stamps and both TROs are regenerated.
  • Every artifact answered the HEAD check.
  • After certification:
    1. uv lock (uv 0.12.14), then release_lock.py.
    2. The dataset_overlays.uk entries were added.
    3. bundle.py generate --include-tros.
    4. bundle.py check --published-spm --include-tros --strict-tros passes.
  • Tests: the tests that pinned the enhanced FRS now pin this pair (test_models, test_release_manifests, test_uk_regions, test_bundle, test_bundle_metadata, test_dataset_runtime).
    • The receipt test helper now points the certificate's core version at the installed core, because this certificate names its core.
    • The automated-qualification tests build policyengine-uk-data candidates. Against the new bundle every one of them would be other_line, so they now run against a frozen copy of the last bundle certified on that line (tests/fixtures/uk_qualification/uk_data_line_bundle.json, main's 6.9.0 release). One shared load_candidate_cli gives the candidate CLI that bundle. The constant-rewrite test still uses the packaged bundle, and takes it to a successor Microcosm cut.
    • The UK model-surface snapshot changes only in its data package name. The household snapshots do not change on policyengine-uk 2.132.6.
  • Docs: countries.md, microsim.md and regions.md describe the new default.

Review fixes

These commits address the review at 8bd01dd, in this order:

  • 271dd5d: the released-bundle replay request works for a dataset-repo release: the repository type is passed, a missing size is read from Hugging Face, and the job warns when the request cannot be built.
  • 22a2c49: certification removes the overlays a new release certifies, and bundle.py check refuses an overlay that shadows a certified name. This keeps a rollback importable.
  • 7e8df2d: the strict TRO check refuses a core pin that differs from a core the certificate names.
  • 8ea62c1: candidate preparation keeps or drops the area routing by data line.
  • 8d76608: recovery and check refreshes skip pairs from another data line. The tests share one CLI loader.
  • 5882164: receipts give a regional release's datasets their own identity.
  • e1ae884: nested bundle artifacts are cached under their release path.
  • 5f3df72: area outputs that lack their codes name the local-area dataset.
  • 02ec73e: the changelog, docs and the skill record the switch's consequences and the UC-floor exception.

These commits address the second review, at 02ec73e:

  • e691df0: one core-drift check covers every basis. The skill and docs/bundles.md say what a named core binds, and the skill's UK commands pass --core-version.
  • 90172bd: one overlay-shadowing rule, used at runtime, in bundle.py check and by certification. Certification also warns about every name that stops resolving.
  • 741b269: recovery skips every pair from another data line, and promotion closes those pairs' pull requests. Tests that need a routed Microcosm main read a frozen copy of this bundle.
  • e40d3ba: the release consumer uses the shared Hugging Face helpers, and the request tests pin every request detail.
  • df5a421: the cache-path guard refuses Windows-style escapes.
  • 3b2db90: preparation refuses, up front, a same-line pair that main's local-area cut cannot serve.
  • 2993625: the weight-matrix removal has its own fragment, which makes this a minor release. The docs say that receipts for area runs are uncertified.
  • 6bed939: the skill records the acceptance of the comparison gates.

These commits address the third review, at 6bed939:

  • 6ee9157: a certificate states how it covers the core. core_binding: bundle_pin is the default and follows the pin; certified names and binds the core. Certification and the strict check warn when the pin is outside a release's claims. UK is re-certified to follow the pin.
  • b25cb78: promotion closes an other-line PR only at the commit its receipt qualified. Promote never drafts a PR it closed. Same-line preparation on a routed main is refused.
  • 1fda069: one overlay refusal message; a cache-path guard for drives in any segment; size requests ask for the identity encoding.
  • 0f5e05b: the skill records both needs-review acceptances, and the changelog the new behaviours.
  • 12cfbc0: the runbook says when promotion closes a cross-line PR.

Acceptance evidence

Budget reform contract 2.1.0 (#574's qualification gate)

This was run on the national cut with policyengine-uk 2.132.6 and policyengine-core 3.33.3.

Status Checks
pass 857
fail 46
pending 34
unsupported 122
error 0
  • Population checks: all 19 pass.
  • The 46 failures are all in extended_freeze, in 2034-35. They check that indexation resumes after the freeze, which needs policyengine-uk 2.134.9 or later. The 2.130.1 pin on main has the same gap. We accept it for this interim pair.
  • The 34 pending checks are the CGT-default checks, which do not gate. 6 of them hold on 2.132.6.
  • On 2.134.9: with the same cut, all 34 CGT-default checks hold and the contract passes with no failures. That was a pre-flight run against a simulated claim, not this certificate.

Other #574 gates

These were also run on the national cut with policyengine-uk 2.132.6.

  • Supported years: pass. Microcosm supports 2024–2032, the same years as the enhanced FRS 1.58.1 pair. That covers the required years (2025–2030) and the Budget horizon (2026–2031).
  • Budget-critical inputs, against the enhanced FRS 1.58.1 pair: needs review.
    • All 52 critical inputs are present and usable.
    • One other model input is missing from the data and was present before: incapacity_benefit_reported. The enhanced FRS column carries no values, because Incapacity Benefit is closed, so both datasets give zero.
    • Accepted by María Juaristi, recorded in the data certification skill.
  • Reform responsiveness: needs review.
    • Nine of the ten standard reforms pass in every year.
    • For the tenth, the tool's own check passes. But the tool's small-cell rule withholds that reform's figures in every year, so the gate cannot recompute them.
    • A disclosure-control item on that reform went to the author separately.
    • Accepted by María Juaristi, recorded in the data certification skill.
  • Not run: the metrics review and the scorecard replay. These measure a candidate against the previous certified pair, and a change of data line moves the totals by design. The totals table below gives that comparison instead, and the data certification skill records the acceptance.

Universal Credit coverage floor (#491)

The floor compares 2025 values for Great Britain with two benchmarks.

Measure Microcosm Ratio to the floor's benchmark Enhanced FRS 1.58.1 ratio
Benefit units with UC 6.0m 0.897 (benchmark 6.7m) fail 0.927
UC spend £76.0bn 0.959 (benchmark £79.3bn) pass 0.977

We accept this result. The floor's benchmarks are not the figures that Microcosm calibrates to:

Measure Floor benchmark Microcosm's target Microcosm against its target
UC caseload 6.7m (policyengine-uk-data's derivation) 6.2m households on UC in payment (DWP Stat-Xplore) −3%
UC spend £79.3bn £76.1bn on target

This is not the failure that #491 describes, where a dataset carried about half of the caseload. The benchmark definitions will be settled when the UK data checks are consolidated with the data pipelines, which is a separate piece of work.

Weighted national totals, 2025, UK (£bn)

Moving policyengine-uk from 2.130.1 to 2.132.6 on the enhanced FRS changes all ten totals by 0.0%. So this table shows only the data change.

Total Enhanced FRS 1.58.1 (today) Microcosm Change Microcosm's target
Household net income 1,650.7 1,858.9 +12.6% none
Income tax 298.0 325.2 +9.1% 325.1 (OBR)
National Insurance 52.6 53.7 +2.1% none
Capital gains tax 12.6 27.3 +116% 22.5 (HMRC)
Stamp duty land tax 15.9 15.0 −5.9% 15.0 (OBR)
Universal Credit (GB) 77.4 76.0 −1.8% 76.1 (OBR)
Child benefit 16.2 13.4 −17.4% 13.4 (OBR)
Pension credit 6.7 6.4 −4.5% 6.1 (OBR)
State pension 122.5 142.0 +15.9% 142.3 (DWP and DfC)
Council tax 49.6 46.9 −5.4% 48.8 (OBR)
  • Most totals: they move onto the agency figures that Microcosm calibrates to. The changes in 2027 and 2030 are similar.
  • Capital gains tax is the exception. It is 21% above Microcosm's own HMRC target.
  • Household net income has no single agency figure to compare it with.
  • The local-area file is within 3% of the national file on all ten totals.

All figures are weighted totals over the whole population.

Known gaps until the next pair

On the same national cut, policyengine-uk 2.134.9 changes most totals by less than 1%. The exceptions:

Also missing until then:

After this merges

  • Data line: the certified UK data line becomes (microcosm-data, policyengine/populace-uk-private). Automated policyengine-uk-data candidates are then other_line: an unforced run stops there, and none is ever promoted.
  • Candidate PRs: on its first run after this merges, the promote job closes the open pull requests of pairs from another data line, each at the commit its receipt qualified. Today that is the 19 uk-data candidate PRs, Certify UK candidate 2.136.0-1.58.2-7f2b5855131e #663 to Certify UK candidate 2.141.4-1.58.2-c20a30ec4be9 #687. The ledger keeps their evidence.
  • US core claim: the strict check now warns that the US Populace release claims core 3.32.5 only, while the pin is 3.33.3. On main the pin was already 3.33.0, so this was already the case; the warning makes it visible.
  • Release: the removed fragment makes this a minor release.
  • Downstream:
    • UK: keep constituency and local-authority results available through a switch to Microcosm, then serve them from its local-area dataset policyengine-sim-api#728 covers serving UK constituency and local-authority results from the local-area dataset. This bundle routes both area types, so sim-api's uk_area_regions_routed() holds.
    • Scorecard and Microcosm, outside this repository:
      • Microcosm's release assemblers should write size_bytes.
      • The scorecard's own discovery needs size_bytes for the pinned cut, so it needs a fallback of its own.
      • The scorecard's Hugging Face token needs read access to policyengine/populace-uk-private. That access is not yet checked.
    • Area receipts: receipts for constituency and local-authority runs are uncertified, because the certificate covers the national dataset only. Extending the certificate and the TRO to the routed dataset is a follow-up.
    • policyengine-api pins policyengine[models]==6.7.5. Its uk_constituency_breakdown and uk_local_authority_breakdown apply the enhanced FRS weight matrices from policyengine/policyengine-uk-data to the default dataset's households. They must change before policyengine-api takes a release with this bundle.

Checks

  • Local: the touched modules and the UK, qualification and receipt modules pass. CI runs the full suite.
  • Bundle: bundle.py check --published-spm --include-tros --strict-tros and release_lock.py pass.

🤖 Generated with Claude Code

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 8bd01dd. There are 15 findings below, most severe first. Six are on lines in the diff and are inline comments. The other nine are in code this PR doesn't touch but whose behaviour the new bundle changes, so their detail is here.

Before merging: the branch conflicts with main since the 6.9.0 release (5f97905), which rewrote manifest.json, both TROs, pyproject.toml and uv.lock. After the rebase the 6.8.0 stamps become 6.9.0 and the TROs need regenerating.

  1. Constituency and local-authority outputs now raise on the default dataset. Inline on manifest.json:106.

  2. With an HF token set, import policyengine fails when the installed policyengine-uk isn't 2.132.6. model_version.py#L151

    • The national release manifest claims policyengine-uk ==2.132.6 only. PolicyEngineUKLatest() is built at import (uk/model.py:489) and calls certify_data_release_compatibility with no guard. With a token it fetches that manifest and raises Data release manifest is not certified for the runtime model version …. So import policyengine fails in that environment, US use included. Without a token it falls back to unverified_data_release_manifest_unavailable.
    • Triggers: policyengine-uk 2.134.9 (the planned next pair, or the #602 work) or 2.130.1 installed next to this release. On main the eFRS publisher claim covered versions newer than its build, which is how 2.130.1 was certified, so drift only warned.
    • tests/test_manifest_version_mismatch.py says version drift "must not raise", but it stubs this call.
  3. The post-release scorecard request can't be built for this bundle, and it fails silently. release_consumers.py#L55-L63, uk_qualification.py#L1768-L1814

    • _hf_file builds https://huggingface.co/{repo_id}/resolve/… with no datasets/. The UK data package is now a dataset repo, so the fetch gets a 401 or 404.
    • Even with the URL fixed, released_bundle_request reads artifact.size_bytes. Microcosm release manifests don't carry it, and the request model requires an integer ≥ 1 (qualification.py:792).
    • The step is continue-on-error (push.yaml:313-318) and the next steps are gated on its outcome. So no released-bundle replay is sent, and the job stays green.
    • test_released_bundle_request_is_valid_for_the_scorecard now runs on the frozen model-repo bundle, with a stub hf_file and a synthetic size, so it can't catch either problem.
  4. The new eFRS overlays make a rollback bundle unimportable. Inline on manifest.json:271.

  5. A core-only bump would silently decertify every UK receipt. Inline on manifest.json:24.

  6. A forced requalification always fails at prepare. uk_qualification.py#L774-L794

    • prepare_candidate_commit runs certify-data without --regional-manifest-uri or --drop-regional-release, and the routed bundle now trips the guard in certification.py:1348-1358. So the forced cross-line run in the runbook (uk-qualification.md:87-88) takes the promote-uk lock and then exits 2. No workflow input can pass either flag. It fails before staging, so nothing is left half-done.
    • The prepare test now seeds from the unrouted frozen fixture (test_uk_qualification_pipeline.py:605), so it can't see this.
  7. Receipts for runs on the local-area file name the national build. receipt.py#L3788

    • _data_identity treats every datasets entry as part of the certified release. A receipt for microcosm_uk_2024_25_dense therefore carries the national cut as build_id and release_manifest_revision, next to the dense dataset_revision. The national release manifest doesn't list the dense file.
    • These receipts are uncertified, so nothing checks the fields. But they misattribute every constituency and local-authority result, and they enter the execution identity.
    • Fix: count an entry as in the release only when its revision is the record's own.
  8. The ledger concurrency guard now passes vacuously. test_qualification_ledger_concurrency.py#L64-L69

    • This file wasn't moved to the frozen bundle. BASE is the packaged manifest and PAIR is a uk-data make_pair, so plan_promotion drops the pair as other-line, and assert plan.ready_commit is None (L284-285) can't fail.
    • A promotable receipt naming B's commit gives ready_commit=None against the packaged bundle and bbbb… against the frozen one.
    • All three uses of BASE (L68, L170, L284) need to move together. Choosing the bundle in one place would have covered this file: a shared CLI loader in tests/qualification_fixtures.py, instead of the ten inline cli.BUNDLE_PATH overrides, five of which are never read.
  9. The catch-up keeps re-dispatching dead uk-data attempts. check_uk_candidate.py#L529-L550

    • cmd_recoverable has no data-line filter. An attempt abandoned around the merge stays recoverable forever after its lease. Each two-hourly re-dispatch (force=false) resolves other_line before the recovery branch (uk_qualification.py:423-463), so nothing clears it: about 12 green no-op runs a day per pair.
    • The ledger currently has one at risk: d14ea200…/attempts/f40946222a8f, whose lease expired at 18:02Z with no replay request.
    • cmd_refresh_pr_checks likewise keeps re-running CI on the 17 uk-data candidate PRs, though that is capped.
    • Fix: skip pairs that are off main's data line.
  10. UK and US calibration_diagnostics.json share a cache path. dataset_materialization.py#L119

    • Materialized files are cached by basename, so in a shared data_dir the two countries overwrite each other. The first result's path then holds the other country's bytes, and every alternate call re-downloads.
    • Nothing materializes both today.
    • Fix: namespace the destination by country or revision.
  11. The constant rewrite collides on this pair's values. Inline on test_uk_qualification_pipeline.py:535.

  12. The changelog and docs miss two consequences. Inline on the changelog fragment.

  13. next() without a default in the receipt test. Inline on test_execution_receipt.py:308.

  14. Each area run loads the whole local-area file, and the simulation cache keeps it. datasets.py#L365-L370

    • load_datasets builds a new PolicyEngineUKDataset on every call. It reads every table of the ~720 MB file (the eFRS is 128 MB) plus the data-year tables, only to keep about 1/650 or 1/361 of households (model.py:193-198). sim-api's executor calls ensure_datasets on each request.
    • Simulation.ensure() caches a shallow copy that keeps the full input dataset (simulation.py:228-236). The LRU holds 100 entries and evicts by count only, though a cache hit reads only outputs. When each request loads its own dataset, warm workers can hold about 1–2 GB per cached area request.
    • The US local-area path already works this way; this PR turns it on for 1,011 UK areas.
    • Fix: a per-process dataset cache, and caching the simulation without its input dataset.
  15. The UC-floor exception isn't recorded where agents read the rule. data-certification.md#L173-L176

    • The skill, which CLAUDE.md makes canonical, says a UK default switch carries the #491 floor within 10%, and uk_uc_floor.json says "must".
    • After merge the certified default fails the floor (0.897), and the acceptance lives only in this PR body. The next agent that applies the skill will flag it again or treat the floor as optional.
    • Fix: record the exception in the skill or uk_uc_floor.json, or at least in the changelog.

},
"default_dataset": "enhanced_frs_2024_25",
"default_dataset_uri": "hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@1.58.1",
"default_dataset": "microcosm_uk_2024_25",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. Constituency and local-authority outputs now raise on the default dataset.

The Microcosm national file has no constituency_code_oa or la_code_oa; only the local-area file does. The eFRS it replaces had both. So on simulations built from the default dataset, these calls raise:

  • compute_uk_geography_disclosure(..., levels=("region", "constituency")), the example in docs/outputs.md:279-284 and docs/disclosure.md:14-19. Error: "gives no household an area in any of ['constituency_code_oa']".
  • compute_uk_constituency_impacts and compute_uk_local_authority_impacts. Error: "require baseline household column 'constituency_code_oa'".

All of these run on main. Checked against the certified files: the national file raises, while the eFRS and the local-area file don't.

Only the existing caveat at docs/regions.md:169-171 says to use the routed dataset. The changelog and those two examples don't.

Suggest a changelog and docs note to run these on microcosm_uk_2024_25_dense, and an error message that names that dataset.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5f3df72. Both errors now name the dataset the bundle routes the area regions to, through a new uk_area_dataset_hint that gives nothing when no area is routed. docs/outputs.md and docs/disclosure.md now say to run constituency and local-authority levels on microcosm_uk_2024_25_dense. The changelog says so too (02ec73e).

"revision": "655dd07e4bb9c777b00dac044949611f1feb824f",
"sha256": "584ae33d80ca0431254610a3f8254d132da73477d31966d6446282861ecae50d"
},
"enhanced_frs_2024_25": {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4. The new eFRS overlays make a rollback bundle unimportable.

These four overlay names are the ones a uk-data release certifies. enhanced_frs_2024_25 is also what the documented area_coded_national flow carries (data-certification.md:63-80). certify_data_release rewrites data_releases without looking at dataset_overlays, so two paths break:

  • A rollback (certify-data on uk-data 1.58.x with --drop-regional-release): the overlay shadows the certified default.
  • The national-first flow: the overlay collides with a certified entry.

Either way certify-data exits 0 and plain bundle.py check passes. Then _apply_dataset_overlays raises on every get_release_manifest("uk"), so import policyengine fails. That includes installs without policyengine-uk, because the countries package builds the UK registry. Reproduced with the real certify-data.

CI would catch it, but the rollback path is the one used under time pressure.

Suggest certify-data apply or prune the overlays before writing, plus a line in the rollback docs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 22a2c49.

  • certify_data_release now removes any dataset_overlays.<country> entry whose name the new release certifies, and lists the removed names in its warnings.
  • bundle.py check now refuses an overlay that shadows a certified name, through validate_dataset_overlays in the stdlib validator.
  • A test runs the rollback: it certifies the enhanced FRS with --drop-regional-release over a routed bundle that carries the eFRS overlays.
  • The re-certification section of the skill now describes this.

"compatibility_basis": "legacy_compatible_model_package",
"data_build_fingerprint": "sha256:c094d7006fdd81df220090284212f16bbffb15624643bc442c4bb3e1b1e5dac8",
"data_build_id": "policyengine-uk-data-1.58.1"
"certified_for_core_version": "3.33.3",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5. A core-only bump would silently decertify every UK receipt.

This is the first built_with_model_package certificate on main to name a core. From here on, receipts certify this value, not the packages pin (receipt.py:3737-3738). Nothing keeps the two in step:

  • bundle.py update-packages --core … changes only packages (prepare_package_bundle_update.py:38).
  • certify-data --country us --core-version … also moves the shared pin.
  • check --strict-tros compares the core only for qualified_exact_pair (generate_trace_tros.py:141).

So a core-only bump makes every UK receipt uncertified while every check passes. Such bumps have happened before (390cb72). The model has a guard against the same drift (test_runtime_pin_update_requires_matching_certified_model), but the core has none.

The test changes hide this:

  • _bundle_for (test_execution_receipt.py:135-137) now overwrites the certified core with the installed one.
  • The differential case uses _certify_core, which edits both fields.
  • No test now reaches the or core fallback that receipts from 6.5.0–6.8.0 verify through.

Suggest:

  • a check that compares the pin with any certified core the certificate states;
  • one test that edits only packages.policyengine-core;
  • one test with the key absent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e8df2d. The strict TRO check now refuses a core pin that differs from any core a certificate names, not only for a qualified exact pair. So update-packages --core, or a US run with --core-version, now fails check --strict-tros until UK is re-certified.

New tests:

  • the strict check with a pin-only change;
  • a receipt after a pin-only edit, where the core the certificate names decides;
  • a certificate that names no core, where the pin is the certified core, in both directions.

SUCCESSOR_CUT = "microcosm-uk-2024-25-national-20261101T000000Z-aaaaaaaa"


def _successor_record(old):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11. The constant rewrite collides on this pair's values.

certified_constant_replacements rewrites by value, and this certificate makes several values coincide:

  • model and built-with are both 2.132.6;
  • build id and revision are both the cut tag;
  • the UK data version "0.1.0" equals the US one.

Simulated on copies of the constant files:

  • A model-only bump also rewrites UK_BUILT_WITH_MODEL_VERSION.
  • A data-version bump rewrites US_DATA_RELEASE_VERSION and the US assertion in test_models.py:140.
  • A uk-data record writes its build id into UK_DATA_RELEASE_REVISION.
  • The dense pins (UK_LOCAL_AREA_RELEASE_ID, UK_CERTIFIED_LOCAL_AREA_URI) are never rewritten.

_successor_record moves the colliding values together and keeps the data version, so this test can't see any of it. It's unreachable today, because resolve only builds uk-data pairs, and wrong pins would fail CI rather than ship. But the Microcosm automation will rely on this test.

Keying the replacements by constant would fix it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed. The rest is deferred.

  • Fixed (8ea62c1): a candidate from another data line no longer rewrites the constants at all, so a uk-data build id can no longer land in UK_DATA_RELEASE_REVISION.
  • Deferred: the other collisions need a promotion on the Microcosm line, and resolve cannot build one yet. I would key the replacements by constant as part of that automation. It also has to rewrite the local-area pins, and a rewrite by value cannot find those.

@@ -0,0 +1 @@
Switch the UK default dataset from the Enhanced FRS 2024-25 (policyengine-uk-data 1.58.1) to Microcosm UK 2024-25 (`microcosm_uk_2024_25`, cut `microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71`), certified with policyengine-uk 2.132.6 and policyengine-core 3.33.3, the versions it was built with. Constituency and local-authority regions now load the Microcosm local-area release (`microcosm_uk_2024_25_dense`, cut `microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe`) and are in the default UK region registry. The Enhanced FRS 2024-25 datasets stay loadable by name. UK results change with the data.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

12. The changelog and docs miss two consequences.

The fragment doesn't mention:

  • parliamentary_constituency_weights, local_authority_weights and local_geography_weights no longer resolve by name. resolve_dataset_reference falls through to the Microcosm release manifest, and materialize_dataset raises "Unknown managed dataset". Downstream bump PRs get the changelog as their notice, and sim-api's region fallback is one caller.
  • Area outputs on the default dataset need the local-area file (see the comment on manifest.json:106).

Separately, two docs are stale:

  • docs/households.md:108-110 still says the bundle certifies policyengine-core 3.33.0.
  • README.md:73-74 still calls the UK data repo a model repo.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 02ec73e.

  • Changelog: it now says that the three weight matrices no longer resolve by name, that area outputs need microcosm_uk_2024_25_dense, and which policyengine-uk version import accepts when a token is set.
  • docs/households.md: it now names core 3.33.3.
  • README: it now says the Microcosm default is in a dataset repository.

Comment thread tests/test_execution_receipt.py Outdated
# Same bytes registered under another certified dataset name.
record = bundle["data_releases"]["uk"]
record["datasets"]["frs_2024_25"]["sha256"] = record["certified_data_artifact"][
other = next(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

13. next() without a default.

A national-only re-certification (--drop-regional-release) leaves one H5 plus the JSON and NPZ files. With no second H5, this next() fails the case with StopIteration.

Either fix keeps the case independent of the routing:

  • register the default entry under a fresh name (not an overlay name);
  • or drop the .h5 filter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e8df2d. The case now registers the default entry under a fresh name, so it no longer depends on the routing.

Maria Juaristi and others added 12 commits October 20, 2026 19:55
…ne-uk 2.132.6

scripts/bundle.py certify-data on the 19 October national cut
(microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71), with its
local-area cut (microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe) as
the regional release, at policyengine-uk 2.132.6 and policyengine-core
3.33.3, the versions both cuts were built with.

Constituency and local-authority regions route to the local-area file
and join the default UK registry. The enhanced FRS 2024-25 datasets stay
loadable by name through dataset_overlays at 1.58.1. uv.lock was
re-resolved with uv 0.12.14 (only policyengine-core and policyengine-uk
move), and both TRO sidecars were regenerated. The tests that pinned the
enhanced FRS, and the UK model-surface snapshot, now pin this pair; the
receipt test helper also points a certificate's named core at the
installed core.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The qualification tests build their candidates on the policyengine-uk-data
line and read the packaged bundle, which is now certified on the Microcosm
line, so every such candidate became other_line. They now read a frozen
copy of the last bundle main certified on the uk-data line (2591af3),
including through the candidate CLI's BUNDLE_PATH. The constant-rewrite
test and the real-CLI replay test still read the packaged bundle: the
constant files pin it, and the rewrite now takes it to a successor cut on
its own line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK data package is now a Hugging Face dataset repository, and
Microcosm release manifests record no artifact sizes. The release
consumer addressed every repository as a model repository, and the
request needs the dataset's size, so the post-release replay request
could not be built, silently, as its step continues on error.

released_bundle_request now passes the repository type to hf_file and
reads a missing size through hf_size; release_consumers.py builds the
datasets/ URL and reads X-Linked-Size from a HEAD request. The release
job warns when the request cannot be built. The request test runs on
both the uk-data model repository and the packaged dataset repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Overlays may not shadow a certified dataset name, but certify-data
rewrote data_releases without looking at dataset_overlays. A rollback to
policyengine-uk-data, or a national-first route carrying the enhanced
FRS, would certify names the bundle keeps as overlays, and then every
release-manifest read, and so import policyengine, would fail.

Certification now removes the overlays the new release certifies and
says so in its warnings, and bundle.py check refuses an overlay that
shadows a certified name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This is the first certificate on main to name its core. Receipts certify
the core a certificate names, falling back to the core pin only when it
names none, but the strict TRO check compared the pin with the
certificate only for a qualified exact pair. So a core-only pin change
(update-packages --core, or certifying the other country with
--core-version) would pass every check and leave every UK receipt
uncertified.

The strict check now refuses a core pin that differs from any core a
certificate names. New tests cover the check, a pin-only edit against a
certificate that names its core, and a certificate that names none. The
receipt test for another dataset name no longer depends on the routing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prepare_candidate_commit ran certify-data without a regional flag, so on
a main routed to a local-area release every forced requalification
failed at preparation: certification refuses to drop the routing
silently, and no workflow input could pass a flag.

A pair on main's data line now keeps main's regional release, which must
serve the pair by its own claims. A pair from another line drops the
routing, which belongs to the line it would replace, and leaves the test
constants alone, since rewriting them by value from another line's
certificate writes into unrelated constants. The preparation test now
also runs on the packaged, routed bundle, and the runbook says what a
forced cross-line run does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmd_recoverable had no data-line filter, so an attempt for a uk-data
pair abandoned around the switch stayed recoverable: every two-hourly
re-dispatch resolved as other_line before recovery and cleared nothing.
It now skips an unforced attempt whose pair is off main's data line, and
cmd_refresh_pr_checks no longer re-runs CI for such pairs, which
promotion never takes.

The qualification tests now choose main's bundle in one place:
load_candidate_cli in tests/qualification_fixtures.py replaces ten
inline loaders, and the ledger concurrency test, which still read the
packaged bundle, now uses the uk-data-line bundle its pair belongs to,
so its promotion assertion is no longer vacuous. That frozen bundle is
now main's 6.9.0 release (5f97905).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_data_identity treated every certified dataset entry as part of the
primary release, so a receipt for a run on the local-area file named the
national cut as its build and manifest revision, beside the local
file's own revision. Only an entry at the primary release's revision
now takes the release's identity; any other entry is named, like an
overlay, by its own revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Materialized bundle artifacts were cached by basename, so in a shared
data directory the UK and US calibration_diagnostics.json overwrote each
other. A nested artifact now keeps its release path under the data
directory; top-level files, every H5 among them, keep their cache path.
A path that is absolute or leaves the repository is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Microcosm national default has no constituency_code_oa or
la_code_oa; only the local-area file does, while the enhanced FRS it
replaces had both. So constituency and local-authority outputs on the
default dataset now raise. Their errors now name the dataset the bundle
routes those regions to, and the disclosure and outputs docs say to run
those levels on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changelog now says that area outputs need the local-area dataset,
that the enhanced FRS weight matrices no longer resolve by name, and
which policyengine-uk version import accepts with a token. The data
certification skill records the accepted Universal Credit floor
exception, so the next certification neither flags it again nor treats
the floor as optional. households.md and the README no longer name the
old core pin or call the UK data repository a model repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-553-certify-microcosm-pair branch from 8bd01dd to 02ec73e Compare October 20, 2026 20:47
@juaristi22

Copy link
Copy Markdown
Contributor Author

Addressed at 02ec73e. The six inline findings have replies in their threads; the other nine are below.

Rebase. The branch is rebased on main at 5f97905 (6.9.0). I re-ran the certification there, so the 6.9.0 stamps and both TROs are regenerated. check --published-spm --include-tros --strict-tros and release_lock.py pass.

  • 2. Import with a token and another policyengine-uk. No code change: main behaves the same way today.
    • The enhanced FRS 1.58.1 claims are exact (==2.122.2, ==2.130.1).
    • The data-build fingerprints of 2.132.6 and 2.134.9 differ from that build's.
    • So on main, a token with 2.132.6 or 2.134.9 installed also raises at import. This PR changes the accepted versions from 2.122.2 and 2.130.1 to 2.132.6, and the changelog now says so.
    • To make import-time drift warn instead of raise would change the certification policy, so I would keep that out of this PR.
  • 3. Released-bundle request. Fixed in 271dd5d.
    • hf_file now gets the repository type, so the URL includes datasets/.
    • A size the manifest does not record is read from X-Linked-Size.
    • The release job now warns when the request cannot be built.
    • The request test runs on both repository types.
  • 6. Forced requalification. Fixed in 8ea62c1.
    • For a pair on main's data line, preparation keeps main's regional release.
    • For a pair from another line, it passes --drop-regional-release.
    • The preparation test now also runs on the routed packaged bundle, and the runbook has a section on pairs from another line.
  • 7. Receipts for local-area runs. Fixed in 5882164, as you suggested.
  • 8. Ledger concurrency guard. Fixed in 8d76608.
    • One load_candidate_cli in tests/qualification_fixtures.py replaces the ten inline loaders.
    • The concurrency test now uses the uk-data-line bundle, which is now the 6.9.0 bundle, so its promotion assertion can fail.
  • 9. Catch-up re-dispatch. Fixed in 8d76608.
    • recoverable skips unforced attempts whose pair is not on main's data line, with a notice.
    • refresh-pr-checks skips receipts from another line.
  • 10. Shared cache path. Fixed in e1ae884. Nested artifacts are now cached under their release path. Top-level files, which include every H5, keep their cache path.
  • 14. Whole local-area file per area run. Not in this PR. The fix (a per-process dataset cache, and caching simulations without their input dataset) changes runtime caching. It matters once sim-api serves UK areas from policyengine.py, which UK: keep constituency and local-authority results available through a switch to Microcosm, then serve them from its local-area dataset policyengine-sim-api#728 covers. I would do it as its own PR.
  • 15. UC-floor exception. Recorded in the data certification skill and in the changelog (02ec73e).

@juaristi22
juaristi22 marked this pull request as ready for review October 20, 2026 21:11

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 02ec73e.

First review:

  • Fixed (12 of 15): findings 1, 3–10, 12, 13 and 15. I checked each fix.
  • 2: agreed. Main is just as strict, since the eFRS claims are exact, so no change is needed.
  • 11 and 14: deferred, as you said.

New: 15 findings, most severe first. 14 are inline on lines in the diff. The rest are below, because they sit in code the diff doesn't touch.

  1. The new core check freezes the shared core pin while it binds, and stops binding after a re-certification done the skill's way. Inline on generate_trace_tros.py:151.

  2. Constituency and local-authority results can no longer carry a certified receipt. receipt.py#L3882

    • A receipt is certified only when the run used certified_data_artifact.dataset with its bytes (receipt.py:3878-3888; docs/receipts.md:13), and the certificate names one dataset, the national file.
    • certify-data does vet the local-area release. It checks the role, the single H5, and that the release's own claims cover the model and core, and it pins the file's bytes in datasets. But none of that becomes part of the certificate, and the TRO doesn't cover the file either.
    • It never showed before:
      • receipts are UK-only, so US state and district runs never had them;
      • until this PR the UK bundle routed no area regions, so area outputs ran on the certified national eFRS.
    • Now compute_uk_geography_disclosure(levels=("constituency",)) has to run on microcosm_uk_2024_25_dense, and its receipt is uncertified. The new test at tests/test_execution_receipt.py:413-427 pins that.
    • docs/regions.md:107 calls the file "a certified local-area release", and nothing else says so.

    Either:

    • document that area results carry uncertified receipts (changelog, regions.md, receipts.md);
    • or extend the certificate and the TRO to cover the routed dataset: certify-data already has its sha256, build and manifest, and the receipt would then accept a run on that dataset.
  3. Forced cross-line attempts still loop in the catch-up. Inline on check_uk_candidate.py:553.

  4. The scorecard's own catch-up still needs size_bytes, which Microcosm manifests don't carry. Inline on uk_qualification.py:1829.

  5. The released-bundle request tests don't pin the Hugging Face request details. Inline on test_uk_qualification_pipeline.py:3379.

  6. The reason given for not running reform responsiveness doesn't hold. Inline on data-certification.md:193.

  7. Pruned overlays are gone for good, but three texts still say overlays survive re-certification. Inline on certification.py:1384.

  8. Cross-line candidate PRs never close, and their body says merging promotes the pair. Inline on uk_qualification.py:855.

  9. On Windows, the cache-path guard misses backslash and drive-letter paths. Inline on dataset_materialization.py:96.

  10. Five new tests assume the packaged bundle is a Microcosm main. Inline on test_uk_qualification_pipeline.py:2946.

  11. The fragment type and a few texts are inaccurate. Inline on certify-uk-microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71.changed.md:1.

  12. Same-line preparation can't pass a model or core change. Inline on uk_qualification.py:774.

  13. release_consumers.py re-implements the shared Hugging Face helpers, with drift. Inline on release_consumers.py:55.

  14. The overlay-shadowing rule is written three times. Inline on bundle_validation.py:27.

  15. Core drift is checked in two branches with two messages. Inline on generate_trace_tros.py:142.

Comment thread scripts/generate_trace_tros.py Outdated
raise ValueError(
f"{prefix}: runtime core differs from the qualified exact pair"
)
elif (

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. The new core check freezes the shared core pin while it binds, and stops binding after a re-certification done the skill's way.

  • Freeze. The certificate names core 3.33.3, and both Microcosm releases claim exactly policyengine-core ==3.33.3.
    • update-packages --core X (still described as a bundle-only PR in docs/bundles.md:119-128) now fails check --strict-tros in PR CI.
    • So does a US certify-data --core-version X, which moves the shared pin (certification.py:1397).
    • Re-certifying UK with --core-version X is refused for both the national and the routed dense release; the regional release gets no qualified-pair escape (certification.py:1171-1181).
    • So no core bump can land until Microcosm publishes cuts that claim the new core, including one a US release needs. The pin has moved 8 times since June, twice core-only.
  • Opt-in. certify-data records a core only when given --core-version (certification.py:1002-1003), and the skill's UK commands don't pass it. The next re-certification done the skill's way drops the guard, and receipts fall back to the pin. Three of the new receipt tests would trip, but only as bare assertion failures.

Suggest:

  • say this in docs/bundles.md, the skill and the changelog;
  • default --core-version to the pin, or add it to the skill's commands;
  • decide whether UK's exact claims should block a US core bump.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in e691df0, together with finding 15.

  • Documented: the skill now says what a named core binds. Because Microcosm claims one exact core, a core bump needs cuts that claim the new core. The other way is to re-certify UK without --core-version: receipts then certify the pin, as they did before October. docs/bundles.md notes the binding next to update-packages --core, and the changelog says it too (2993625).
  • Guard kept: the skill's UK commands now pass --core-version, so a re-certification done the skill's way keeps the guard.
  • Tests: the receipt tests now set the named core they test, and no longer assume the packaged certificate has one.
  • Still open: whether UK's exact claims should block a US core bump. Until the next Microcosm pair claims its own core, re-certifying UK without --core-version is the way to move the core.

Comment thread scripts/check_uk_candidate.py Outdated
candidate = read_json(attempt / "candidate.json")
# An unforced run of a pair from another data line resolves as
# other_line before any recovery, so dispatching it clears nothing.
if not candidate.get("force") and (

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. Forced cross-line attempts still loop in the catch-up.

recoverable skips only unforced off-line attempts, but the catch-up re-dispatches what it prints with -f force=false (certify-uk-release.yaml:464). candidate_action then returns other_line before reaching the recovery branch (uk_qualification.py:423-434).

  • After the switch every new uk-data attempt is forced, because resolve only builds uk-data pairs.
  • A forced run "for evidence" that is cancelled, times out or finalizes incomplete therefore stays recoverable for good once its lease passes. It is re-dispatched every 2 hours.
  • Simulated through the CLI: 11 dispatches over 2 days, with the ledger unchanged. Once uk-data's main moves, each re-dispatch also fails resolve with a mismatch.
  • The new test asserts the forced attempt is printed (test_uk_qualification_pipeline.py:2972-2973), while the runbook says recovery skips these pairs.
  • The notice also fires before attempt_is_recoverable, so attempts that are still live log "not recovering" every 2 hours.

Suggest skipping every off-line pair, as refresh-pr-checks does (or passing force through), with the check after attempt_is_recoverable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 741b269.

  • Recovery now skips every pair from another data line, forced or not.
  • The skip comes after the recoverability check, so live attempts log nothing.
  • The test covers both cases.
  • The runbook says to re-run a forced cross-line run by hand.

)
data_repo_id = reference.get("repo_id") or data_package["repo_id"]
data_repo_type = reference.get("repo_type") or repo_type
size_bytes = artifact.size_bytes

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4. The scorecard's own catch-up still needs size_bytes, which Microcosm manifests don't carry.

The X-Linked-Size fallback covers only this push path.

  • When that request fails, which is now only a warning, the scorecard's scheduled discovery reads released["size_bytes"] from the release manifest (pipeline/uk_replay_discover.py:471).
  • Microcosm's assemblers never write it (tools/assemble_uk_release_dir.py:462-469 and the dense one), and the pinned national manifest has none.
  • So discovery raises KeyError, logs "not requested" and retries on every run.

A one-line size_bytes in each assembler fixes both paths for future cuts. For this pinned cut, only a scorecard-side fallback works.

Worth checking: the scorecard's docs describe its Hugging Face token as reading policyengine-uk-data-private only. If so, it can't fetch populace-uk-private data even through the fixed push path.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Both fixes belong outside this repository, and the PR body now lists them as follow-ups.

  • Microcosm's release assemblers should write size_bytes. That fixes both paths for future cuts.
  • For the pinned cut, the scorecard needs a fallback of its own. It also needs read access to policyengine/populace-uk-private for its Hugging Face token. I have not checked that access yet.

Comment thread tests/test_uk_qualification_pipeline.py Outdated
repo_type = record["data_package"]["repo_type"]
fetched = []

def hf_file(repo, rev, path, kind):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5. The released-bundle request tests don't pin the Hugging Face request details.

The stubs record only repo and type, the size call's arguments are never asserted, and the release_consumers HEAD stub ignores the URL.

In a mutation run, all four cases still passed after each of these changes:

  • hard-coding the model namespace in _hf_file or _hf_size, which is item 3 of the first review;
  • sending the size HEAD to the wrong repo, revision, path or type;
  • dropping the Authorization header;
  • not passing hf_size.

Suggest asserting the full (repo, revision, path, repo_type) for both calls, and the URL and headers the script builds.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e40d3ba.

  • Request test: it now asserts the full repository, revision, path and repository type of both Hugging Face calls.
  • Release-consumer test: it asserts:
    • the URLs, quoted;
    • the Authorization header, which now also comes from HUGGINGFACE_HUB_TOKEN;
    • each HEAD answer the size reader handles: a redirect with X-Linked-Size, a direct 200, and a redirect without a size that it follows.

weighted national totals only. Before quoting any other figure from UK data
weighted national totals only.

**Recorded exception (policyengine.py#685, October 2026).** The Microcosm UK

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6. The reason given for not running reform responsiveness doesn't hold.

The PR body says metrics review, reform responsiveness and replay "measure a candidate against the previous certified pair". That's true for metrics review and replay. It isn't for reform responsiveness:

  • reform_responsiveness(report, standard_reforms, years) (uk_candidate.py:2062-2068) takes no previous report;
  • gate_responsiveness never reads the previous certificate. The gate checks the candidate's own reform responses against fixed minima.

So it could run on the national cut, and it matters here: the PR's known gaps include a rate schedule this cut has no carriers for (the SDLT higher rates).

Suggest running it and reporting pass or fail. Otherwise, record an acceptance for the comparison gates next to the UC-floor exception here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: reform responsiveness checks the candidate against fixed minima, not against the previous pair. I ran it on the national cut at 2.132.6, and the result is needs review.

  • Nine of the ten standard reforms pass in every year.
  • For the tenth, the tool's own check passes. But the tool's small-cell rule withholds that reform's figures in every year, so the gate cannot recompute them.
  • A disclosure-control item on that reform went to the author separately.

The skill now records the acceptance of the two comparison gates, the metrics review and the replay, next to the UC-floor exception (6bed939). The PR body is corrected.

@@ -0,0 +1 @@
Switch the UK default dataset from the Enhanced FRS 2024-25 (policyengine-uk-data 1.58.1) to Microcosm UK 2024-25 (`microcosm_uk_2024_25`, cut `microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71`), certified with policyengine-uk 2.132.6 and policyengine-core 3.33.3, the versions it was built with. Constituency and local-authority regions now load the Microcosm local-area release (`microcosm_uk_2024_25_dense`, cut `microcosm-uk-2024-25-dense-20261019T000432Z-ee5c2dbe`) and are in the default UK region registry. UK results change with the data. The national file has no constituency or local-authority codes, so constituency and local-authority outputs (`compute_uk_geography_disclosure` at those levels, `compute_uk_constituency_impacts`, `compute_uk_local_authority_impacts`) need simulations on `microcosm_uk_2024_25_dense`. The Enhanced FRS 2024-25 datasets stay loadable by name, but its weight matrices (`parliamentary_constituency_weights`, `local_authority_weights`, `local_geography_weights`) no longer resolve. With a Hugging Face token set, importing policyengine with any policyengine-uk other than 2.132.6 raises, as it did before for versions the Enhanced FRS release did not claim. The certified default is below the Universal Credit caseload floor (0.897 of DWP's benchmark), an exception accepted and recorded in the data certification skill.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11. The fragment type and a few texts are inaccurate.

  • Fragment type. It says the weight matrices "no longer resolve", which is a removal. bump_version.py:86-88 makes removed a minor bump, so as typed this ships as 6.9.1. Earlier default switches shipped as patches, so this is a judgment call.
  • "0.897 of DWP's benchmark". The PR body calls the 6.7m benchmark policyengine-uk-data's derivation. DWP's Stat-Xplore figure, which Microcosm targets, is 6.2m.
  • Stale texts.
    • The uk_qualification module docstring still says prepare always regenerates the test constants.
    • test_execution_receipt.py:335-336 says receipts from 6.5.0 to 6.8.0 name no core. 6.9.0's don't either.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2993625 and 741b269.

  • Fragment type: the weight matrices moved to a removed fragment, so this ships as a minor release. To ship it as a patch instead, fold that fragment back into the changed one.
  • The 0.897: it now reads "of the floor's 6.7m benchmark, policyengine-uk-data's derivation", with Stat-Xplore's 6.2m beside it.
  • Stale texts: both are corrected. The module docstring is fixed in 741b269, and the receipt-test comment in e691df0.

return []
if data_line(pair) != certified_data_line(bundle):
return ["--drop-regional-release"]
return [

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

12. Same-line preparation can't pass a model or core change.

It keeps main's dense release, which claims exactly ==2.132.6 / ==3.33.3, and the regional release gets no qualified-pair escape.

A probe with a same-line pair:

  • 2.132.6 / 3.33.3 prepares;
  • 2.140.0, or core 3.34.0, is refused.

It's unreachable today, but the first automated Microcosm qualification would fail here, and only the docstring says so.

Suggest carrying a regional cut in the candidate, or refusing same-line routed preparation up front with a clear message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3b2db90. Preparation now refuses a same-line pair up front when its model or core differs from what main's local-area cut was certified for. The message says that a local-area cut for those versions is needed.

Comment thread scripts/release_consumers.py Outdated


def _hf_file(repo_id: str, revision: str, path: str) -> bytes:
HF_REPO_PREFIXES = {"model": "", "dataset": "datasets/", "space": "spaces/"}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

13. release_consumers.py re-implements the shared Hugging Face helpers, with drift.

HF_REPO_PREFIXES, _hf_url and _hf_headers duplicate https_dataset_uri and hugging_face_auth_headers in manifest.py. The copies differ in three ways:

  • Token. With only HUGGINGFACE_HUB_TOKEN set (listed in this script's own TOKEN_VARIABLES), both requests go out unauthenticated. CI sets HUGGING_FACE_TOKEN, so only manual runs are affected.
  • Quoting. The copy skips the quoting the shared helper applies.
  • Small-file branch. The _hf_size docstring says a small file "answers directly with its length", but Hugging Face answers HEAD for a non-LFS file with a relative 307 and no X-Linked-Size. So the Content-Length branch never runs. That is harmless for the LFS default H5.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e40d3ba. The script now uses https_dataset_uri and hugging_face_auth_headers.

On the small-file branch: at a tag revision, Hugging Face answered HEAD for the 4 KB release manifest with a 200 and its Content-Length, which I checked, so that branch does run. _hf_size now also follows a redirect that carries no size.

)


def validate_dataset_overlays(bundle: Mapping[str, Any]) -> None:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

14. The overlay-shadowing rule is written three times.

The three copies:

  • the runtime _apply_dataset_overlays;
  • this validator;
  • the pruning in certify_data_release.

They share no code. The certify copy's or name == data_release["default_dataset"] can never fire, because the default is always in datasets.

One helper here, which runtime code and scripts already import, would keep "certify output passes check" true by construction.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 90172bd. shadowed_dataset_overlays states the rule once. The runtime, the validator and certification all use it, and the default check that could never fire is gone.

Comment thread scripts/generate_trace_tros.py Outdated
prefix = f"{country.country_id}: strict TRACE release"
certification = country.certification
core = bundle.get("packages", {}).get("policyengine-core", {})
if certification and certification.compatibility_basis == "qualified_exact_pair":

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15. Core drift is checked in two branches with two messages.

The exact-pair branch and the new elif compare the same pin and named core. The exact-pair message has no versions and no remedy.

One unconditional comparison, plus the exact-pair check that it names a pair and a core, raises on the same inputs (checked over 72 combinations). It would give every basis one message, and one test instead of two.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e691df0. One comparison now covers every basis and gives the versions and the remedy. The exact-pair branch only checks that the pair names its pair and its core. The exact-pair test now matches the shared message.

Maria Juaristi and others added 8 commits October 20, 2026 23:33
The strict check compared the core pin with the certificate's core in
two branches, with two messages. One comparison now covers every basis
and names the versions and the remedy; the exact-pair branch only checks
that the pair names its pair and core.

A named core binds the core pin, and Microcosm UK releases claim one
exact core, so a core bump now needs Microcosm cuts that claim the new
core, or a UK re-certification without --core-version. The skill and
docs/bundles.md say so, and the skill's UK commands pass --core-version
so the binding survives a re-certification done its way. The receipt
tests set the named core they test instead of assuming the packaged
certificate has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule that an overlay may not shadow a certified name was written
three times: at runtime, in the maintenance validator, and in the
pruning certification does. shadowed_dataset_overlays in the stdlib
validator now states it, and all three use it.

A pruned overlay does not come back, so a later certification that
stops certifying its name (rolling forward after a rollback, or a
ship-together run after a national-first one) dropped the name
silently. Certification now warns about every name that stops
resolving. The texts that said overlays always survive re-certification
now say which ones do, and the skill has a re-add step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The catch-up re-dispatches recoverable attempts without force, so even a
forced run of a uk-data pair resolved as other_line and cleared nothing:
it was re-dispatched every two hours for good. Recovery now skips every
pair from another line, after the recoverability check so that live
attempts log nothing.

A forced cross-line run opens a candidate pull request that can never be
promoted, and its body said merging promotes the pair. Promotion now
closes the open pull requests of pairs from another line (the ledger
keeps their evidence), and such a PR says it is never promoted. The
runbook no longer says an unforced run is recorded.

Tests that need a routed Microcosm main now read a frozen copy of the
bundle this PR certifies, so a rollback of the packaged bundle does not
break them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
release_consumers.py had its own URL and header helpers, which missed
HUGGINGFACE_HUB_TOKEN and the shared helper's quoting. It now uses
https_dataset_uri and hugging_face_auth_headers. _hf_size reads a small
file's Content-Length directly, and follows a redirect that carries no
size.

The released-bundle request test now asserts the full repository,
revision, path and repository type of both Hugging Face calls, and the
release-consumer test asserts the URLs, the Authorization header and
each HEAD answer the size reader handles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cache-path guard validated with PurePosixPath but joined with the
platform Path, so on Windows a backslash, drive-letter or UNC path
passed the check and landed outside the data directory. Repository paths
use forward slashes, so the guard now also refuses backslashes, drives
and roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preparation keeps main's regional release for a pair on main's data
line, but that release was certified for main's model and core and gets
no qualified-pair escape, so certification refused any other model or
core deep inside certify-data. Preparation now refuses such a pair up
front and says a local-area cut for its versions is needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The weight matrices that no longer resolve by name are a removal, so
they move to their own removed fragment, which makes the release a minor
one; the switch itself stays a change. The change fragment now says that
receipts for constituency and local-authority runs are uncertified,
since the certificate covers the national dataset only, that the named
core binds the core pin, and that 0.897 is against the floor's 6.7m
benchmark, not DWP's 6.2m. regions.md and receipts.md say what
certification does and does not cover for the local-area release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The metrics review and the scorecard replay measure a candidate against
the previous certified pair, which a change of data line makes
meaningless, so they were not run for the Microcosm switch. The skill
now records that acceptance beside the UC-floor exception. Reform
responsiveness checks the candidate against fixed minima, so it was run
on the national cut. The UKDS output rule is its own paragraph again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Contributor Author

Addressed at 6bed939. All 14 inline findings have replies in their threads. This comment covers the rest.

  • 2. Area receipts. Documented in 2993625:
    • The changelog, regions.md and receipts.md now say that runs on the local-area file get uncertified receipts, because the certificate covers the national dataset only.
    • regions.md no longer calls the file "certified".
    • Extending the certificate and the TRO to the routed dataset is listed as a follow-up in the PR body.
  • Reform responsiveness: it now runs. The result is in the reply on data-certification.md.
  • Still open: whether UK's exact core claim should block a US core bump. For now, the skill gives the way out: re-certify UK without --core-version.

The branch is still on main 5f97905. check --published-spm --include-tros --strict-tros and release_lock.py pass, and the touched test modules pass in the order CI runs them.

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 6bed939.

Round 2:

  • 13 of the 15 findings are fixed. I checked each fix.
  • Finding 1 stays as designed: the escape is kept so that UK and US cores need not move together. The escape is re-certifying without --core-version, after which receipts certify the pin.
  • Finding 4 is the cross-repo follow-up listed in the PR body.

New: 14 findings, most severe first. 14 are inline on lines in the diff.

  1. Core binding: three text fixes. Inline on generate_trace_tros.py:163.

  2. Promotion closes an in-flight off-line PR, and never closes one without a receipt. Inline on uk_qualification.py:1634.

  3. Promote returns PRs it just closed to draft. Inline on check_uk_candidate.py:992.

  4. No recorded acceptance for the two needs-review gates. Inline on data-certification.md:227.

  5. The path guard misses a drive letter in a later segment. Inline on dataset_materialization.py:98.

  6. The redirect branch can't get a size from Hugging Face. Inline on release_consumers.py:86.

  7. The same-line check compares with main's certificate, not with the local-area release's claims. Inline on uk_qualification.py:779.

  8. The request test still doesn't pin the per-dataset override. Inline on test_uk_qualification_pipeline.py:3452.

  9. Nothing records that promotion now closes other-line PRs. Inline on certify-uk-microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71.changed.md:1.

  10. Stale text. Inline on uk_qualification.py:1752.

  11. The new closing paths are barely tested. Inline on uk_qualification.py:1644.

  12. The off-line close loop copies the superseded one. Inline on check_uk_candidate.py:977.

  13. The overlay refusal has three wordings. Inline on manifest.py:334.

  14. on_main_line defaults to True. Inline on uk_qualification.py:1722.

Comment thread scripts/generate_trace_tros.py Outdated
raise ValueError(
f"{prefix}: runtime core {core.get('version')} differs from the "
f"certified core {certification.certified_for_core_version}; "
"re-certify the data release for the new core, or without "

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. Core binding: three text fixes.

The escape stays as designed. Three texts around it are off:

  • Exact pairs. This message offers "or without --core-version" for every basis. For a qualified_exact_pair certificate, certify-data refuses that ("A qualified exact pair names its core version", certification.py:327-332). Dropping the pair id as well is also refused, unless a publisher claim covers the model. Suggest offering it only for the other bases.
  • Skill commands. The skill's main UK command (data-certification.md:25-26) and the UK example in docs/bundles.md:138-141 omit --core-version, while the other skill commands pass it. Say which is the default, and when to bind the core.
  • Wording.
    • regions.md:109-110 says certification checks a local-area release's own core claim. It does that only with --core-version.
    • bundles.md:131 cites check --strict-tros, which also needs --include-tros.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6ee9157. The rule itself also changed, per María's ruling: the UK certificate now follows the pin, and says so.

  • Default: without --core-version a certificate records core_binding: bundle_pin and names no core. Certification and the strict check warn, but do not refuse, when the pin is outside a release's claims. The TRO records the binding.
  • Binding: --core-version (core_binding: certified) names and binds the core, and the strict check refuses drift. Its remedy now offers "without --core-version" only for bases other than an exact pair. An exact pair is told to requalify.
  • Docs:
    • The skill's UK commands drop --core-version.
    • A new "The core" section says which mode is the default and when to bind.
    • regions.md and bundles.md say the same, with --include-tros.
  • UK: re-certified this way, so it warns about nothing, because both releases claim the pin.
  • US: the check now shows that the US release claims only 3.32.5. The pin was already past that on main.

# ledger keeps its evidence.
receipts = list(receipts)
line = certified_data_line(bundle)
off_line = sorted(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. Promotion closes an in-flight off-line PR, and never closes one without a receipt.

  • close_off_line takes every open branch whose off-line pair has a published receipt. It never checks that the PR head is that receipt's commit; the target path does check (line 1696).
  • In-flight close. The runbook and the recovery notice both say to re-run an off-line pair with force. When that pair already has a receipt:
    • The re-run pushes a new commit and opens a new draft PR.
    • Its gates take hours. Any promote job in that time closes the PR: the cron runs every 2 hours, promote runs if: always(), and it shares no lock with qualify.
    • The verdict step then finds no PR and exits 1.
  • Never closed. An off-line attempt whose replay job dies before publishing keeps its PR open for good: recovery skips it, and promotion only sees receipts.

Suggest any of:

  • close only when the open head equals receipt.harness.git_commit, or when no newer attempt is live;
  • close receipt-less off-line PRs once their attempt is no longer live;
  • let the verdict step tolerate a closed PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78 and 12cfbc0.

  • Closing: an other-line PR now closes only when its open head is the commit its receipt qualified. A forced re-run in flight at a newer head is left to its own receipt.
  • Dead attempts: the PR of an attempt that dies before publishing a receipt stays open, and the runbook now says to close it by hand.

],
check=False,
)
# Only the plan's target may be ready, and only while _promote finds it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. Promote returns PRs it just closed to draft.

  • The draft sweep below works from the PR listing taken before the closes.
  • So a non-draft PR that was just closed gets gh pr ready N --undo. gh refuses that for a closed PR, before any API call. The run then prints "could not return N to draft" and exits 1.
  • It needs a PR left ready, by a person or by an interrupted run. So it is rare with automatic promotion, and the superseded case already had it.
  • gh pr close failures are also ignored (check=False).

Suggest skipping closed branches in the sweep (or un-drafting before closing), and warning when a close fails.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78. One _close_pull_requests helper closes the PRs, warns when a close fails, and returns the PRs it closed. The draft sweep skips those.

- **Comparison gates.** The metrics review and the scorecard replay measure a
candidate against the previous certified pair, and a change of data line
moves the totals by design, so they were not run for that switch; its pull
request compares the two lines' weighted totals instead. The other gates,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4. No recorded acceptance for the two needs-review gates.

  • uk.json:38 requires a named acceptance for a needs-review receipt, and the UC-floor entry here records one.
  • Reform responsiveness and budget-critical inputs both came back needs review. Neither this list nor the PR body records an acceptance, and "were run" reads as a pass.

Suggest one line for each gate. For example: "Reform responsiveness: needs review. The metrics tool's small-cell rule withholds a standard reform's figures, so the gate cannot recompute its verdict. Accepted by in #685."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recorded in 0f5e05b. Both acceptances are in the skill, next to the UC-floor exception, and in the PR body.

  • Inputs: the line gives the reason. The enhanced FRS column carries no values, because Incapacity Benefit is closed, so both datasets give zero.
  • Responsiveness: the line names no reform and gives no counts.

relative = PurePosixPath(path)
# Repository paths use forward slashes. A backslash, drive or root would
# make the platform path leave data_dir on Windows.
windows = PureWindowsPath(path)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5. The path guard misses a drive letter in a later segment.

  • PureWindowsPath(path).drive and .root look only at the first segment, and the join then restarts at a later drive.
  • So on Windows, releases/D:x.h5 or a/b/E:/x.h5 lands outside data_dir. With the default ./data, any drive letter does.
  • A drive in a middle segment stayed inside under main's basename code, so that part is new.

Suggest checking the joined path (destination.is_relative_to(data_dir) and destination != data_dir, as verify._resolve_local does), or refusing any segment that has a drive. Add these shapes to the test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1fda069.

  • The guard now refuses a drive or root in any segment.
  • It also checks that the joined path stays inside the data directory.
  • The test has your two shapes, plus an empty path.

verdict = receipt.verdict if receipt else "pending"
if on_main_line:
outcome = (
"Merging this PR promotes the pair. Until cos d1344 is ruled, "

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10. Stale text.

"Until cos d1344 is ruled, promotion is a reviewed merge": uk.json records d1344 as ruled on 15 October, with automatic promotion on. The sentence moved unchanged into the new branch. Suggest dropping the clause.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78: the clause is gone.

)
receipts = _same_line_receipts(receipts, bundle)

def plan(**fields: Any) -> PromotionPlan:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11. The new closing paths are barely tested.

  • Only the no-target plan asserts close_off_line.
  • No test mixes a same-line target with an off-line open PR, runs promote --apply on a Microcosm main, or runs finalize --pr-body for an off-line pair.
  • So deleting the close loop, or forcing on_main_line=True, still passes the suite.
  • The plan(**fields) closure also means a future direct return PromotionPlan(...) would drop the closes. Computing the off-line set once, shared with _same_line_receipts, would avoid that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78.

  • New tests:
    • a same-line target beside an other-line PR, which is closed only at its receipt commit;
    • promote --apply on a Microcosm main, where the PR is closed and not returned to draft;
    • finalize --pr-body for an other-line pair.
  • Refactor: the plan now splits receipts by data line once, and the same-line plan returns plain PromotionPlans, so no return can drop the closes.

Comment thread scripts/check_uk_candidate.py Outdated
],
check=False,
)
for branch in plan.close_off_line:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

12. The off-line close loop copies the superseded one.

It copies the loop above it, including the ignored exit code and a numbers.get(branch, branch) fallback that can't fire. One helper, or one loop over (branch, comment) pairs, would also be the place for finding 3's fixes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78. There is now one helper over (number, comment) pairs, and it uses numbers[branch].

Comment thread src/policyengine/provenance/manifest.py Outdated
"additive, not overrides."
)
certified_datasets[overlay_name] = overlay_reference
if default_dataset in shadowed:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

13. The overlay refusal has three wordings.

There is one helper now, but three messages. The runtime keeps separate default and collision messages and names only the first shadowed overlay; the validator names them all. One message naming every shadowed overlay would do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1fda069. shadowing_overlays_message is now the one refusal. It names every shadowed overlay, both at runtime and in the check.

*,
ledger_url: str,
prepared: Optional[Mapping[str, Any]] = None,
on_main_line: bool = True,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

14. on_main_line defaults to True.

No caller uses the default, and it is the misleading value for a caller that forgets the flag. Suggest making it keyword-required with no default. cmd_finalize also builds the body when --pr-body is absent.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b25cb78. on_main_line is now keyword-required with no default, and finalize builds the body only when --pr-body is given.

Maria Juaristi and others added 5 commits October 21, 2026 11:57
The bundle has one core pin, shared by both countries. A certificate
that names no core follows it, so another country can move the pin, but
until now that showed only as a missing field, and nothing compared the
pin with the data release's own core claims.

A certificate now says how it covers the core: core_binding bundle_pin
(the default, no --core-version) or certified (--core-version names and
binds it). Certification checks the bundle's pin against each release's
build core and claims and warns when it is outside them; the strict TRO
check warns the same way later, and refuses only a pin that differs from
a named core. The TRO records the binding. UK is re-certified to follow
the pin: both Microcosm releases claim 3.33.3, the pin, so it warns
about nothing. The check now also shows that the US release claims only
3.32.5, which the pin has been past since before this PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… routing

- Promotion closed an other-line PR whenever its pair had a receipt,
  even while a forced re-run was in flight at a newer head, whose verdict
  step then found no PR. It now closes such a PR only at the commit its
  receipt qualified. The plan splits receipts by data line once, and the
  same-line plan returns plain PromotionPlans, so no return can drop the
  closes.
- Promote closed PRs and then tried to return them to draft, which gh
  refuses. One helper now closes PRs, warns when a close fails, and the
  draft sweep skips what it closed.
- candidate_pr_body's on_main_line has no default, finalize builds the
  body only when asked for it, and the stale cos d1344 clause is gone.
- Preparation now refuses a same-line pair on a routed main outright: the
  routing would need a local-area cut for the pair's versions, which a
  candidate does not carry, and the claims check it replaced compared
  with main's certificate rather than the cut.

New tests cover a same-line target beside an other-line PR, promote
--apply on a Microcosm main, and finalize for an other-line pair. The
frozen routed bundle is refreshed to this PR's certificate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The overlay rule had one helper but three messages, one naming only
  the first shadowed overlay. shadowing_overlays_message is now the one
  refusal, naming every shadowed overlay, at runtime and in the check.
- The cache-path guard looked for a drive only in the first segment, and
  the join restarts at a later one on Windows. It now refuses a drive or
  root in any segment and checks that the joined path stays inside the
  data directory.
- requests asks for compressed answers, which carry no Content-Length,
  so the size HEADs now ask for the identity encoding, as
  huggingface_hub does.
- The request test now has a dataset entry with its own repository,
  revision and type, and asserts both calls and the whole data block
  literally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The data certification skill records María Juaristi's acceptance of the
two needs-review gates: the missing incapacity_benefit_reported, whose
enhanced FRS column carries no values, and the reform the small-cell
rule withholds from reform responsiveness. The changelog says how
certificates now cover the core, that promotion closes other-line PRs,
and that recovery no longer re-dispatches them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Contributor Author

Addressed at 12cfbc0. All 14 findings have replies in their threads.

  • Core rule. Per María's ruling, UK certificates now follow the bundle's core pin openly (core_binding: bundle_pin), and certification and the strict check warn when a release does not claim the pin. --core-version still binds the core. The skill's new "The core" section has the rule.
  • Needs-review gates. María accepted both. The skill and the PR body record the acceptances.
  • Checks. check --published-spm --include-tros --strict-tros passes; its only warning is the US core claim, which predates this PR. release_lock.py passes, and the touched test modules pass in the order CI runs them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK: certify a Microcosm national dataset as the UK default, with separate national and local-area datasets and a corrected local-authority list

1 participant