Repository navigation
fix(graph): verify sealed retirement and preserve retry state - #4819
ScriptedAlchemy wants to merge 16 commits into
Conversation
Refs #4616 Combine the proven bounded receipt/ownership cleanup fixes with post-unlink collection-state ordering and compatible filesystem regressions. Replace the old separately reviewed PR history without rewriting either branch, and preserve all exclusion and fail-closed checks.
|
Refs #4616 Reconcile all eleven formatting deltas from the exact-head Bazel repository-gates failure without modifying retirement behavior or tests.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Reviewed commit 65bd98f against its parent and the real sealed-build call path. The pre-change missing/malformed receipt branch could delete foreign real-directory contents; the revised path refuses those entries and tests negative preservation plus subsequent legitimate recovery. The pre-existing sealed-root symlink test similarly proves no foreign staged writes and positive recovery. Scope is one graph-db source file, no CI weakening. This is partial #4616 safety work only: parent identity and post-check pathname races remain unresolved; Bazel/native verification is pending, so this is not an approval or merge recommendation.
Repair three hosted automation_runner_test failures by supplying the configured executable identity required by the production prefilter. Use the same identity for the prefilter and retained runner, preserving manual-schedule, exact-ledger, live-lock and lock-race assertions. Add a production-prefilter regression for unconfigured refusal, deduplication and same-occurrence recovery after binding, with zero backend calls and no effectful terminal. Do not relax backend admission, change timeouts, or alter the graph publication changes. Refs #4532
Refs #4616 Use the strict generation/digest-bound retirement authority on failed recovery adoption and post-install reopen failures. Preserve unverified artifacts instead of path-deleting them. Cover missing, corrupt, wrong- digest and owned receipts plus unrelated sibling preservation.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Reviewed the exact 882d642 diff against its predecessor and the production sealed build/install path. Previously, std::fs::rename(staging, generation) could silently replace an empty existing real directory on Linux/macOS. The new production call uses the existing capability-relative rename_noreplace authority via a no-follow sealed-root/staging helper. The native regression constructs exactly that empty occupied destination, proves refusal and staging preservation, then proves successful installation after the destination is removed. A separate filesystem regression proves that a symlinked sealed root or staging entry cannot redirect publication into a foreign tree. Two source files changed; no test or CI gate was weakened.
Remaining safety boundary: ambient store ancestors are not yet pinned to the original store-owner identity, and staging writes remain pathname-based. A real directory substituted between staging creation and subsequent writes or between the staging handle check and rename requires a separate owner-bound I/O fix. This is a partial Refs #4616 fix only, not an approval to merge/close. Hosted exact-head CI is pending; no Bazel/native macOS pass claimed.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — reviewed exact commit e2f3371c against its parent and the real recovery path (open_sealed_generation_store_if_present -> discard_failed_sealed_adoption -> ownership-checked retirement). The previous restart_recovery_adopts_or_discards_the_on_disk_artifact assertion demanded removal of a tampered/unverified receipt, contradicting the new fail-closed ownership contract and causing a native regression failure once this PR's safety code executes. The repaired test checks all three outcomes of the real recovery journey: no sealed adoption, byte-for-byte preservation of the unverified on-disk receipt/directory, and successful reads from staging rows. It still proves positive sealed adoption before tampering. One test file, no behavior or CI weakening; exact-head nonforce FF verified. This is partial, not acceptance of all #4616 or approval to merge. Bazel/native macOS results remain outstanding; CI run 42540628881 is queued.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — reviewed exact commit b568770d against its parent and build_or_open_sealed_store's production stage/create/install path. The previous unconditional remove_child_directory_nofollow was capability-relative but authenticated no ownership: a pre-existing real .staging-<generation> directory was recursively erased before its contents were examined. The new exclusive create_dir refuses occupied staging (including symlinks), with a native regression that checks byte-exact foreign-sentinel preservation, no artifact installation, and successful verified sealing after fixture cleanup. Only sealed_store.rs changed; no gate, timeout, release, or other PR was modified.
This is a narrowly reviewed safety repair, not approval for full #4616 acceptance. Outstanding separately: the store-open abandoned-staging sweeper still deletes by filename pattern without an ownership receipt, and store-ancestor/staging-path substitution remains unsafe without a stable owner capability. This head has no Bazel/native macOS pass. Confirm exact-head CI and nonvacuous Bazel graph-db tests before any merge.
Refs #4616 Do not treat a staging-shaped name as deletion authority. Retire only staged children with bounded matching sealed-generation receipts through held no-follow capabilities, preserving unknown or substituted contents. Exercise real store reopen for owned cleanup and unverified preservation.
Refs #4616 A copied receipt with rewritten physical namespace must not authorize recursive cleanup of an unrelated staging directory. Exercise the forgery and malformed-receipt refusals through real store reopen.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independently reviewed the exact two-commit staging-sweep repair 5de9d92c..0bf8a868 against the prior b568770d and the actual store-open/build/receipt paths.
Previously sweep_abandoned_sealed_staging handed every filename-shaped .staging-<hex> child to a no-follow receipt-free recursive remover. A foreign real directory with an arbitrary payload was therefore deleted on store open. The change reuses the existing held-directory, no-follow, bounded-receipt collector; it rejects unreceipted/malformed/oversized/linked entries and binds the receipt's logical namespace + projection + generation to its canonical physical namespace before authorizing deletion. An interrupted seal with a matching real receipt still reclaims. No changes to store admission, CI, model work or other branches.
The real graph-store reopen regression exercises positive reclamation plus preservation of exact foreign payloads, missing receipt, wrong physical name, forged/copied receipt, malformed/oversized receipt, symlinked directory and installed serving continuity. This is behavioral evidence in the source tree, not an executed test result. CI is queued at https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42552653839; Bazel/native results remain unproven. Remaining #4616 root risks include trusted ancestor identity before capability acquisition, path substitution during staging writes, unrecoverable pre-receipt interrupted staging, N>3 end-to-end retirement and master acceptance. This review is a scope/safety review, not approval to merge.
Keep accepted master tests and isolate sealed-generation retirement changes. Refs #4616
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Refs #4616 — independently reviewed the published bd4f39eb regression correction and the accepted master synchronization 95e8a8b6.
Root cause: the previous direct_build_recovers_from_interruption_and_matches_the_staged_build test created an unreceipted occupied .staging-<generation> containing a torn container and required the builder to erase it and rebuild. The production builder now deliberately refuses that name collision (ownership cannot be established), so that pre-existing test was both failing and demanding unsafe behavior.
The revised native test asserts the precise GraphDbError::Unavailable refusal, byte-exact preservation of the torn file, absence of an installed generation, then removes the test fixture and proves direct/staged/parallel payload equivalence and successful install. No production guard is relaxed.
The merge commit has two exact parents: existing PR head bd4f39eb and accepted master 83694e00. The resulting tree is byte-identical to its PR parent; master comparison now contains only six graph-db/private-fs files, with the previously overlapping automation-runner test file absent. Both updates were nonforce expected-old-head fast-forwards. This review is source-level evidence, not a Bazel/native-test pass. Exact-head CI run 42554042152 is queued; do not merge until CI and independent acceptance are proven.
Root cause and scope
Refs #4616 — partial ownership/OOM safety and collection-state repair only. This is a nonforce, clean-history replacement for #4804 and #4818, which could not be combined for CI because #4804 contains an immutable >72-character commit header and its Linux graph-db test disagrees with the independent strict-retirement error/receipt behavior. The original branches and commits remain intact.
Exact provenance / scope
Single commit
c8a1effe74cbbc15ae8fcf754cc5817924b58310, based onmaster91c72aba7750a398dd28e83d64bf0c6d6fef4172. Five paths only: the four changed by #4804 pluscrates/tracedecay-graph-db/src/generation_runtime.rsfrom #4818. The branch was created at the exact base and advanced usingexpected_shawithforce=false; neither original head was rewritten. No changes from the held mixed draft #4776, Kev work, release #4763, other issue owners, production or infrastructure.Validation and gaps
No local Bazel/native macOS run is claimed (no checked-out workspace or Bazel in this execution environment). Require hosted repository/commit gates,
bazel test //crates/tracedecay-graph-db:unit_test //crates/tracedecay-graph-db:graph_db_suite //crates/tracedecay-private-fs:unit_test --test_output=errorsas applicable to available targets, explicit nonzero test execution, and independent source review before merge. CI on the old #4804 head failed repository commit messages and a graph-db unit test; this clean-history joint branch addresses both reasons but does not establish a pass before CI runs.Remaining #4616 acceptance: complete N>3 end-to-end sealed retirement under actual retention and busy writers, stable owner identity before directory capability acquisition, integration with unrelated unsafe artifacts (#4752 is merged), Doctor reclaim (#4799), and large-store report full/partial memory behavior (#4809), all proven on master. Keep #4616 open. Kev/decision-model PRs and mixed #4776 must remain draft and never merge/auto-merge; release #4763 read only.
Review verification
At
e2f3371c, the macOS Bazel sealed-store contract tests passed 22 cases, including the restart case raised in review. It exercised successful verified adoption, refusal to adopt the altered receipt, preservation of its bytes, and successful reads from staging. One existing opt-in case was skipped. Full incident-scale acceptance for #4616 remains open.