Skip to content

fix(retention): bound active pointer reads in storage reports - #4809

Open
ScriptedAlchemy wants to merge 7 commits into
masterfrom
fix/4616-bound-code-index-pointer-report
Open

ScriptedAlchemy wants to merge 7 commits into
masterfrom
fix/4616-bound-code-index-pointer-report

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Root cause

Refs #4616 — partial OOM-safety repair, not a sealed-generation collector or full issue closure.

The code-generation retention planner reads active-code-generation-v1.json with an unbounded std::fs::read. Although its writer caps the durable pointer to 512 KiB, malformed/stale/foreign files need not obey that invariant. Both the daemon's storage-report planning and other metadata-only retention paths still deserialize this pointer before their entry/digest budgets protect the rest of the traversal. A sparse multi-GiB pointer could therefore force an unbounded allocation or block the worker if substituted with a FIFO.

Focused fix

  • Reuse the existing tracedecay-private-fs::framed_log::read_bounded file authority already depended upon by this crate: one held no-follow, nonblocking regular-file read capped at MAX_DURABLE_PUBLICATION_POINTER_BYTES_V1, with length verification before allocation and after reading.
  • Preserve typed unsafe-state refusal for oversized, malformed, empty, nonregular, and symlinked pointers. Do not invent a valid pointer or permit collection against an unknown active head.
  • Native Rust behavioral regressions exercise a normal pointer, oversized yet JSON-valid padded pointer, a sparse 15 GiB pointer, symlink-to-foreign-pointer preservation, FIFO refusal, and refusal from the real metadata-only retention planner.
  • A native production-report regression sizes 70 sealed generations (~15 GiB sparse) + 15 GiB code-index data. The targeted report must refuse an oversized pointer as retention-plan unavailable while returning exact measured store-family bytes; the same fixture then runs the full offline build_storage_report entry point with complete profile/store coverage assertions. No multi-GiB payload is allocated or copied by the test.

Verification and boundaries

Initial source commit: 23f65556a9c3e81e2e6f0bb1d78112cc596e7c64; focused assertion/format follow-up f20bf8b67db5736c4a6697c52299200d2dfa21ad; full-report coverage 69e4e17f89f199afb75223c30901e53ce2b0be8e; parent master 91c72aba7750a398dd28e83d64bf0c6d6fef4172. Published through expected-old-head, force=false fast-forward. Only two issue-specific source/test paths changed; this branch does not derive from or move the mixed held #4776 work.

Not yet a Bazel/native test pass: run bazel test //crates/tracedecay-code-index-retention:unit_test //crates/tracedecay-maintenance:unit_test --test_output=errors; check a nonzero regression count and native macOS CI. Do not use Cargo. Root/nested AGENTS.md rules and active peer ownership respected.

This covers one concrete unbounded allocation in the report/retention journey and now exercises the full offline report collector on the large sparse store. Until actual Bazel/native runs prove it, it does not close OOM acceptance, and it does not prove N>3 sealed retirement, verified deletion under owner/root substitution or sustained-writer progress. Keep #4616 open until all acceptance criteria pass on master. Mixed draft #4776 and Kev/decision-model work remain excluded from merging; release #4763 and production/infra untouched.

Reject oversized or nonregular publication pointers before allocation,
including metadata-only retirement plans used by storage reports.
Exercise oversized valid JSON, sparse large files, symlinks, FIFOs,
and a 70-seal plus 15 GiB code-index project report.

Refs #4616
@changeset-bot

changeset-bot Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f263b7c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

The oversized-pointer regression now verifies the directory inventory
budget is untouched on refusal. Align the sparse report fixture with
workspace Rust formatting conventions.

Refs #4616
The sparse 70-generation/15 GiB code-index regression now exercises
both the targeted retention-planning report and the full offline
storage-report entry point with complete size-coverage assertions.

Refs #4616
The large-store reporting regression invoked a non-existent StorageReport
method. Assert the populated FullProfileSizeV1 field instead so the native
Bazel test compiles and checks complete profile coverage.

Refs #4616
Apply the three exact Rust 2024 formatter corrections reported by
Repository gates for the bounded pointer and storage-report regressions.
Keep assertions and production logic unchanged.

Refs #4616
Repair three hosted automation_runner_test failures by supplying the
configured executable identity required by the production prefilter.
Use the same identity for the prefilter and retained runner, preserving
manual-schedule, exact-ledger, live-lock and lock-race assertions.

Add a production-prefilter regression for unconfigured refusal,
deduplication and same-occurrence recovery after binding, with zero
backend calls and no effectful terminal. Do not relax backend admission,
change timeouts, or alter the graph publication changes.

Refs #4532
devin-ai-integration[bot]

This comment was marked as resolved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant