Skip to content

fix(storage): stream growth measurements and preserve registry errors - #4794

Open
ScriptedAlchemy wants to merge 32 commits into
masterfrom
fix/4099-report-registry-presence-20261107
Open

ScriptedAlchemy wants to merge 32 commits into
masterfrom
fix/4099-report-registry-presence-20261107

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Scope

Refs #4099. This remains an issue-specific partial repair, not a claim that the deleted 153 GB profile has been reconstructed or that the 24-hour master acceptance has passed.

Measurement-path root cause and repair

The actual reader-worker TableSizes command joined one dbstat row per database page to sqlite_schema, then used GROUP BY to recover one total per table. That creates a temporary grouping sorter proportional to the number of pages being measured and consumes avoidable SQL VM work inside the existing measurement deadline. Large-store measurement was doing expensive page sorting in addition to the necessary page walk.

The current reader initializes one total per user table from sqlite_schema, then walks SELECT name, payload FROM dbstat once on the same snapshot. It includes empty and rootless user tables, excludes index and internal pages from their totals, and returns an interruption or arithmetic error before publishing results. The earlier correlated per-table SUM at 8b403d1 was an intermediate implementation.

The existing registry fixes are preserved: the offline report distinguishes true global.db absence from metadata/read errors and refuses registry symlinks rather than importing foreign registration identities into a profile whose census cannot account for those linked bytes.

Behavioral regressions

Three new filesystem-backed tests call the production census implementation:

  • Exact equality with an independent per-page reference on nonzero overflow payloads, WITHOUT ROWID storage, an empty table, and a rootless virtual table. Index/internal-table bytes are not misattributed. A subsequent insert raises the live measurement; deletion lowers it while physical page count stays high; vacuum of only the disposable test fixture lowers page count without changing live payload totals.
  • The same nonzero fixture measures actual VM work for both implementations and requires the production census to use less than half the old grouped query's work. Restoring the old query makes this regression fail. No wall-clock threshold or source/query-plan string assertion is used.
  • An interrupted real SQLite census returns an error, never partial/zero success, and the same connection successfully retries with the exact nonzero sizes.

The earlier dangling/valid-external-registry behavioral tests remain in the PR.

Validation and review

  • A separate executable SQL reproduction using SQLite 3.46.1 returned identical payload totals on a 1,040-page fixture: 19,043 VM steps before, 6,577 after. Its events table contained 4,194,384 measured payload bytes. Deletion/compaction accounting and interruption/retry checks passed in this reproduction. These are SQL mechanism results, not a repository Rust test pass.
  • Original worker source was materialized from the immutable GitHub revision and matched blob 9edee1472e796e6bde5bc1564b49a452c56e75d0 before editing. Published blobs match the locally inspected source. Comparison against prior head 43cef92 shows only the production dispatch change and the new census/test module; the concurrent formatting fix is preserved.
  • Published through a non-force fast-forward with exact expected old head 43cef92bc121f1bf9b451de1a3b92c8e40fd624f.
  • This runner has no Bazel/toolchain or full checkout, so no local Rust compilation, formatter or Rust test pass is claimed. Hosted Repository gates on 8b403d1 found only pinned-rustfmt differences in the new test module; the exact reported formatting corrections were published as d9559c8 using a non-force fast-forward with expected old head 8b403d1. The new-head CI run #42437543266 is queued; the old run's Rust jobs were cancelled by supersession, not passed. Required validation on the exact head:
    bazel test //crates/tracedecay-rusqlite-runtime:unit_test //crates/tracedecay-maintenance:unit_test --test_output=errors
    bazel build //crates/tracedecay-rusqlite-runtime:rustfmt_check_2024 //crates/tracedecay-maintenance:rustfmt_check_2024
  • No Cargo, delegation, policy/budget increases, new dependencies, CI changes, operator-data reclamation, release or infrastructure changes. No files from held fix: consolidate follow-up application issue repairs #4776 are transplanted or modified.

Still required before closing #4099

This removes a demonstrated source of measurement work amplification; it does not make the physical page walk constant-time or prove that every large store fits its deadline. Per-family bounds, daemon-owned retention/reclamation and the fresh-profile 24-hour normal-use acceptance must still be demonstrated on master. Keep #4099 open; keep Kev/decision-model work and mixed #4776 draft and excluded from merging.

Current CI repair (679f636)

On the prior head d9559c8, pinned Repository gates passed. Clippy failed on storage_report.rs:2492 with clippy::assert_is_empty (-D warnings): the added test used assert!(absent.stores.is_empty()). Commit 679f636 replaces that with a typed assert_eq!(absent.stores, Vec::<StoreSizeReportEntry>::new()), retaining the failure check and adding value-level diagnostic output. Published via exact-head expected_sha=d9559c8, force=false, with no concurrent commits overwritten.

The earlier fleet Linux job failed with UNAVAILABLE: io exception from the external Bazel remote executor (not a repository test assertion). The earlier Clippy job additionally observed that transient executor failure. Fresh hosted CI is pending on 679f636. No Bazel executable or Rust toolchain is available locally, so the current Rust test and Clippy results remain unverified. Do not claim passing gates, merge, or #4099 closure until actual results and master acceptance exist.

@changeset-bot

changeset-bot Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: cec4106

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Refs #4099. Aggregate each table's dbstat payload inside a correlated
subquery instead of materializing all pages for GROUP BY. Preserve the
existing reader snapshot, cancellation, deadline and typed-error path.

Add nonzero filesystem-backed regressions for exact payload attribution,
growth, logical deletion versus physical compaction, VM-work reduction,
and interruption followed by a successful retry. No retention policy or
measurement budget is relaxed; full 24-hour profile acceptance remains open.
@ScriptedAlchemy ScriptedAlchemy changed the title fix(storage): distinguish unreadable registry from absent registry fix(storage): stream growth measurements and preserve registry errors Nov 8, 2026
Refs #4099. Apply the exact rustfmt delta reported by the hosted
Repository gates; no production logic or acceptance criteria changed.
Refs #4099. Use a value assertion so the hosted Clippy
assert_is_empty policy can display an unexpected store on failure.
Refs #4099. LIKE treats the underscore in sqlite_% as a wildcard,
silently excluding valid user tables such as sqlitemessages. Match the
literal reserved sqlite_ prefix instead without changing the streaming
page census or its cancellation and snapshot boundaries.

Add a file-backed regression that requires the omitted table's nonzero
payload and subsequent growth to match an independent dbstat sum while
still excluding sqlite_sequence. Preserve the concurrent CI repair.

Copy link
Copy Markdown
Owner Author

Published a3654ac directly to this PR through GitHub (Refs #4099). The concurrent CI repair 679f636 is preserved as its parent; the branch update used that exact expected old head with force=false.

Production bug fixed: the census excluded LIKE 'sqlite_%'. _ is a wildcard there, so a valid user table such as sqlitemessages silently disappeared from the table-growth measurement. The production filter now uses NOT GLOB 'sqlite_*' to match the literal reserved prefix. Snapshot ownership, streaming aggregation, cancellation, deadlines, and physical-footprint accounting are unchanged.

Regression added: the file-backed Rust test calls the production reader, requires a nonzero sqlitemessages measurement, inserts more payload, checks its exact total against an independent named-table dbstat sum, and confirms that the real internal sqlite_sequence table remains excluded. The prior three regressions remain unchanged.

Executed SQL verification: using SQLite 3.46.1 on temporary files and SQL extracted from the candidate source, the old filter omitted 1,048,581 payload bytes; the new filter reports them and subsequently reports 2,097,162 bytes after another insert. The original fixture still has exact reference parity and 6,577 versus 19,043 VM steps. Interruption returns SQLITE_INTERRUPT, and retry returns the complete exact result. These are actual SQL checks, not a Rust/Bazel test pass.

The uploaded source blob f24bfed629aca65edaaca2b120a10dca1e798eba matches the locally inspected candidate; the separately reviewed GitHub diff is exactly one file, +39/-1. No code from held #4776, CI configuration, release files, production data, or other task branches was changed.

Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42442890231 (queued at verification). Local Bazel and Rust toolchain are unavailable. Require the real current-head Bazel/Clippy/format results before merging. This is a bounded correctness fix, not evidence that the original 153 GB incident or #4099's 24-hour master acceptance is resolved.

Refs #4099. Align the new user-table growth assertion with the
exact format expected by the Bazel rustfmt gate; no behavioral changes.

Copy link
Copy Markdown
Owner Author

Published 4bcc943de6ecf4cc84243cb3beed2e87d8d38a2e directly to this issue-specific PR (Refs #4099), fixing the exact pinned-rustfmt failure on prior head a3654ac.

The Repository gates log showed only a formatting difference in the new sqlitemessages payload-growth regression at reader/worker/table_sizes.rs:211. I applied the formatter's exact multiline assertion, without changing SQL, tests' semantics, limits, or CI. The commit was independently diff-checked (one file, +5/-1) and published with expected_sha=a3654ac, force=false; concurrent branch changes remain intact.

On a3654ac, Clippy passed. The full Linux fleet suite failed two unrelated targets: //crates/tracedecay:automation_runner_test (three backend_unconfigured vs scheduler-expected assertions) and //crates/tracedecay:runtime_acceptance_suite (LSP frame acknowledgment DeadlineExceeded). Neither belongs to the three-file #4794 patch; I did not alter their owners' code or weaken tests. Fresh CI on the exact new head was queued at verification. This environment has no Bazel executable or repository checkout: no local Rust-test pass is claimed, and full suite acceptance is still blocked.

The master baseline is unchanged at 91c72ab; this remains a partial measurement-path repair, not a 24-hour fresh-profile bounded-growth/reclamation proof. Keep #4099 open, and leave mixed #4776 draft and release #4763 untouched.

Refs #4099. The offline registry snapshot followed symlinked WAL/SHM
members and could copy arbitrarily large foreign files into scratch.
Refuse nonregular members before copying, and keep physical family byte
counts from charging symlink targets outside the inspected profile.

Add real offline-report and family-copy regressions for symlinked source
members, plus partial-coverage and size assertions for user sessions.
No operator data is copied or deleted.

Copy link
Copy Markdown
Owner Author

Published 1e3d62f (Refs #4099), direct to this issue-specific PR. One-file update in crates/tracedecay-maintenance/src/retention/storage_report.rs; the parent is exactly 4bcc943 and the PR branch advanced with expected_sha=4bcc943, force=false. No peer branches or held #4776 were changed.

New root-cause finding: the offline storage report copies the registry's SQLite main/WAL/SHM family into temporary scratch. Its sidecar check used Path::is_file(), which follows a symlink. A linked global.db-wal or -shm could therefore import arbitrarily large foreign bytes into the report scratch before the diagnostic runs. Separately, database_family_bytes used following metadata(), misattributing a foreign linked file as profile-owned physical storage, while the canonical footprint walker refuses that link. This contradicts the report's read-only, profile-scoped accounting.

Fix: fail closed on every nonregular/linked snapshot member via symlink_metadata before copying any member; use no-follow metadata for the physical SQLite family byte tally. The full profile census still marks linked entries unavailable rather than fabricating complete coverage. No reclamation or operator data mutation.

Behavioral regressions: a real on-disk SQLite registry with a symlinked WAL and SHM, each targeting a sparse 512 MiB foreign file, must reject copying before a destination is created; the end-to-end offline report must return a typed Database failure for a linked registry WAL; and an offline report with a regular 4096-byte user-sessions.db plus a foreign linked WAL must count exactly 4096 owned bytes and report partial profile coverage. The foreign files' metadata must remain unchanged. These tests exercise production functions and are not source-shape checks.

Independent checks: manually reproduced the original stat/is_file symlink-follow behavior on a disposable SQLite fixture with a 1 MiB foreign WAL; independently reviewed the uploaded GitHub commit patch and parent comparison (one file, +140/-7). This is mechanism evidence, not a Bazel/Rust test pass. This environment lacks Bazel and the Rust toolchain. Hosted exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42458659562 (pending at check). Require Repository gates, Clippy, and nonzero Bazel //crates/tracedecay-maintenance:unit_test / //crates/tracedecay-rusqlite-runtime:unit_test before merge.

This is partial correctness only; #4099 remains open until 24-hour fresh-profile, multi-store bounds and daemon-owned reclamation are proved on master. No release/infra/CI changes. No transplant from mixed draft #4776.

Refs #4099. Apply the exact formatter hunk reported by the hosted
Repository gates on 1e3d62f. Preserve source behavior and all assertions.

Copy link
Copy Markdown
Owner Author

Published 8913817dbe908e6f27c0bcd317ac8f6c683679e6 on this issue-specific PR (Refs #4099).

The hosted Repository gates on 1e3d62f reported exactly one pinned-rustfmt diff in the new symlinked-SQLite-sidecar regression (storage_report.rs:2571): the symlink(...).unwrap() expression needed one line. Applied that exact formatter change only; all production source, behavioral assertions and limits are preserved. The commit changes one file (+1/-2) and was published as a non-force fast-forward using expected_sha=1e3d62f5c2c687df2cae16e5791416291ef29e97, preserving concurrent work.

Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42462849057 (queued at check). The previous run's Repository gates failed only on this formatting hunk; the Linux/Clippy jobs were still queued there at review time and are not evidence of a pass. There is no local Bazel executable or supplied checkout, so I do not claim a local Rust test pass. Require passing real nonvacuous //crates/tracedecay-maintenance:unit_test, //crates/tracedecay-rusqlite-runtime:unit_test, formatting and Clippy gates before merging.

The master accepted #4123 footprint implementation remains in place. #4099's 24-hour fresh-profile per-store bound and daemon-owned reclamation acceptance remains unproven on master; keep it open. Held mixed #4776 remains draft; release #4763 and other issue-owner branches were untouched.

devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4099. The SQLite snapshot enumerated sidecars before capturing
its validation generation. A writer creating a WAL in between could
commit new project registrations that the report then misclassified as
unregistered. Capture the generation first, then inspect only regular
members and validate after copying.

Add a deterministic on-disk WAL race regression: stale generations must
fail closed, while a fresh generation preserves the committed project.
Also fix the pinned Clippy collapsible-if failure in family sizing.
Refs #4099. The registry snapshot preflight rejected symlinks, but
fs::copy reopened each checked path and could follow a link swapped
in after inspection, copying foreign bytes into report scratch.
Open through a held parent capability with no-follow leaf policy,
then copy from that file handle. A real filesystem regression swaps
a checked WAL member to a foreign link and proves the copy refuses
without creating a scratch member. Preserve existing CI repairs.

Copy link
Copy Markdown
Owner Author

Published f9827683e484336fcd6d7fd752084f6eb85932f0 on issue-specific PR #4794 (Refs #4099), by exact expected-old-head eb751efd4e5c4db616b65cd7b5ab52bc4ad237f0, non-force fast-forward.

New independently identified correctness/safety issue: The offline SQLite family copier checked each source member with symlink_metadata, then used fs::copy to reopen by pathname. A WAL/SHM member replaced with a symlink after preflight could make the report copy foreign bytes into scratch, despite the earlier link guard. The fix opens each checked member through a held parent directory with FollowSymlinks::No, copies from that file handle, preserves the generation validation and current deadline/snapshot ownership, and changes no CI/retention limits or dependencies. A new filesystem-backed Rust regression replaces a prechecked WAL member with a symlink and requires refusal with no copied destination WAL; reversing the production no-follow open would fail it. One-file diff +61/-3.

The Linux filesystem TOCTOU mechanism was independently reproduced using a regular-file probe followed by symlink replacement: ordinary copy followed the foreign target while O_NOFOLLOW refused it. This is not a Bazel/Rust test pass. No local Bazel/toolchain was available; exact-head hosted CI is queued. Keep #4099 open: 24-hour fresh-profile per-family bounds and daemon-owned reclamation remain unproven on master. Mixed #4776 remains draft, release #4763 untouched, and no other owner branch or production state was modified.

Refs #4099. A no-follow open still accepts a regular-file replacement
between preflight and copy, potentially copying a huge foreign WAL before
post-copy generation validation rejects it. Compare the opened descriptor
against checked identity and length before creating scratch; bound the copy
to the inspected length plus one. Add a filesystem regression for same-sized
and oversized regular-file swaps without reading the foreign bytes.

Copy link
Copy Markdown
Owner Author

Published 2d839d8 directly to this focused PR (Refs #4099). The PR was fast-forwarded non-force with exact prior head f9827683e484336fcd6d7fd752084f6eb85932f0; earlier repairs and concurrent work were preserved.

Independent bug found: The preceding no-follow fix rejects symlink replacement but still opens an ordinary file swapped into the WAL/SHM pathname after the first metadata check. Post-copy SourceGeneration::validate would reject the changed path, but only after the report may have copied an arbitrarily large unrelated regular file into scratch. Merely checking regular-file type or matching length cannot reject a same-sized inode swap.

Production repair: Preserve checked metadata for each SQLite main/WAL/SHM member; compare the opened, held file descriptor's identity and inspected length before creating the scratch member (Unix device/inode; non-Unix conservative size/time checks with existing generation validation). Cap each copy at the previously measured length plus one byte and reject a concurrent size change, rather than chasing an unbounded WAL tail. No dependencies, CI policy, thresholds, retention ownership, or production data were changed.

Nonvacuous Unix regression: Replaces a previously checked, 4 KiB regular WAL with a different regular inode of equal size and again with a 512 MiB sparse file. It calls the production copy helper, requires a typed InvalidData refusal and no output WAL for either case, and verifies the replacement files remain unchanged. Removing the new identity guard allows the old implementation to copy the replacements. Existing link-swap and generation-change regressions remain.

Independent filesystem reproduction: On Linux, rename of a different same-sized regular inode leaves is_file and length checks true, but the checked versus opened device/inode differs. This confirms the mechanism, not a repository Rust test pass. This runner has no Bazel checkout/executable. Fresh hosted CI on the exact new commit is queued; require real nonzero Bazel tests, pinned rustfmt and Clippy before any merge. The 24-hour fresh-profile #4099 acceptance remains unproven on master; do not close. Mixed #4776 stays draft and release #4763 is untouched.

devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4099. Keep inode identity verification unchanged while moving
MetadataExt to the module's explicit imports as required by AGENTS.md
and the open review on PR #4794.
Refs #4099. A rollback-mode writer may spill uncommitted registration
pages into global.db while the undo journal remains. Copying only the
main/WAL/SHM family would turn those pages into false registry truth.

Fail closed on the rollback journal before writing scratch. Add direct
and end-to-end real-SQLite regressions for refusal during an open
transaction and correct registration after rollback. Preserve all
prior snapshot generation and no-follow protections.
devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Published focused Refs #4099 correctness fix on PR #4794: commit 32aa388.

Root cause: the offline registry reporter copies only global.db, -wal, and -shm. Under SQLite rollback-journal mode, an in-flight writer can spill uncommitted pages into the main file while its undo journal remains at global.db-journal. Copying the main file without that journal can make uncommitted project registrations look authoritative. An independent SQLite 3.46.1 reproduction on disposable files demonstrated a main-only snapshot reading /repos/uncommitted while the open writer later rolled back to /repos/original….

Published repair: refuse a present rollback journal before writing snapshot scratch and propagate the existing typed report failure instead of classifying stores using an unsafe registry. Two real-SQLite Rust behavioral regressions test the lower-level snapshot refusal (including no output) and the end-to-end build_storage_report refusal followed by correct registered-store reporting after rollback. This is intentionally conservative for persistent-journal configurations; it does not delete or checkpoint any live journal.

The exact-head non-force fast-forward preserved 7f54a84 as parent. Verified GitHub commit diff: one issue-owned file (storage_report.rs), +98/-0; no CI, release, infra, production data, #4776 or sibling branches touched. Bazel/Rust testing is unavailable locally, so no Rust test, Clippy, or formatting pass is claimed. Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42484464871 (queued on inspection). Keep #4099 open: the 24-hour fresh-profile multi-store bounded-growth and daemon-owned reclamation acceptance has not been proven on master.

Apply eight exact Rust 2024 rustfmt differences from hosted Bazel
Repository gates. No behavior, schema, or test assertion changes.
Refs #4099. Allow committed PERSIST/TRUNCATE journals while rejecting
active/uncertain rollback state and verifying the journal witness
across the registry snapshot copy. Cover committed, active and rolled
back states with real SQLite report regressions.
Refs #4099. Apply the exact format-only correction reported by Bazel
Repository gates to the committed-journal probe. No behavior or test
assertion changes; preserve the existing concurrent issue-specific work.

Copy link
Copy Markdown
Owner Author

Published 0c7597a to this issue-specific PR (Refs #4099). The previous exact-head CI run #42490632388 reported a single pinned rustfmt mismatch at storage_report.rs:1590: the inspect_inert_rollback_journal signature must be on one line. The eight scripts/policy tests completed successfully; Repository gates failed on formatting. This commit applies exactly that format-only correction, with no test or behavior change. GitHub compare confirms one file, +1/-3. It was published by a nonforce fast-forward from expected old head 7990e35140d6aca26a161060b0a1e4ee837269f2, preserving all concurrent work.

Fresh exact-head CI: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42493906252 (queued at check). No local Bazel/Bazelisk executable or Rust workspace checkout is available, so Rust/Clippy/format acceptance on the new head is not claimed. Do not merge until current-head gates and review are verified. Issue #4099 still requires complete 24-hour fresh-profile, per-store bounded-growth and daemon-owned reclamation proof on master; mixed #4776 remains draft and untouched.

Refs #4099. Avoid a correlated dbstat rescan per table while retaining
exact payload totals, interruptible page mode and bounded table-only state.
Add a many-table VM-work regression that fails under the prior query.
Refs #4099. Keep the SQLite page name ValueRef alive while its UTF-8
slice is used to update the bounded table totals.

Copy link
Copy Markdown
Owner Author

Published an additional focused Refs #4099 measurement-path fix on #4794 as e64168b and the reviewed borrowed-value correction 1b9bf85. Both were exact-expected-head, non-force fast-forwards, and only crates/tracedecay-rusqlite-runtime/src/reader/worker/table_sizes.rs changed.

New root cause: the previous correlated SUM(payload) FROM dbstat WHERE name = schema_entry.name removed the page-sized SQL GROUP BY sorter, but still executes a virtual-table page census for each table. That scales poorly for a profile with many tables and risks keeping table-growth measurement unavailable under deadline. The production reader now enumerates user tables (including empty/rootless tables) once, scans page-mode dbstat once, and accumulates only table totals in a sorted map. Existing snapshot, interruption, typed errors and physical footprint accounting stay intact. It validates payload conversion and checked sums.

Behavioral evidence: a 36-table real-SQLite fixture test requires the new reader's exact payload totals to match the old independent census and its measured VM work to be less than one-third of the grouped baseline. The existing overflow, WITHOUT ROWID, logical deletion/physical compaction, user-table-name and interruption/retry regressions remain. An independent SQLite 3.46.1 mechanism reproduction used 36 tables with 32-KiB payloads: 14,353 grouped, 10,525 correlated and 1,798 single-page-stream VM steps (the Rust implementation has not been run locally). No arbitrary deadline or budget was raised.

Fresh exact-head CI is pending; Bazel/Bazelisk and the Rust checkout are unavailable locally, so no Rust pass is claimed. #4099 stays open pending master verification, a 24-hour fresh-profile bounded-growth run and evidence of daemon-owned reclamation. Held draft #4776 and Kev lanes, release #4763, production and other owner branches are untouched.

Refs #4099

Apply the precise rustfmt diff from CI run 42506693304 without
changing census behavior, bounds, or regression assertions.

Copy link
Copy Markdown
Owner Author

Published focused CI repair for Refs #4099 in issue-specific PR #4794: bb53e20. Hosted CI run 42506693304 confirmed //crates/tracedecay-rusqlite-runtime:rustfmt_check_2024 failed only the exact table_sizes.rs:35 map_err formatting diff; eight Repository-gate script tests passed. Applied the pinned formatter's exact change without modifying the one-page dbstat production algorithm, bounds, or its behavioral assertions. Published via expected_sha=1b9bf85, force=false; the verified compare is one file (+2/-3), one descendant commit, no peer changes overwritten. Fresh CI run 42508705498 is queued.

Independently reread the registry generation ordering, no-follow metadata import and persisted-journal checks in storage_report.rs; all three older reviewed findings have corresponding current-source corrections and review replies, so their stale inline GitHub threads are now marked resolved. Bazel/Bazelisk and the full Rust checkout are unavailable in this runner; repository Rust tests, Clippy and exact-head CI are not yet proven passing. This does not establish 24-hour fresh-profile bounded growth, complete daemon-owned reclamation or acceptance on master. Keep #4099 open; held mixed #4776 and Kev/decision-model PRs stay draft, release #4763 and sibling branches untouched.

Refs #4099

A committed PERSIST journal remains allocated but the report's registry
and user-sessions family totals counted only main, WAL, and SHM bytes.
Count the rollback journal without following foreign symlinks and test a
real persisted journal through the offline report and footprint census.

Copy link
Copy Markdown
Owner Author

Refs #4099 — exact-head CI blocker / independent gate review (2026-11-08).

On current PR head 2fdaffbe101e24afef30a4365ef1fc33d3fbe0e4, CI run #42512529574 completed Repository gates, Clippy, Feature gates, dashboard, and both affected Bazel unit targets successfully: //crates/tracedecay-maintenance:unit_test (26.9 s) and //crates/tracedecay-rusqlite-runtime:unit_test (23.2 s). This is hosted Bazel evidence for the new storage-report and table-growth regressions, not a local Cargo run.

The required Linux fleet gate is still red, for failures outside the three files changed by this PR:

  • //crates/tracedecay:automation_runner_test: 167 passed, 3 failed — jobs::retained_scheduler_runner_reacquires_after_due_prefilter, jobs::scheduler_prefilter_live_lock_wins_over_not_due_summary, and jobs::scheduler_trigger_skips_repeat_and_respects_lock_discipline. The latter two observe backend_unconfigured instead of the expected scheduler-lock / manual-schedule states.
  • //crates/tracedecay:dashboard_api_test: 88 passed, 1 failed — memory_curation::retained_admin_journey_commits_add_update_feedback_and_remove timed out while calling fact_store_remove.

The same three automation-runner assertions also failed on unmodified master run #42328697300 (Nov 7) and run #42287676890 (Nov 6), proving this portion of the Linux failure predates the issue-specific patch. I did not weaken gates, change scheduler/retained-memory owner files, or rerun known-red work just to manufacture a green status. Windows remains in progress as of this review.

Blocker: PR #4794 cannot be represented as fully CI-passing and must not merge or claim closure while required Linux CI is red. #4099 additionally still lacks the 24-hour fresh-profile bounded-growth and daemon-owned reclamation proof on master. Mixed draft #4776, Kev/decision-model lanes, release #4763 and other owners' branches remain untouched.

Repair three hosted automation_runner_test failures by supplying the
configured executable identity required by the production prefilter.
Use the same identity for the prefilter and retained runner, preserving
manual-schedule, exact-ledger, live-lock and lock-race assertions.

Add a production-prefilter regression for unconfigured refusal,
deduplication and same-occurrence recovery after binding, with zero
backend calls and no effectful terminal. Do not relax backend admission,
change timeouts, or alter the graph publication changes.

Refs #4532

Copy link
Copy Markdown
Owner Author

Published focused Doctor footprint correctness fix for Refs #4099 on issue-specific PR #4794: 1a80a56 plus independent test-fixture review correction edb27a3. Both commits were exact-expected-head nonforce fast-forwards; only crates/tracedecay-daemon-service/src/doctor_kernel.rs changed (+140/-8).

Root cause: on master, profile_footprint_read called registry.list_code_projects(...).await.ok()?. When registry enumeration failed, Doctor skipped the entire filesystem census and mapped footprint evidence to Unknown, even if a project or the whole profile exceeded its physical bound. The registry need not be readable to measure regular-file sizes. The new production path always runs that bounded metadata-only census even on a failed registry read, but marks project registration/root liveness as unverifiable, never falsely unregistered or missing. Healthy/overbound evidence is still derived from measured bytes and configured limits; no retention deletion or unsafe orphan recommendation is triggered.

The new filesystem-backed Rust regression makes a 4 KiB project WAL and 2 KiB LCM payload, imposes 1 KiB project/profile bounds, requires named degraded Doctor findings for both, and distinguishes registry-failure root state from a successfully empty registry and a confirmed deleted registered root. It uses only a disposable TempDir; no user profile or credentials touched. Rechecked against master 91c72aba and verified this PR advances dcca6bca by exactly two commits, with one changed file. Existing PR #4799 changes footprint presentation, and held mixed draft #4776 changes a different Doctor function; neither branch nor their changes were modified.

Validation: exact-head CI is queued; local Bazel/Bazelisk are unavailable, so no Rust compilation, Clippy, formatting or test pass is claimed yet. This is a partial repair. Do not close #4099 before CI, master integration, daemon-owned reclamation and the required 24-hour fresh-profile growth evidence. #4776 and Kev/decision-model PRs remain draft and excluded from merge; release #4763 unchanged.

devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4099. Validate all offline and daemon report project IDs at the common store-sampling boundary, before constructing or walking their paths. Keep direct project reports and persisted registry rows on one canonical path-segment policy. Regression proves path escape is rejected before any scan and a normal store is still measured.
Refs #4099. Keep the invalid-identity fixture list compliant with the pinned formatter.

Copy link
Copy Markdown
Owner Author

Refs #4099 — published registry-path containment fix on issue-specific PR #4794: 0bb47ea and formatting follow-up aa507c4.

Root cause: The offline storage-report snapshot's code_projects.project_id strings and the live paged registry path both reached append_project_report without the canonical project-ID check used by direct project reports. An invalid ID such as ../../outside could make profile/projects/<id> resolve outside the requested profile and have the report measure an unrelated tree. This is especially dangerous during profile-growth diagnosis: a foreign subtree could consume the walk budget and contaminate the reported bytes.

Fix: Validate IDs using the existing tracedecay_runtime_core::storage::validate_project_id at the common registered-store sampling boundary, before path construction or any filesystem observation. Bad registry identities now return typed TraceDecayError::Config rather than being followed. The patch neither deletes data nor changes thresholds, registry writes, or daemon maintenance.

Regression: The actual append_project_report production helper is exercised against seven invalid identities (parent traversal, nested paths, backslash, empty, dot); each must refuse before scanning or spending the entry budget. The fixture also confirms an external sentinel file is preserved, and an ordinary proj_valid report still measures its exact nine bytes. Restoring the old unchecked helper makes this test fail. An independent temporary-directory filesystem reproduction confirmed the old path joined into the foreign tree. This reproduction is not a Rust/Bazel pass.

Published using two exact expected-old-head, non-force fast-forwards (from abd9956 and 0bb47ea). Verified GitHub compare: two descendant commits, only storage_report.rs modified (+68), no peer commits overwritten. Overlapping open #4809 touches the same file's separate end-of-file regression; its branch was not changed. Draft #4776, Kev/decision-model PRs, release #4763, production and all other owner branches are untouched.

Verification gate: New-head hosted CI #42541449261 is queued; no local Bazel executable or Rust workspace checkout is available, so the new regression is not yet verified by Rust CI. Do not merge solely on this reproduction. The 24-hour fresh-profile bounded-growth, daemon-owned retention/reclamation and complete acceptance on master are still outstanding; keep #4099 open.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Audited the current table-size census at aa507c4, specifically reader/worker/table_sizes.rs. The current implementation is newer than the body summary: it initializes user tables from schema and accumulates a single SELECT name, payload FROM dbstat page walk, not one correlated SUM per table. It preserves empty/rootless tables, excludes index/internal payload from user-table totals, checks negative/overflow payloads, and propagates interruption rather than publishing partial success.

Independent executable check in this audit: SQLite 3.46.1, five synthetic schemas (0, 1, 4, 32 and 128 additional indexed tables), 45 assertions passed. Compared the literal production SQL plus equivalent accumulator against an independent grouped page reference; included WITHOUT ROWID/overflow data, rootless virtual and empty tables, insert/delete payload changes with unchanged physical page count, actual SQLite interruption, and successful retry. This is SQL/algorithm evidence only—not execution of Rust/Bazel, native macOS, or the rest of this PR's filesystem/registry paths, and not a wall-clock speedup claim.

No merge clearance is implied: current-head full applicable CI and the rest of the PR remain separate review gates. Keep #4099 open for its complete master-backed storage-growth acceptance. The stale correlated-query description should not be used as the reviewed implementation.

Refs #4099. Retain measured baselines and latest evidence across restarts, but keep deadline-based skip markers process-local so a new reader can retry after improvements. Add a real-store restart regression that reproduces the previous 24-hour suppression.

Copy link
Copy Markdown
Owner Author

Published cec4106 on this issue-specific PR (Refs #4099).

Root cause: crates/tracedecay-maintenance/src/telemetry.rs serialized the 24-hour OverBudgetTableSizeWalk timeout-suppression marker into maintenance/table-growth-v1.json. A daemon restarted with a faster reader (including the newly optimized single-page dbstat census in this PR) still reloaded the old executable's timeout and refused to measure the same-sized store for up to 24 hours. This prolongs exactly the Doctor 'table payload growth measurement unavailable' symptom, even if the newly deployed algorithm would finish.

Fix: Keep deadline-based skip state process-local (#[serde(skip)]) while continuing to persist the actual latest typed read and per-table watermarks. Same-process timeouts still throttle retry until a store shrinks or its mark ages out; restart can now re-probe under the current reader. Old persisted timeout fields are ignored rather than used as a durable limit. No duration/budget/CI policy was increased.

Regression: Added a real database + maintenance-registry restart test that establishes a baseline, inserts nonzero payload, simulates a recent timeout marker, verifies the same process skips and persists its last unavailable state, then restarts and requires a new census with positive measured growth from the original baseline. Before this fix the restart would still skip and fail this test.

Publication: Exact prior SHA aa507c45c1b67cba37bf73b0859d68116b1fa234, force=false. Verified comparison: one descendant commit, one changed owned file (telemetry.rs), no overlapping owner PR edits. New-head CI #42545596782 was queued at verification; local Bazel is absent, so no Rust/Clippy pass on this head is claimed. This is not 24-hour master acceptance or a full GC fix. Keep #4099 open, #4776 and Kev/decision PRs draft, release #4763 read-only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant