Skip to content

fix(code-index): fence stale progress boundaries - #4800

Open
ScriptedAlchemy wants to merge 28 commits into
masterfrom
fix/4508-progress-boundaries-ci-clean
Open

ScriptedAlchemy wants to merge 28 commits into
masterfrom
fix/4508-progress-boundaries-ci-clean

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Refs #4508 — partial correctness repair, not a complete fix. This replaces closed PR #4795 without rewriting, force-pushing, or deleting its branch/history. Repository-gates CI rejected immutable commit 9dca677 on that PR because its header is 75 characters (72 maximum). The exact same two source/test file blobs are reapplied as one new ordinary commit from master 91c72aba7750a398dd28e83d64bf0c6d6fef4172, with a compliant header.

Implementation

  • Preserve the maximum published progress time per generation owner, including verified staging retries; reject out-of-order lower committed page/chunk/import/payload counters and a delayed retreat from Ready.
  • Allow verified SQLite staging restarts to reset committed counts without rejecting the real new attempt. Fence wrong generation/owner publications.
  • Fence daemon incarnation, producer incarnation, and immutable sealed-source digest across even a verified staging restart, so a foreign report cannot claim the cleared slot. A fifth falsifiable regression exercises each mismatch before/after reset and legitimate successor ownership.
  • Deduplicate repeated blocked reasons and unchanged phase/batch retry reports; allow a true Ready transition to clear a blocked status.
  • Five nonzero regression cases in the existing Rust scheduler test suite exercise stale generations, every committed counter, stage restart, resumed progress, unchanged phase retries, and blocked/ready transitions.

Validation gates

No Bazel execution is claimed in this environment; it has no Bazel toolchain or repository checkout. Hosted CI must execute native Rust tests for this exact head (e.g. bazel test //crates/tracedecay-code-index-runtime:unit_test --test_output=errors) and all applicable gates. The workflow title's native=none denotes no optional dispatch-only macOS/Windows native-target override. The ordinary ready-PR Linux CI still uses Bazel (bazel test --config=ci), and its actual job result must be inspected; neither its presence nor its title proves a Rust test pass. Independent human review is not recorded. Do not merge based on static checks alone.

Remaining #4508 scope

Cold pre-seal capture and parse progress are still absent; native graph page/segment publication does not report committed milestones. End-to-end cold + resumed status polling, final phase transitions, and stall/refusal behavior are not proven. Keep issue open, do not mark Fixes #4508 until all acceptance is demonstrated on master. This PR stays distinct from protected draft Kev/decision-model #4776. No release, tags, infrastructure, or credential changes.

2026-11-08 identity-fencing follow-up

Commit 8270bfcd81e1 adds the missing identity check, including the previously unprotected post-reset window, and a nonzero regression. No Bazel test pass is claimed. Hosted workflow for this exact head is queued.

2026-11-08 committed-observation fix

Commit 6dc0f5804450 closes a remaining false-progress path: publish_text_progress_boundary can replay an unchanged durable cursor with only different elapsed time/rates/latency, which previously incremented progress_epoch and renewed last_progress_micros despite no committed work. The shared slot now deduplicates such reports while continuing to publish source scan observations, actual phase/batch boundaries, committed pages, new refusals, and verified Ready recovery. A sixth falsifiable regression was added. No native Bazel pass is claimed. CI for this head is queued. Capture/parse before sealing, committed graph milestones, full cold/restart acceptance, and independent review remain unresolved; this stays Refs #4508, open, and unmerged.

Refs #4508

Reapply PR #4795 without changing its existing branch or commits.
The previous immutable history failed the 72-character commit header gate.
@changeset-bot

changeset-bot Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: bb3f090

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Refs #4508

Reject foreign daemon, producer and sealed-source identity reports,
even after a verified SQLite staging restart clears committed counters.
Add a nonzero regression for the production progress authority.

Copy link
Copy Markdown
Owner Author

Taking the #4508 producer implementation on this existing branch, starting from exact head 29c1258. File ownership for this pass: pre-seal capture/parse observation and its separately typed attempt authority, native graph writer progress, and the corresponding freshness/status consumers and regressions. Maintenance/sweep owners: please do not duplicate or rewrite this work; preserve all commits and use expected-head nonforce fast forwards. #4616 owns sealed-store reader/receipt/retention changes; I will keep graph writer changes separate and reconcile any overlap against the exact current heads. Existing boundary tests are not completion evidence. Refs #4508; no closure/merge claim, and #4776/Kev stays draft and #4763 stays read-only.

ScriptedAlchemy commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner Author

Published commit 9f3bfe04cbac4dee01979cd44791d57c5a68b99c directly to this PR, fast-forwarding exact head 29c1258b8f491def3cf34c5cd6e4b4a6b5e194f6 with expected_sha and force=false.

Source changes

  • dashboard/src/workspaces/observatory/CodeIndexPipeline.tsx: keep active polling when rebuild_in_flight is true or immutable-artifact restore_progress exists, even when the serving envelope/previous report is already ready. useObservatoryReads therefore keeps its existing 1-second active interval instead of prematurely falling back to 30 seconds.
  • An in-flight build without a committed report now renders “code-index build in progress; progress not yet reported”, not “no active code-index build.” It does not manufacture a generation, phase, percentage, or completed-work counter.
  • Added CodeIndexPipeline.dom.test.tsx: eleven polling cases plus two React rendering journeys for unreported → committed → ready → retired progress and bounded restore → restored state. Existing tests are unchanged.

Verification performed

The original source was reconstructed byte-exactly and matched GitHub blob 5a3704cdfe5ec3b429fb0b9086e7acc67f35c0b9. A Node test harness executed the actual TypeScript-transpiled exported polling predicate, with unused UI imports guarded against invocation: 11 checks, 4 failing before; 11 passing, 0 failing after. This is focused predicate execution, not a React renderer or full Vitest run. Both changed TSX files also passed syntax transpilation. Published source/test blobs match the locally checked bytes (571d7eb4…, bc61e12c…). The final remote compare is one ordinary commit and exactly these two files (12 source insertions/2 deletions plus the new regression file).

Still required: run pnpm --dir dashboard test src/workspaces/observatory/CodeIndexPipeline.dom.test.tsx, dashboard typechecking, and native Bazel checks for the PR's Rust changes. This container has no Bazel and cannot resolve GitHub for a full checkout/dependency installation; no native Rust pass, full Vitest pass, DOM pass, or independent review is claimed. Exact-head hosted CI is queued.

Backend capture/parse and direct-seal/fallback graph producers remain owned by the active implementation pass above; this commit does not touch those files or claim to complete them. Refs #4508 only. Keep the issue/PR open pending full acceptance; #4776/Kev and #4763 remain untouched.

Keep active polling for in-flight rebuilds and lexical artifact restores,
even when the serving envelope or previous generation is already ready.
Do not render an in-flight build without a report as an idle pipeline.

Add polling regressions and DOM transition coverage for unreported work,
committed progress, completion, and bounded artifact restoration.

Refs #4508
Carry daemon/producer epochs, generation, last-progress time, committed text counters and blocked reason into MCP search and graph-read summaries. Distinguish absent progress from reported zero work without inventing pre-seal authority. Add nonzero rendering regressions for repeated reads, blocked recovery and restart/remount epoch scope.

Refs #4508. Producer instrumentation and cold/restart end-to-end acceptance remain incomplete; no Bazel execution is claimed.

Copy link
Copy Markdown
Owner Author

Published 0fc10fe as a non-force fast-forward with expected head 9f3bfe0, preserving the concurrent dashboard commit and all earlier commits.

This is a narrow production MCP propagation fix, NOT completion of #4508. Search and graph-read summaries previously dropped the daemon's progress authority, epoch, last-work timestamp, committed counters and blocked reason. They now carry generation + daemon/producer incarnation, progress epoch, the original last_progress_micros, committed TEXT artifact counts/bytes and the typed blocked reason. Text counters are explicitly labelled as text, not native graph writes. An in-flight rebuild without a producer report renders progress=unreported rather than inventing a pre-seal generation/count/clock.

Four new tests exercise the actual search_freshness/freshness_lines renderers with nonzero counters: cold_unreported_work_has_no_fabricated_progress_authority; repeated_reads_do_not_turn_elapsed_time_into_progress; blocked_and_recovered_reports_preserve_work_and_name_the_real_phase; restart_and_remount_keep_epoch_scope_visible. Existing coverage and restore/park tests are retained. These are renderer regressions, not cold-build/restart end-to-end proofs.

Validation: diff inspected against exact parent; no Rust build/test/check/codegen has been run locally. This session has no Bazel/Bazelisk executable or checkout-capable network path (git fails proxy DNS), and the public download path also failed. No Cargo, alternate Rust toolchain invocation, generated-contract hand edits, infrastructure changes, or test bypasses were used. Exact-head native CI run 42439812291 is queued; no pass is claimed.

Independent PR-maintenance review requested for 0fc10fe, especially text-vs-graph attribution, owner-scoped epochs, absence of read-time clock fabrication, and the renderer assertions. Keep this PR unmerged: separately typed pre-seal capture/parse production instrumentation, direct native graph durable-write instrumentation, and cold/interrupted/restarted end-to-end acceptance are still unfinished. The earlier unreferenced producer-draft commit 95aac68 is incomplete and must not be adopted; no producer draft was pushed onto this branch. Refs #4508. #4776/Kev remains draft; #4763 untouched.

Refs #4508. Keep pre-seal work under separately typed attempt authority, not a fictional sealed generation. Capture on clean-HEAD and dirty paths; instrument actual staged parser-backed file work; reject stale, duplicate and delayed callbacks. Durable native-graph work, codegen/wire and restart e2e remain open.

Copy link
Copy Markdown
Owner Author

Published 091bf9d86c8e7dbc1e9bd661fe60a06ea10366ae on this existing PR by expected-head non-force fast-forward from 0fc10fed2121688077c5427072b495ee0f393690 (all prior boundary, dashboard and MCP commits preserved). Refs #4508; partial producer work, NOT acceptance or a close/merge signal.

  • Introduced CodeIndexPresealWorkV1 (capture vs parser-evidence materialization) and a separately typed attempt slot keyed by daemon incarnation, producer incarnation and monotonic pass sequence. The slot has no fictional generation ID or sealed digest. Only actual positive producer events move work counts or last_progress_micros; duplicate/out-of-order callbacks cannot refresh a stalled attempt, and cancellation/retry/Drop fences an old producer.
  • Recorded successful real capture decisions inside the parallel dirty-worktree worker, rather than after the whole batch, and inside the clean-HEAD Git-tree visitor, which is the primary cold path. Clean traversal has no invented total. Reused rows are not counted as new capture.
  • Recorded parser-backed file materialization only after its segment has successfully entered preparation, explicitly not as a durable text artifact or native graph commit. Reused parser evidence is not labelled a fresh parser invocation.
  • Added two nonzero slot regressions (no heartbeat/rollback, stale retry and incarnation fencing) and one real two-file cold reconcile regression asserting positive capture + parser-evidence observations.

Not claimed: no local Rust/Bazel execution (this environment lacks Bazel/Bazelisk and checkout-capable network); exact-head CI run 42447419869 was queued at last check. The attempt slot is not yet surfaced through the freshness wire, CLI/MCP or dashboard. Native durable graph writer/page progress and end-to-end restart/resume, stalled and blocked/recovered transitions are still required, as are hosted Bazel tests and independent review. Do not treat these three tests as #4508 completion. #4776 remains draft; #4763 was untouched.

…ied compact segment

Refs #4508. Distinguish real WAL-settled page transactions from adopted receipts; direct seal is one compact artifact, not invented durable pages. Add nonzero persistent write/replay/close-reopen regression. Status wiring and restart E2E remain open.

Copy link
Copy Markdown
Owner Author

Follow-up producer commit de2d6134efac295d253b915424fd7a4976ac04ae has now been published on the same #4800 branch, fast-forwarded from exact head 091bf9d86c8e7dbc1e9bd661fe60a06ea10366ae with expected_sha and force=false.

This targets the actual native graph writers, not project_graph_publication_phase's read-side projection:

  • Native fallback staging pages now yield a typed Applied vs Reused transaction verdict from the real run_gated_batch write path. The graph_native_stage_page_applied event is emitted only after an actual row+receipt transaction completes, including WAL settlement for persistent stores; replaying an already-applied receipt emits no fake write event. It carries real page ordinal/kind, mutation and live-byte counts, and whether WAL sync applies.
  • The dependency-free direct-seal path does not write pages. It emits graph_native_compact_segment_verified only after its single compact container is installed and reopened with recovered-digest proof; no fabricated per-page progress during encoding.
  • Added a real nonzero persistent-graph regression which applies an actual native page, verifies replay returns Reused instead of Applied, then closes/reopens the WAL-backed store and proves the page receipt and rows survived.

This is real producer instrumentation, not yet the progress-status pipeline. Neither the native events nor the new separately typed pre-seal attempt slot are wired to freshness/CLI/MCP/dashboard yet. No local Bazel/Rust test has been executed; wait for an actual exact-head hosted run, not queued/skipped jobs. Full cold + restart/resume, stalled, blocked/recovered end-to-end evidence remains required. Refs #4508, not Fixes; keep PR and issue open, and do not touch #4776 or release #4763.

Refs #4508. A concurrent verified segment may be adopted after a losing staging install; only the writer that actually installed a new directory emits the native durable-segment event. Add real sealed-container collision, cleanup, and verified-reopen regression. Status wiring and interrupted-restart acceptance remain open.

Copy link
Copy Markdown
Owner Author

Published follow-up b329b67 on this same PR with a checked non-force fast-forward from exact parent de2d6134. Refs #4508 (partial).

Native writer correctness: the compact sealed-store path previously emitted graph_native_compact_segment_verified even when this writer lost the directory-install race and merely reopened another writer’s verified artifact. The new install verdict suppresses that false new-write observation while preserving the required digest-verified adoption and cleanup. Added concurrent_sealed_install_candidate_does_not_count_as_new_write, a nonzero real-container fixture that installs the winning segment, exercises an actual losing rename into the nonempty destination, verifies loser staging cleanup, and reopens the real winner with the expected recovered digest and nonzero entity/relation counts.

Scope coordination: this touches only the native compact writer-install/reporting block of sealed_store.rs; #4616/#4804 owns bounded receipt reads, read-side cleanup and retention. No changes to those reader/retention sites, no conflict rewrite or dropped commits. Independent review requested for the race outcome, platform-specific directory rename behavior, and whether this remains compatible with #4804.

Not acceptance: no local Bazel/Bazelisk or checkout-capable path; no Rust compile/format/test result claimed. Exact-head CI 42450800548 was queued when last sampled. Pre-seal typed attempt and native durable events still require daemon → CLI/MCP/dashboard status wiring and real cold/interrupted-restart/stall/blocked/recovered end-to-end tests. Keep #4508 and this PR open. #4776/Kev remains draft; #4763 untouched.

Copy link
Copy Markdown
Owner Author

Newly confirmed CI blocker — exact #4800 head b329b671 (2026-11-08)

Hosted Repository gates job 128639401232 executed and failed commitlint, rather than merely being queued. Its output: header must not be longer than 72 characters ... current length is 88 [authored-header-max-length]. This is the existing immutable de2d613 subject, feat(graph): observe genuinely committed native stage pages and verified compact segment (88 characters). A follow-up additive commit does not repair an earlier commit checked individually.

Unblocking this mandatory gate normally requires replacing that historical commit with a compliant-header commit (which would rewrite history), or publishing its content on a clean replacement lineage (which would duplicate this active PR). Both violate the explicit preserve all commits / no rewrite / no duplicate constraints. I have not force-pushed, modified policy, bypassed checks, discarded any commit or created another PR. This is a hard acceptance blocker that requires a policy decision outside the current authorization; later additional code changes cannot make this already-published header pass. The benchmark harness self-tests succeeded, but no native Bazel test or full #4508 end-to-end acceptance is established. Issue #4508 and PR #4800 remain open; protected draft #4776 and release #4763 remain untouched.

Refs #4508

Carry the real capture and parse attempt across the mounted scheduler,
canonical status wire, CLI wait display and MCP search. Preserve attempt
owner, epoch and work timestamp independently of sealed generation data.
Add nonzero wire and renderer regressions. No generated-contract edits.

Copy link
Copy Markdown
Owner Author

Published 2dc0150edc57 onto the existing #4800 branch using expected_sha=b329b671… and force=false, preserving all ten prior commits. Refs #4508 (partial). This connects the already-produced, separately typed pre-seal capture/parse attempt to the actual mounted registry's lock-independent freshness read. The canonical Rust wire now carries the attempt's daemon/producer incarnation, sequence, phase, completed/total source decisions, progress epoch, original work timestamp, and active state. CLI init --wait and MCP graph/search summaries show those observed fields without inventing a sealed generation, durable text commit, graph page, or polling heartbeat. Three focused nonzero wire/renderer tests were added, but none has been Bazel-executed here.

I also checked master, current root AGENTS.md (repository tree contains no nested AGENTS.md), paginated issue comments, PR discussion, reviews (none), the exact earlier CI jobs, and competing #4508 PRs. The preexisting immutable 88-character de2d613 header still fails mandatory commitlint: the preservation/no-rewrite/no-duplicate instructions preclude correcting it in this lineage. Generated dashboard contract outputs must be regenerated with the mandated Bazel codegen, not hand edited; the new field is on the Rust/daemon/MCP/CLI wire but the React dashboard panel is not yet updated. Native graph writer events still need canonical status observations, and cold/restart/stall/blocked/recovered integration acceptance remains unfinished. Exact-head CI was queued when inspected.

Independent review requested on the public wire change, whether pre-seal attempt expiration correctly fences old owners, correct graph-vs-text attribution, and all fully mounted paths. #4616/#4804 reader-retention ownership is untouched; #4776/Kev remains draft and #4763 release remains read-only. Do not merge or close on this partial evidence.

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Continuing #4508 on exact head 2dc0150. Taking the CLI first-generation wait implementation and its behavioral tests: the live wait currently resets its stall timer merely because rebuild_in_flight/Restoring/Verifying stays true, even when all producer counters are frozen. I am replacing that activity-as-progress inference with observed work tracking, retaining typed refusals and leaving daemon work running when the client stops waiting. Maintenance/sweep owners: preserve these edits and all existing commits; no duplicate/rewrite/force update. Native graph callback/status integration and Bazel-generated dashboard contracts remain separate unfinished acceptance, not made complete by this slice. #4804 owns sealed receipt/retention hunks, #4796 owns graph source reader work; neither is changed by this CLI slice. Refs #4508 only; #4776 remains held draft and #4763 read-only.

Copy link
Copy Markdown
Owner Author

Refs #4508 — preserving the active #4800 branch by expected-head non-force fast-forwards only.

Published after 2dc0150:

  • 5e2c877: init --wait no longer treats an unchanged capture/parse or actually reported text-commit epoch as an indefinitely healthy first build merely because a scheduler owner is in flight. The window advances on real producer observations or a new owner, not on status polling; graph publication without a native freshness authority remains unreported instead of falsely stalled. Three nonzero, deterministic-clock tests.
  • 4a51e51: scope that stall decision to observed phases; relationship preparation, index creation, verification, and graph publication still lack corresponding reliable work markers and are NOT declared stalled from phase labels.
  • 126b0eb: fix the independent review finding on retained/restarted capture. Retained full-source verification now mints a separately typed pre-seal attempt, forwards its observer into real capture/parser work, and retains the RAII authority through any rebuild/publication, fencing it on every return. Added a real cold-then-reopen capture regression asserting nonzero source work, timestamp, and guard shutdown.
  • c56d895: preserve actual attempt observations even when restore/verification are (correctly) excluded from the canonical rebuild_in_flight flag.

Independent review flag for omitted dashboard generated decoder is valid and unresolved; Rust-owned contracts must be regenerated and checked using Bazel, not manually patched. The prior immutable 88-character commit subject in de2d613 still blocks repository commitlint; under the explicit no-rewrite/no-duplicate/no-bypass instruction it cannot be repaired by another additive commit. No Bazel/Bazelisk or network checkout here, so no local Rust build/test/codegen passes are claimed. Exact-head CI on c56d895: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/42463162959 (queued at this check).

Native graph page/compact-segment writes are currently real low-level production events, but their typed owner-scoped daemon -> CLI/MCP/dashboard freshness propagation and the cold/restart/stall/blocked/recovered end-to-end qualification remain unfinished. Keep #4508 open, keep this PR unmerged pending proven acceptance. Please review the new retained-source guard and CLI stall behavior against any concurrent sweep edits; protected draft #4776 and release #4763 remain untouched.

Refs #4508

Import the production phase type and require actual capture, parse or
text counters to advance before renewing init --wait's silence window.
Keep independent producer watermarks and reject retired-owner reports.
Empty retries, phase-only changes, blocked/recovered metadata and adopted
restart counters are not new work. Preserve uninstrumented graph phases
as unknown rather than reporting a fabricated writer stall.

Add six nonzero behavioral regressions, including three tests through the
real wait loop, and strengthen the existing empty-retry expectation.
No Bazel execution or complete cold/restart acceptance is claimed.
devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Published bc417b9 with a non-force expected-head fast-forward from c56d895. Preserved all four concurrent commits (including retained-capture guard ownership), and changed only crates/tracedecay-cli/src/commands/index.rs. The uploaded blob exactly matches the locally reviewed 1699838bfaf5b4d1ed61cdf3599a9d665beafb2a.

Production repair: the concurrent CLI code used CodeIndexBuildPhaseV1 without a production import; that import is restored. More importantly, comparing the whole epoch/timestamp marker renewed init --wait on metadata alone. The wait now advances on real capture/parse counts or text-work counters, retaining independent watermarks and rejecting retired-daemon reports. Empty retries, phase entry with zero work, blocked/recovered metadata and adopting unchanged counters after restart do not renew the deadline. Actual resumed work does. Uninstrumented native graph/index/verification phases remain unknown, not fabricated writer-stall evidence.

Six new regressions use nonzero observations. Three drive the actual hold_until_first_generation loop through canonical serialized status: frozen counters with newer epoch/time must refuse before an arranged later Ready; empty retries after captured work must refuse; actual capture -> parse -> text commits must keep waiting until the reached proof. Three exercise deterministic clock boundaries for blocked/recovered metadata, delayed capture callbacks, and daemon restart adoption. The peer retry test is strengthened to require positive resumed work, not merely a new attempt ID. Existing tests and timeout budgets are preserved.

Independent review requested on bc417b9, specifically work-vs-publication epochs, the coverage gaps, restart adoption and all six new tests. Required native target: bazel test //crates/tracedecay-cli:tracedecay_unit_test --test_output=errors. No Rust build/test/check/codegen ran locally: Bazel is absent and checkout DNS failed; no Cargo or alternate Rust checker was used. No native pass or end-to-end daemon/restart proof is claimed. This remains Refs #4508, not completion: native write observations still need typed status propagation, generated dashboard contracts need Bazel regeneration, and full cold/interrupted/restarted acceptance remains outstanding. The immutable de2d613 commitlint blocker remains; history and gate policy were not changed. #4776 and #4763 untouched.

Refs #4508

Record actual completed compact builder windows during cold native graph
publication. Distinguish accepted in-memory rows from durable page and
verified segment commits; do not emit status-poll heartbeats.

Add a nonzero real-manifest regression covering monotonic entity/relation
windows and cancellation without a fabricated completed window.

This remains only operator tracing; owner-scoped status propagation,
Bazel verification and full cold/restart acceptance are still required.
devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published commit 2c909d34c479 on the existing #4800 branch by expected-head bc417b9 → 2c909d3, force=false. Remote ref and PR head independently verified; all 16 previous commits and peer modifications are retained. No new PR or history rewrite.

Actual native direct-seal producer improvement: the cold dependency-free manifest writer now reports bounded, nonzero graph_native_compact_rows_encoded tracing boundaries from inside the real compact builder, immediately after each successfully accepted entity/relation row window. Each carries generation, kind, completed/total row counts and durable=false. This is not a timer, read-side phase projection, fake native page or committed-segment claim. The existing authenticated install/reopen event still separately reports the single durable verified compact segment. It addresses the previously silent in-memory portion of the real direct-seal path, not just the fallback staging path.

A new production-writer regression exercises a real 1,101-entity/1,143-relation manifest on a one-worker pool, requires multiple monotone accepted windows and exact final counts, and injects an actual cancellation in the second entity window to require only the first 512-row observation (no relation/full-generation fabrication). Only crates/tracedecay-graph-db/src/sealed_store.rs changed (134 additions). Existing tests remain unchanged.

Not acceptance yet: these real writer events are still operator tracing, not an owner-scoped daemon freshness authority; init --wait, MCP, and dashboard cannot consume the native row milestones yet. Further typed callback and durable graph page/segment status integration is required, plus Bazel-only dashboard contract generation and the full cold/interrupt/restart/blocked/recovered E2E. No local Bazel toolchain/network checkout exists in this runner, so this new regression has NOT executed, and no Rust compile, test or CI pass is claimed. Exact-new-head CI is queued and optional macOS is skipped. Keep #4508 and #4800 open and unmerged; Refs only. The historical commitlint blocker remains; do not waive it. Please independently review the real window boundaries, cancellation semantics, and the distinction between encoded and durable work. #4804/#4796/#4801, draft Kev PRs, and release #4763 remain untouched.

Refs #4508

Provide an explicit graph publication observer and emit authenticated native
page completion only after the real transaction settles. Replayed receipts
do not count as work. Preserve graph operation context across commit grants.
Add a nonzero real-persistent-stage regression. Status bridging remains open.
devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4508

Report accepted direct-seal encoding windows as non-durable physical work.
Report only installed and digest-verified native compact segments as durable.
Replay adoption and losing installs must not claim fresh writes. Add a real
writer/reopen regression and correct the test module import boundary.
Graph-status and daemon propagation remain incomplete.
Refs #4508

Bind real native graph page and compact writer events to one leased
worktree publication attempt. Preserve physical encoding separately from
durable stage-page receipts and digest-verified compact installs; carry
owner-scoped work epochs and original timestamps through daemon freshness,
CLI wait and MCP summaries. Add nonzero slot and wire regressions.

Dashboard generated contracts and full Bazel restart acceptance remain open.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published cd722ecb08a9 to the existing #4800 branch by exact-head expected_sha=d9cff3ac5202e1dbbdf98c61759bd33a654a6e08, force=false. Independently re-fetched PR + Git ref and confirmed both point to cd722ec; all previous 19 commits were preserved. One new ordinary commit, 11 source/test files, no changes to CI, release, infrastructure, or protected model work.

This fixes the previously unconsumed native graph observer port, not another read-side phase projection. During actual daemon graph publication, both the pre-decode sealed build and fallback activation now create a generation/digest/daemon-incarnation/producer-incarnation/attempt-sequence bound observer. The existing graph-db writer invokes it for real accepted compact encoding windows (physical, not durable), actually receipted native stage-page writes, and installed/reopened digest-verified native compact segments. The adapter threads the callback from the mounted worktree through CodeGraphSeatLeaseV1 into the real GraphPublicationOperationContextV1. A retired/restarted attempt cannot overwrite the successor; duplicate/out-of-order/zero-work events cannot refresh the original work timestamp. Native graph progress is a separate typed authority, not text committed page counts.

The daemon lock-independent freshness reader publishes these owner-scoped native work counts/epoch/time, and the CLI init --wait and MCP graph/search summaries consume them. init --wait now measures native graph work only after an actual observed callback and cannot infer a stall from a phase label alone. New nonzero Rust regressions cover real-vs-durable attribution, replay deduplication, restarts/stale attempts, generation identity, wire roundtrip, CLI stall evidence and MCP rendering. These tests are published source, NOT Bazel-executed. No native test pass or independent human review is claimed; please review observer lifecycle, count monotonicity, and callback propagation. Exact-head CI #42474443677 was queued at inspection.

This is still PARTIAL acceptance. Rust-generated dashboard contracts/decoders and actual React display must be regenerated using Bazel (never hand edited), and a cold first-build + interruption/daemon restart + blocked/recovered + stalled/writer-silent end-to-end run is still required. The preexisting immutable 88-character de2d613 commitlint ancestry failure remains a mandatory gate and cannot be corrected by an additive commit without violating no-rewrite/no-duplicate restrictions. Do not merge or close #4508. #4776/#4797 remain draft; release #4763 untouched.

devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4508

Once actual direct-seal encoding windows complete, the single native
compact-container write and digest verification have no intermediate
progress callbacks. Mark that window unknown rather than timing it out
as a stalled publisher. Prefer active native graph work in status output;
add a nonzero regression for the final uninstrumented writer phase.

Bazel and full daemon restart acceptance remain unverified.
devin-ai-integration[bot]

This comment was marked as resolved.

Refs #4508

Carry the immutable manifest relation count on every accepted compact
row window, including an actual zero count. The owner-scoped progress
snapshot therefore knows relation work remains after entities finish.
Do not suspend init --wait stall detection before the first relation
window. Preserve unknown final compact container writes as uninstrumented.

Strengthen real native writer and progress-slot tests and add an
observable frozen-relation-window regression. Bazel tests are pending.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published ab3334f8a487 to this existing PR with expected_sha=f9b7d038409fec1e302cd5580dbb558e3c3e179a, force=false. Independently verified the PR and branch ref both point to the new commit; all 21 preceding commits, including concurrent native graph observer integration, are preserved. Exactly four production/test files changed; no new PR or rewritten history.

Concrete native progress bug fixed: review thread relation encoding stalls go undetected correctly identified that a completed entity stream with total_relations=None was mistaken for complete compact encoding. Before its first relation window, a genuinely stalled encoder could therefore make init --wait suppress the work marker indefinitely. The actual direct-seal writer now forwards the immutable manifest's relation bound on each accepted entity/relation row window, including explicit zero. The generation/owner-scoped native slot validates this bound and rejects conflicting later callbacks. The CLI enters the uninstrumented final Grafeo compact write only when both actual row streams have known complete totals, not when the relation total is missing.

Nonzero regressions published: the real native sealed-container observer test requires the bound 9 on both entity and relation observations; the native progress-slot test requires a known 1,143-relation bound even while zero relations are encoded and refuses a mismatching subsequent 9,000-relation callback without renewing progress; a new deterministic init --wait test checks that 1,024 completed entities with 10,000 still-pending relations remain stall-observable at 120 seconds and that fully completed or genuinely zero-relation encoding enters the unknown final-write phase.

These are source assertions, not executed Rust tests. No local Bazel/Bazelisk or checkout-capable DNS is available; no Cargo or stand-alone Rust toolchain was used. New exact-head CI run 42475944328 was queued when checked; the preceding workflow was cancelled, not green. Please independently review the source-bound propagation, zero-relation behavior, and race/stall semantics. Refs #4508 only. Full generated dashboard contracts, end-to-end cold/restart/blocked-recovered and verified nonzero Bazel acceptance are still outstanding. The existing immutable 88-character de2d613 commitlint ancestry blocker is unchanged; no rewriting or bypass. Keep #4508 open, #4800 unmerged; #4776/#4797 remain DRAFT and #4763 read-only.

Refs #4508

The real sealed-graph writer uses a fresh one-shot commit context after
journal classification. Preserve its generation-bound native observer
there instead of silently discarding compact encoding, durable stage-page,
and verified segment observations. Keep the distinct commit probe/grant.

Exercise the actual persistent sealed publication path with nonzero native
encoding and a verified compact segment, then verify idempotent replay
emits no new native work. This is not cold daemon/restart acceptance.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published acce42f2b68f4a15be742e4c5d92e68bb66730f0 on the existing #4800 branch via expected-head ab3334f8… and force=false. PR head and branch ref independently confirm all 22 prior commits were preserved. Exact diff: three files, in the store/runtime graph publication boundary and its existing persistent-graph test suite.

Production root cause: the original publish_verified_snapshot_with_admission context already received the scheduler’s native observer, but the actual native writer in publish_prepared_sealed_generation creates a different one-shot GraphPublicationOperationContextV1 for the verified commit. That nested context silently discarded the observer. The direct-seal row encoding, fallback StagePageApplied receipts, and digest-verified compact segment callbacks therefore had no recipient on the real journaled commit path despite the earlier port and scheduler wiring. The nested commit context now explicitly inherits only the original observer, keeping its separate cancellation/deadline/one-shot commit grant intact. Both native staging preparation and final publication now use that context.

Falsifiable production regression: sealed_graph_writer_reports_native_work_through_journal_commit_boundary uses a real Git-sealed generation, retained project GraphDB runtime and actual journaled publication. It requires nonzero entity encoding callbacks plus exactly one positive-byte installed-and-digest-verified compact segment, then republishes the same verified head and requires no additional native write callbacks. The pre-fix nested context delivers zero callbacks, so this test would fail before this commit. Existing tests were not weakened.

Validation status: source and exact commit ancestry reviewed; no Bazel/Rust build, test, format, or codegen ran locally because the container has no Bazel/Bazelisk and cannot resolve the checkout host. Exact-head hosted CI #42481271346 was queued at inspection; no pass is claimed. Independent PR-maintenance review requested especially for second-context observer inheritance and exact native-event behavior. Do not merge or close: dashboard generated contracts/React display, real daemon cold/restart/stall/blocked recovery E2E, current-head nonzero Bazel/native results, and the unchanged historical commitlint blocker are outstanding. Keep #4776/#4797 draft; release #4763 read-only.

Refs #4508

An initial sealed graph publication can emit a verified native segment,
then a decoded-graph activation reopens the same head without writing.
Preserve the original real work and timestamp until activation produces a
new physical write; reject stale and invalid callbacks on both paths.

Exercise the real daemon freshness/search journey through cold generation,
graceful restart, and a new post-restart generation. This is not crash or
interruption-resume acceptance. Bazel execution and dashboard generation
remain required.
Refs #4508

An active native graph attempt with zero actual work used to fall through
to an uninstrumented phase and clear the wait silence window forever.
Observe its owner as an uncredited zero-work baseline, timing out sustained
silence while preserving concurrent real text progress and the monolithic
compact-write exception. Add nonzero serialized status wait and deterministic
work-handoff regressions. Move the reported inline import to module scope.

Bazel/native execution remains unverified.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published c7d7de802993 to existing PR #4800 by expected-old-head 8acb055ade70 and force=false. Verified branch ref and PR head both equal the new commit; all 24 earlier commits, including concurrent cold/restart graph evidence, are preserved. Exact delta: one CLI production/test file, 115 added / 7 removed. No competing issue PR opened.

New production no-progress defect repaired: an active native graph attempt whose first encoded-row window or durable staging page never completed had last_progress_micros=None. The wait previously fell through to an uninstrumented GraphPublication phase, returned no marker on every poll, and reset its silence timer indefinitely. Now the native attempt's typed owner and zero genuine counters establish an uncredited baseline at the first status observation; continued silence for the configured 120s causes the existing typed wait refusal. No fake write, timestamp, completed counter, phase heartbeat, or read-side update is emitted. While native work is still unreported, actual concurrently committed text work remains eligible to renew the window. Already-finished native encoding still enters the uninstrumented single-container write without a false timeout.

Falsifiable regressions: a real serialized hold_until_first_generation status loop refuses a zero-write native attempt despite changing epochs before a later scripted ready response; a separate deterministic 120s test asserts that parallel real text commits are not hidden, and that a genuine first native encoding window renews the timer but later frozen counters do not. The former fails before this change. Also moved the existing inline pre-seal enum import to module scope per the unresolved repository review. Existing regressions unchanged.

Not verified: No local Bazel/Bazelisk or checkout-capable GitHub DNS path. New source/tests have not compiled or run; exact-head hosted CI is queued. Please independently review first-native-work liveness, the text/native handoff, and actual nonzero //crates/tracedecay-cli:tracedecay_unit_test execution. Do not infer a Rust pass from a queued/skipped/zero-test job.

Still open: canonical Bazel-generated dashboard schema/SDK/types/decoders and real React rendering, full physical daemon kill/resume/blocked-recovered acceptance, exact-head native tests and independent approval. Immutable ancestor de2d613 commitlint failure remains a separate known non-bypassable merge blocker under no-rewrite constraints. Keep PR/issue open and Refs #4508; draft #4776/#4797 and read-only release #4763 untouched.

Refs #4508

Exercise a real paused first-generation parse through physical daemon
status, hard-kill before any seal, then restart and require a verified
native graph segment and a readable newly sealed generation.
Polling while held must not fabricate a progress epoch. Bazel execution
and broader interrupted-stage acceptance remain outstanding.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published fe6a66c3517d on the existing #4800 branch, fast-forwarded with expected_sha=c7d7de80299327788d71bc644314387c08c342d1 and force=false. Exact branch and PR heads independently reconfirmed; all 25 predecessor commits preserved. One file changed, crates/tracedecay/tests/daemon_suite/indexing_lifecycle_test.rs (+141/-0); existing lifecycle tests untouched.

Physical cold-crash/resume regression (source published, not executed): New hard_killed_cold_preseal_attempt_recovers_a_verified_native_graph uses the existing production daemon fixture and test-helper hold immediately after actual cold extraction. It waits for positive parser-evidence work through the real daemon tracedecay_status transport, requires an active separately scoped pre-seal attempt, genuine timestamp/epoch and no invented sealed generation/text or native graph commit, and checks a second status read does not refresh a paused producer. It then actually SIGKILLs the daemon before the first seal, starts a new physical daemon against the same durable profile/Git tree without the hold, requires ready searchable source plus a newly sealed on-disk generation, and polls for exactly one real positive-byte verified compact native segment with a retired writer attempt. This complements rather than replaces the existing graceful cancellation and ready-generation hard-restart journey. No mock status transition and no synthetic writer progress.

Verification limit: The integration regression has not been run under Bazel. The authoring runtime has no Bazel/Bazelisk or checkout-capable network path. Exact-head CI #42491276147 is queued; no Rust, native macOS or end-to-end passing result is claimed. Please independently review the new test's physical hold/restart setup and whether the pending native graph attempt survives the correct readiness boundary. The preexisting de2d613 historical commitlint failure remains and is not bypassed.

Still not #4508 completion: dashboard generated schema/types/decoders + real React progress UI need Bazel regeneration; full production blocked/recovered and all-phase stall/restart acceptance, nonzero exact-head tests, and human review remain outstanding. Refs #4508, issue/PR open and unmerged. Held #4776/#4797 and release #4763 untouched.

Refs #4508

Exercise the real persistent native graph stage, cancelling after the
first receipted entity page and reopening the store. Require only the
newly committed relation page to emit a write event, then verify the
recovered graph digest and silent idempotent replay. Import native work
types explicitly in the graph-db test module.

Copy link
Copy Markdown
Owner Author

Published fec033760b76471313822d51534895f0efe54732 on existing #4800 by an expected-head (fe6a66c) non-force fast-forward; all 26 prior commits preserved. Refs #4508 only.

This is a concrete native-writer restart acceptance repair in crates/tracedecay-graph-db/src/generation_runtime.rs, not a renderer/slot-only check. The graph-db test module's prior real native observer regression also referenced GraphNativeWorkV1 without importing that type; the commit adds explicit module-level native-work imports. New regression interrupted_native_stage_resumes_receipts_without_fabricating_writes builds a real two-page manifest (5,000 entity rows plus a relation), uses the actual persistent WAL-sync graph writer, interrupts immediately after its first receipted entity page's callback, closes/reopens the graph store, resumes and requires only the new relation page to be reported. It then performs a recovered-digest proof and insists a completed replay returns Reseated with no additional native write events. This would fail if callbacks were synthetic, lost across cancellation, or fired on receipt replay.

Validation: regression source is published, NOT Bazel-executed locally; this runtime still has no Bazel/Bazelisk or checkout-capable GitHub DNS. Exact-head hosted CI was queued at inspection; no native pass, formatting pass, independent approval or full #4508 acceptance is claimed. Please independently inspect post-commit callback semantics, WAL reopen, and two-stage replay correctness. Important remaining acceptance: Bazel-generated dashboard schema/decoder/SDK, React preseal/native graph progress, physical interrupted/recovered end-to-end, blocked/recovered status, and nonzero exact-head Bazel/native CI. Immutable commitlint ancestry blocker is unchanged; no rewrite/bypass/duplicate PR, and no protected #4776/#4797 or release #4763 modifications.

Refs #4508

Retain actual native progress on query-driven graph activation by binding
its observer to the mounted scheduler's incarnation and sealed graph digest.
Preserve the distinct write-attempt boundary through activation.

Remove unused legacy staging and sealed-container wrappers; call the real
observed production implementations from existing tests. Fix the missing
stage plan observer initializer and both updated graph activation callers.
No assertions, checks, or validation gates are weakened.

Copy link
Copy Markdown
Owner Author

Refs #4508 — published production/CI repair bb3f090dd246 on existing PR #4800. The update was an expected-head fec033760b76471313822d51534895f0efe54732, force=false fast-forward. I independently verified PR and branch both now point to bb3f090 (28 total commits); no other work was replaced.

Fresh hosted Bazel evidence that required repair: On the previous exact fec0337 CI run 42492448575, Clippy job 128773228465 and Linux fleet job 128773228543 executed and failed before Rust tests:

  • generation_runtime.rs:4062 was missing the new mandatory GenerationStagePlan.observe field (E0063).
  • ignored_dependencies.rs:904 and tests/code_index_scheduler/reconcile.rs:2672 still invoked activate without its new native-work observer (E0061).
  • apply_generation_unverified_with_digest_observed and build_sealed_container were obsolete wrappers that Clippy flagged as dead code under -D warnings.

Published solution: Remove both redundant writer wrappers without allow(dead_code); retain stage tracing on the actual observed production writer. Route all prior test calls through the work-aware production methods, explicitly passing None, preserving their original assertions. Supply observe: None in the stage-plan test. Pass an actual generation/digest/daemon+producer incarnation-scoped native graph observer through query-driven ignored-dependency activation instead of a silent no-op; the memory-only activation test passes an inert callback because it has no physical writer. The new ignored-dependency guard uses the scheduler's existing bound incarnation after its durable commit and fences that exact owner on scope exit. Five existing source/test paths changed, with no protocol/CI policy/limits changes.

This is a compilation-path repair and real production observer propagation, not proof of full #4508 acceptance. No local Bazel/Rust tests/codegen were executed (no Bazel checkout/runtime here); current-head CI #42504825853 is pending. Please independently review native progress ownership in ignored-dependency activation, retention of real graph writer callbacks, the removed facades, and the nonzero test execution once CI completes. The historical immutable de2d613 commitlint gate still blocks merging under the no-history-rewrite restriction. Generated dashboard contracts/React status and complete physical daemon stall/interruption/restart acceptance remain outstanding. Keep PR/issue open, Refs #4508 only. Protected Kev drafts #4776/#4797 and release #4763 untouched.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant