Repository navigation
fix: consolidate follow-up application issue repairs - #4776
ScriptedAlchemy wants to merge 77 commits into
Conversation
Refs #4535. Read at most 1 MiB plus one byte from the selected endpoint. Refuse oversized advertised responses and streaming responses without a length before parsing untrusted JSON, as well as malformed Content-Length. Avoid allocating an unbounded remote response. Cover all three refusal paths against a real loopback HTTP server. The isolated transport probe passed; hosted Bazel tests are still needed.
|
Refs #4535. The active /v1/systemone answer contract carries a type tag. Previously an untyped answer with a noul field was accepted as a successful prediction, obscuring protocol drift during model qualification. Refuse missing or mismatched tags before persisting rows or raw samples. Update test server doubles and cover missing-tag denial with partial-log preservation.
Refs #4535. Reject duplicate object keys at every JSON depth before an inference result can enter the resumable evidence log or sample. The loopback regression covers both duplicate answer maps and duplicate Noul values, followed by a successful valid resume.
Refs #4535. Reject duplicate object fields and non-JSON numeric constants in dataset and server manifests, partial resumes, and scored results. Preserve completed summaries and resumable evidence on refusal.
Refuse stale or swapped raw samples on new runs and resumed partials. Record the raw response digest in completed rows; verify sampled experiment, case, request and response before further inference. Keep one capture for the first configuration and reject competing publication without replacing foreign evidence. Refs #4535
Refuse oversized partial headers, rows and raw sample captures before unbounded JSON reads, and preserve resumable evidence unchanged on denial. Add behavioral regressions for oversized partial and sample files followed by successful exact resumption. Refs #4535
Use nearest-rank selection so sparse runs cannot understate their slowest observed decision at p90/p95. Include direct metric and CLI regressions without extrapolating unmeasured latencies. Refs #4535
Preserve resumable evidence and first-response captures when a model replies with non-object timing metadata; reject malformed subset containers before network inference or partial-file creation. Add behavioral recovery tests. Refs #4535
A partial run can already contain the first response when --sample-out is added on retry. Refuse rather than label a later response as the experiment's first raw evidence. Preserve durable partial rows and permit an ordinary resume without a capture. Add a behavioral interruption regression. Refs #4535
Refs #4535 Reject sample destinations under results before inference. Otherwise a raw sample may be misread as a completed run or occupy its final path. Exercise direct, normalized, nested collisions and a valid external sample.
Refuse finite-looking JSON exponents that Python decodes to infinity in untrusted datasets, resumable rows and scored result files. Preserve the last durable evidence/summary instead of admitting nonfinite extra data. Regression coverage includes pre-network dataset rejection, read-only resume rejection and preservation of a previously scored summary. Refs #4535
A removed first-response sample must not be silently recreated from a later case if it disappears while the second HTTP inference is in flight. Preserve the settled first row and allow recovery without a raw sample. Refs #4535
Removing the Plan 38 reference left a target reading as a current fact. `Storage::maintain` does not exist in the code, and an over-budget store only produces a Doctor finding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ge-design-work docs: remove stale code storage design passages
Retain fail-closed Codex executable admission while making scheduler/manual and lock-order tests use an actual configured executable identity. Add a separate regression for the typed unconfigured-backend diagnostic. Resolve three automation_runner_test failures observed on master CI.
Retain first-response evidence during malformed resume tests, supply realistic HTTP headers for the wire fingerprint test, and invoke the correctly named interrupted-capture regression callback. Integrate the already merged lucide-react lockfile update.
Reject resumed HTTP rows and scored experiment files that omit or malform their recorded raw response digest. An incomplete evidence row must not be silently treated as a verified model observation. Cover rejection before reinference, scored-summary preservation and exact recovery once the durable row is restored. Refs #4535
|
Master-merge hold under the owner's latest instruction: “Merge any passing pr after you review and fix them. Don’t merge the decision model ones like kev.” This integration still contains decision-model/HTTP-evaluation changes (including current head 9cf67ec), so it is excluded from merging into master even if CI becomes green. Review and corrective commits remain authorized. Preserve all contributor work; do not force-rewrite, remove excluded changes, reopen superseded source PRs, or create a competing application lane to bypass this hold. Both existing TraceDecay maintenance tasks now carry this exclusion with their schedules unchanged. Ready-for-review status is not merge authorization. No merge or code change was performed by this review pass. |
Count each unlinked SQLite TEMP physical file once when multiple Unix file descriptors refer to the same inode. Preserve unreadable-entry coverage and other platforms' behavior. Regress duplicate aliases that otherwise produce a false over-budget doctor finding. Refs #4152
Ignore nonregular entries if a descriptor is reused between resolving its (deleted) pathname and sampling metadata. Keep existing inode deduplication and unreadable-entry reporting. Regression covers a directory with a matching TEMP basename without changing the valid spill total. Refs #4152
Avoid opening and closing independent Unix file descriptors for verified read snapshots: their close releases unrelated POSIX SQLite record locks. Verify the canonical inode on both sides of SQLite HAS_MOVED and reject unsupported VFS proof without a raw pin. Preserve Windows pinned handles. Exercise rollback and WAL writer locks against an external process while live and immutable readers open and close, then verify committed data. Refs #4513
Require exactly one SQLite database descriptor for a verified Unix reader; keep clone and final-drop assertions while rejecting the obsolete raw-pin contract. Format the fail-closed HAS_MOVED refusal as required by rustfmt. Refs #4513
ScriptedAlchemy
left a comment
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Scoped retention review — Refs #4787
Reviewed the current cold_store.rs patch at c9ef27635e484111b77411a563cef44cc2eb0c19. This is a retention-scoped review, not an approval of the mixed 74-commit integration.
Outstanding ownership defect: checkpoint capability acquisition still reopens the profile through Dir::open_ambient_dir(profile, ...). The no-follow check protects only the maintenance child. Replacing the profile pathname before acquisition can therefore make this code adopt a different profile and publish its checkpoint there. The existing held-directory regression swaps the maintenance path after acquisition, so it does not prove the missing profile-owner invariant. The inline comment identifies the repair and regression boundary. Coordinate that shared ownership work with #4616 rather than overwriting another owner's changes.
The registry-pagination correction is sound in scope: after_project_id must come from page.next_cursor, not the filtered census entries, so a full registry page containing no profile-sharded stores can still advance. However, this JSON checkpoint does not control the separate 48-row canonical-body/replay SQL frontiers; it is not proof that the original 90–110% CPU incident is resolved.
The active issue-specific SQL work is reviewed separately in #4798, now at 91ffd680, including stored-frontier verification and rollback regressions. Do not conflate those tests or CI results with this mixed branch, and do not close #4787 based on either unmerged branch.
Keep this PR DRAFT. Never merge or enable auto-merge, regardless of CI, because it contains the owner-excluded Kev/decision-model work. This review does not authorize moving excluded work elsewhere to bypass that hold. Release #4763 remains read-only.
| .parent() | ||
| .and_then(Path::parent) | ||
| .ok_or_else(|| io::Error::other("maintenance cursor has no profile root"))?; | ||
| let root = Dir::open_ambient_dir(profile, cap_std::ambient_authority())?; |
There was a problem hiding this comment.
Outstanding P1: bind this capability to the admitted profile owner before reopening a pathname. open_ambient_dir(profile, ...) can follow a substituted profile-root path; the later open_dir_nofollow("maintenance") only protects the child. When the replacement profile already has a real maintenance directory, both operations can succeed and persist_cursor_in_dir can replace that other profile's cursor. The current regression replaces the maintenance path after obtaining the capability, so it misses this case. Derive checkpoint I/O from the existing admitted owner/held ancestor capability, or verify that authority without a path-check/open race. Add behavioral tests for profile-path substitution before acquisition with both symlink and real-directory replacement, asserting foreign checkpoint bytes and entries remain unchanged. Preserve intentionally symlinked layouts through explicit ownership rather than a blanket symlink ban; coordinate the owner-identity boundary with #4616. This remains separate from the SQL-frontier CPU diagnosis.
ScriptedAlchemy
left a comment
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Scoped SQLite TEMP review — Refs #4152
Reviewed only the projection-rebuild TEMP-removal slice at c9ef276, not the entire mixed integration. The added test catches retention of the two named TEMP tables, but it does not exercise the destructive ownership predicates on populated data; the inline comment specifies the missing data-preservation regression.
The successful workflow 42429786605 is not Rust behavioral evidence for this slice: its Linux, Windows, macOS and Clippy jobs were skipped. Require nonzero execution of the relevant regression and affected tests through Bazel; do not use Cargo, relax assertions, or change CI to conceal the gap. No Rust test pass is claimed by this review.
#4802 separately removes the preflight TEMP table and now has an inline review identifying a stored-metadata fixture mismatch. Its result must not be conflated with this branch's clear-operation coverage. Neither slice proves the original memory-page-corruption cause or permits closing #4152 before acceptance is established in master.
Keep #4776 DRAFT, with no merge or auto-merge, regardless of green checks. The older PR-body instruction to keep it ready for review is superseded by the owner's explicit hold. Do not move held changes elsewhere to bypass it. This review leaves the other issue owners' edits and release #4763 untouched.
| let harness = RegisteredGlobalDbHarness::open("projection-clear-no-temp-cache").await; | ||
| let transaction = harness.registered.begin_write_transaction().await.unwrap(); | ||
|
|
||
| clear_active_projection(&transaction, "generation.no-temp") |
There was a problem hiding this comment.
[P2] Exercise populated ownership state before treating this as a safe-reclamation regression. This calls clear_active_projection on a fresh store without inserting any message, provenance, or staged-output rows. It usefully catches the old named TEMP tables, but a broken replacement that deletes every durable message, drops the foreign-owner exclusion, or drops the staged-output exclusion would still pass. Seed committed messages through the existing production fixture/ownership APIs and distinguish an exclusively created output eligible for removal, a shared/foreign-owned output, a staged output, and an unowned or non-created output. After clearing, assert the exact surviving message bodies/locations and provenance, then verify an explicit rollback preserves the previously committed state and the database passes integrity checks. Keep the same-connection TEMP assertion as an additional check. The disposable SQL comparison described on #4152 is supporting evidence, not a substitute for this registered-storage behavioral regression.
Refs #4535 Read native Kev's reported state-token and used-token counts with its truncated flag instead of assuming state_tokens_dropped is exposed. Reject contradictory telemetry, retain original counters, and reconstruct dropped tokens during qualified scoring and exact partial resume. Missing counters remain unknown rather than measured zero. Add falsifiable runner/scorer/partial-resume regression cases, including truncated and inconsistent responses and rehashed normalized drop tampering. No model inference, local Bazel validation or replacement claim is made. Keep #4797 and mixed #4776 DRAFT — NEVER MERGE.
With one projected output in 100, 10,000, and 100,000 raw messages, SQLite VM steps for the ownership/delete query grew from 1,890 to 1,800,090. The paged query executes 264 steps at each size and deletes only the owned output.
Consolidated application follow-up
The prior application integration #4772 merged into
masteron November 5, 2026. This is the single new application integration PR for subsequent issue fixes. Add focused commits here rather than opening separate issue PRs/CI lanes; preserve concurrent changes with exact-head, nonforce updates.Included now
Content-Lengthbefore reading, and also reject an oversized body with no length header.Verification and gates
An independent local Python HTTP transport probe passed for a valid response, an oversized declared response and an oversized streaming response. No repository Bazel test pass is claimed: the workspace checkout and Bazel toolchain are unavailable in this execution environment. Run
bazel test //evals/intent_engine:http_runner_test --test_output=errorsand all applicable integration CI on this PR's exact head, including review of the response cap against real service contracts.This is partial progress on #4535: canonical Scout evidence export, independent labels, actual Kev inference, runtime parity and model qualification are still outstanding. Do not close #4535 now. No release, production, dependency or infrastructure changes are included.
Keep this PR ready for review. CI remediation and gated master merging remain the responsibility of the existing TraceDecay PR-maintenance workflow.