Repository navigation
DDIR: reuse ordered reduction proxies and correction storage - #942
Merged
frankmcsherry merged 1 commit intoOct 18, 2026
Merged
Conversation
Presentation assigns each real key's values structurally ordered proxies; the proxy harness sorts and consolidates each bracket by proxy. Min can select its first nonzero proxy, and Collect can expand positive copies in that established order, avoiding another candidate gather and sort. Reconcile output in the current key's tail of the corrections buffer, removing a temporary Vec per key. Document the bracket-order guarantee and compare Vec/Corgi results across signed, ragged inputs and retractions.
This was referenced Oct 17, 2026
frankmcsherry
added a commit
that referenced
this pull request
Oct 18, 2026
* DDIR: reuse ordered reduction proxies and correction storage Presentation assigns each real key's values structurally ordered proxies; the proxy harness sorts and consolidates each bracket by proxy. Min can select its first nonzero proxy, and Collect can expand positive copies in that established order, avoiding another candidate gather and sort. Reconcile output in the current key's tail of the corrections buffer, removing a temporary Vec per key. Document the bracket-order guarantee and compare Vec/Corgi results across signed, ragged inputs and retractions. * DDIR: batch reduction equality over borrowed columns Collect key-run endpoint checks across a reduction window and send their coordinates to Corgi in one batch. Likewise batch output equality for non-colliding keys with at most one desired and one prior output. Neither path gathers payload columns solely for comparison. Keep the existing general reconciliation for multi-output or colliding keys, avoiding a Cartesian expansion of comparisons. Skip batch setup when there is no old output. Add a compound-key/list regression across multiple windows and retractions. Pin the Corgi borrowed-pair API on corgi-batched-equality (WIP PR #105). This draft builds on the reduction cleanup in differential PR #942. * DDIR: share batched key-run validation with join Move the iterator-based endpoint check into corgi/chunk.rs and adapt reduction's window matches to it. Keep the primitive-key bypass. The join PR carries the identical shared helper so each PR builds independently; merging both retains one implementation. Pin the stable-order borrowed equality implementation from WIP #105. * DDIR: batch comparisons directly by source column pair * DDIR: pin merged indexed Corgi comparisons
frankmcsherry
added a commit
that referenced
this pull request
Oct 18, 2026
* DDIR: reuse ordered reduction proxies and correction storage Presentation assigns each real key's values structurally ordered proxies; the proxy harness sorts and consolidates each bracket by proxy. Min can select its first nonzero proxy, and Collect can expand positive copies in that established order, avoiding another candidate gather and sort. Reconcile output in the current key's tail of the corrections buffer, removing a temporary Vec per key. Document the bracket-order guarantee and compare Vec/Corgi results across signed, ragged inputs and retractions. * DDIR: batch reduction equality over borrowed columns Collect key-run endpoint checks across a reduction window and send their coordinates to Corgi in one batch. Likewise batch output equality for non-colliding keys with at most one desired and one prior output. Neither path gathers payload columns solely for comparison. Keep the existing general reconciliation for multi-output or colliding keys, avoiding a Cartesian expansion of comparisons. Skip batch setup when there is no old output. Add a compound-key/list regression across multiple windows and retractions. Pin the Corgi borrowed-pair API on corgi-batched-equality (WIP PR #105). This draft builds on the reduction cleanup in differential PR #942. * DDIR: share batched key-run validation with join Move the iterator-based endpoint check into corgi/chunk.rs and adapt reduction's window matches to it. Keep the primitive-key bypass. The join PR carries the identical shared helper so each PR builds independently; merging both retains one implementation. Pin the stable-order borrowed equality implementation from WIP #105. * DDIR: release unused desired reduction columns * DDIR: batch comparisons directly by source column pair * DDIR: pin merged indexed Corgi comparisons
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Presentation already assigns each real key's values structurally ordered proxy IDs, and the proxy harness sorts and consolidates each input/output bracket by proxy. Min and Collect were gathering and ordering those candidates again. Select the first nonzero proxy for Min and expand positive copies in the established order for Collect. Document the bracket-order guarantee on
ProxyReduceBackend.Reconcile directly in the current key's tail of the shared corrections vector, removing a temporary
Vecper key. Existing borrowed scalar value-equality checks are unchanged. No Corgi dependency or comparator changes are included.The regression compares Vec and Corgi across four epochs with signed/ragged values, negative multiplicities, list prefixes, retractions and restoration. It preserves Min's nonzero-input semantics and the difference between Collect's all-negative group (an empty list) and a fully retracted group (no output). Existing backend tests cover real-key hash collisions.
An earlier isolated reconciliation control with 50,000 keys and eight values per key measured the following existing-output cases. Both sides held the same separate allocation-free Corgi comparator fixed; these timings do not describe this branch's unchanged dependency pin in isolation.
Apple M4, System allocator, scalar
u64time; three interleaved runs with eleven timings each and separate allocation counting. Presentation and result verification are outside the timer. No whole-query gain from this DDIR change alone is established. A gathered-equality experiment was slower with cheap scalar comparison and is excluded.Validation:
cargo test -p interactive --lib --test ordered_reduce --test corgi_backend --test reduce_termandgit diff --checkpass.Based directly on current
master-next(0b41ba7); independent of join scratch reuse. The diff contains the reducer, its backend-contract documentation and one integration test. Benchmark programs and raw reports remain in the local audit, outside this PR.