Repository navigation
DDIR: batch reduction equality over borrowed columns - #943
Merged
frankmcsherry merged 6 commits intoOct 18, 2026
Merged
frankmcsherry merged 6 commits into
frankmcsherry merged 6 commits into
Conversation
Presentation assigns each real key's values structurally ordered proxies; the proxy harness sorts and consolidates each bracket by proxy. Min can select its first nonzero proxy, and Collect can expand positive copies in that established order, avoiding another candidate gather and sort. Reconcile output in the current key's tail of the corrections buffer, removing a temporary Vec per key. Document the bracket-order guarantee and compare Vec/Corgi results across signed, ragged inputs and retractions.
Collect key-run endpoint checks across a reduction window and send their coordinates to Corgi in one batch. Likewise batch output equality for non-colliding keys with at most one desired and one prior output. Neither path gathers payload columns solely for comparison. Keep the existing general reconciliation for multi-output or colliding keys, avoiding a Cartesian expansion of comparisons. Skip batch setup when there is no old output. Add a compound-key/list regression across multiple windows and retractions. Pin the Corgi borrowed-pair API on corgi-batched-equality (WIP PR TimelyDataflow#105). This draft builds on the reduction cleanup in differential PR TimelyDataflow#942.
This was referenced Oct 17, 2026
frankmcsherry
marked this pull request as ready for review
October 17, 2026 19:56
Move the iterator-based endpoint check into corgi/chunk.rs and adapt reduction's window matches to it. Keep the primitive-key bypass. The join PR carries the identical shared helper so each PR builds independently; merging both retains one implementation. Pin the stable-order borrowed equality implementation from WIP TimelyDataflow#105.
This was referenced Oct 18, 2026
…lity # Conflicts: # interactive/src/corgi/reduce.rs # interactive/tests/ordered_reduce.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduction checks real-key run endpoints across each window and compares the common case with at most one desired and one prior output per non-colliding key. Group the borrowed row indices by source-column pair and call the existing Corgi indexed equality kernel once per group. Omit self/duplicate endpoint checks and the redundant hash lane. General multi-output/collision reconciliation keeps its existing fallback.
Builds on #942. The key-run validator in
corgi/chunk.rsis identical in #943 and #944, so their merge keeps one copy. Pins merged WIP master 2c9a8ce from Corgi #105. There is no pooled Corgi row-coordinate API or scalar-comparator rewrite.Fresh matched controls compare scalar DD, the previous pooled API, and this direct indexed implementation:
M4/macOS 15.6/Rust 1.95, release/LTO, mimalloc, one worker, deterministic 2000-person graph with 16 posts/person. Identical resolved dependency locks; all builds include merged Corgi #106-#108 and the mixed-width fix. Three alternating process runs, nine measured cycles after warmup. Feed plus tick latency is timed; peeks and independent initial/changed/restored answer checks are outside it and all pass. These are synthetic query controls, not official LDBC results. Raw reports and benchmark sources stay outside the code diff.
Validation: 181 interactive/server tests pass, five existing ignores. The combined #943-#947 stack passes 191 tests with five existing ignores, and workspace Clippy passes with existing repository warnings. Tests cover multiple windows, unchanged/changed/retracted outputs, multi-output expressions and forged hash collisions.