Repository navigation
Conversation
This was referenced Oct 26, 2026
…l k (B * k lookups -> k) A type with B buckets has B vector indexes; the search loaded and wrapped every hit of every bucket and kept k. GraphCalls.topHits merges the hit lists by score (stable, as the Python sort was), cuts to k, and loads those records in one crossing. Same records, order and scores; a missing record is still named, but now only when it is among the final k. New test file pins the result against a merge of the raw per-bucket lists and counts k wraps and k lookups (the no-bridge-jar fallback too). Port of humemai/arcadedb-embedded-python#534. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
force-pushed
the
port/find-nearest-top-k-first
branch
from
October 29, 2026 12:20
09b90ba to
a352125
Compare
Collaborator
Author
|
Rebased onto main |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
VectorIndex.find_neareston a type with B buckets loaded and wrapped every hit of every bucket index (one record lookup and one Python wrapper per hit, B * k of them) and then sorted and kept k. This merges the per-bucket hit lists by score first and loads only the final k: a new bridge callGraphCalls.topHits(db, pairLists, k)takes the Java hit lists, sorts the candidates by score with a stable sort (equal scores keep the bucket order, as the Python sort of the concatenated hits did), cuts to k, and returns the k records, scores and RIDs in one crossing. Without the bridge jar the Python fallback does the same, reading each candidate's score and looking up only the k winners. This is the port of humemai/arcadedb-embedded-python#534 (backlog row B-01, JVM half); the patch applied tosrc/arcadedb,src/javaandtestsunchanged apart from thearcadedb_embeddedtoarcadedbimport rename.Results are identical: the same k records in the same order with the same scores. Checked with a fixed-seed dump of 120
find_nearestcalls (k = 1, 5, 10, 50; 600 vectors, 16 dimensions, 4 buckets) taken on humemaiorigin/mainand on this branch, byte-identical (rid andrepr(score)per hit, plus the hit shape). One edge differs on purpose: a hit whose record is missing now raises only if it is among the final k (before: if it was anywhere in any bucket's list); the message is unchanged.What it saves, untimed: record lookups and wrappers per search are k instead of B * k (40 to 10 at B = 4 and k = 10, counted by the new tests). With B = 1 nothing changes, every hit is a final hit. No timing was run.
Local checks (on the committed bytes, bridge jar rebuilt from this branch's
src/javain eclipse-temurin:25-jdk against the jars of an installedarcadedb-embedded26.11.1.dev0 wheel, tests run withPYTHONPATH=src, not a wheel built from this repo):tests/test_vector_top_k_first.py(7 new tests: result equals a stable merge of the raw per-bucket lists for k = 1, 5, 10, 50, 400; k wraps; k lookups on the no-jar path) passes, and its two count tests fail on the oldvector.py(40 vs 10) while the result tests pass on both; together withtest_vector.py,test_vector_second_pass.py,test_vector_params_verification.py,test_vector_delta_visibility.py,test_vector_sql.pyandtest_typed_rows.pythat is 147 passed; the existing test that countedGraphCalls.hitscalls (test_lsm_vector_search_looks_up_all_hits_in_one_bridge_call) now countstopHits; black, isort and bandit are clean on the changed files. Not run: the full suite,scripts/build.sh, CI, the pre-commit hook set as a whole. No doc page in this repo mentionsGraphCallsorfind_nearest, so no docs changed.🤖 Generated with Claude Code