Repository navigation
vectors: rerank_oversample on BINARY, queryNodes where, indexed pre-filter (ArcadeDB 26.11.1) - #148
Merged
Merged
Conversation
…ilter (26.11.1)
Three engine changes in ArcadeDB 26.11.1 made statements here wrong.
- #10975 (fix of #10968): a BINARY index scores nodes on its one-bit codes and
reranks k * rerankOversample candidates (the setting PRODUCT uses), and the
rerank option of vector.neighbors works on it. create_vector_index refused
rerank_oversample unless quantization="PRODUCT", on both backends. It now
accepts PRODUCT and BINARY. Engine source (LSMVectorIndex.findNeighbors,
"Ignored when the index is neither PRODUCT nor BINARY") says NONE and INT8
ignore the setting, so those, and every pq_* and fused_pq, keep the
ValueError.
- #10808 (#10685): db.index.vector.queryNodes takes a fourth argument, a
{where, filter} options map, since 81c30122fc (first in 26.11.1; 26.10.2
answers "expects 3 arguments but got 4"). The page, example 29 and the
examples README said the procedure has no filter.
- #10957 (fix of #10921): a selective where on an indexed property takes the
property index; unindexed conditions are unchanged. The Filtered search
section says so and example 29 now indexes the property it filters on.
Tests: BINARY accepts and reads back rerank_oversample (create_vector_index and
a CREATE INDEX statement), still refuses pq_* and fused_pq, search with the
rerank option returns exact distances; NONE, INT8 and None refuse
rerank_oversample; queryNodes {where} returns only matching rows for an indexed
and an unindexed property. Version-dependent branches follow the existing
probe (has_pq_rerank) and _engine_version() patterns, no skips. Ran on the
313f23e234 engine jars, on e353330c5c and on 26.10.2; the BINARY tests fail on
the old check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the native build too Native parity on PR #148 failed test_query_nodes_takes_the_where_options_map with JVMNotRunning: _engine_version() read com.arcadedb.Constants through JPype, which the native build does not have. The behaviour is shared by both builds, so per the conftest rule (a test native shares must pass on native) the check now reads SELECT version() through the test's database. The two older tests that were on the NA-JVM list only because of that JPype read (test_where_map_and_condition, test_where_combines_with_filter) come off it. Checked: the three where tests pass on the native wheel (engine e353330c5c) and the JVM wheel (same engine), run from a copy of tests/ against the installed wheels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the vector guide, the
rerank_oversamplecheck and their tests in line with three ArcadeDB 26.11.1 engine changes.BINARYindex scores nodes on its 1-bit codes and reranksk * rerankOversamplecandidates (default 10), or the wholeefSearchbeam when larger; thererankoption works on itcreate_vector_index(rerank_oversample=...)acceptsPRODUCTandBINARYon both backends;NONE,INT8and no quantization keep theValueError, as dopq_*andfused_pqwithoutPRODUCT. Guide, docstrings updateddb.index.vector.queryNodes(index, k, vector, {where, filter})options mapexpects 3 arguments but got 4whereon an indexed property takes the property index; broad and unindexed conditions unchangedNOTUNIQUEonFact.tenantWhy
rerank_oversampleis not opened for every quantization:LSMVectorIndex.findNeighborsdocuments the setting as "Ignored when the index is neither PRODUCT nor BINARY", andLSMVectorIndexMetadatamarks it "PRODUCT and BINARY". The metadata parser itself accepts the key on any index, so the stricter Python rule stays forNONEandINT8.Verification:
python -m pytest, pinnedtaskset -c 4,5, Python 3.13) ran against a scratch layout of the JVM wheel with the engine jars of upstream313f23e234(contains all three changes) and this branch's Python sources overlaid; no wheel was built. 139 passed acrosstest_vector_product_quantization.py,test_vector_sql.py,test_vector.py,test_vector_index_introspection.pyandtest_docs_vectors_page.py. The same files pass one353330c5c(has #10808, not #10957 or #10975) and the new queryNodes test and theBINARYtests pass on 26.10.2 jars through their old-engine branches.BINARYtests fail on the old check (5 failed onorigin/mainsources over the313f23e234jars).BINARYsearch reranksk * rerankOversamplecandidates (a recall difference of a few percent with no deterministic assertion) and that the indexed pre-filter plan is taken (the new test pins the returned rows, which are the same whichever plan runs). Both are covered by upstream's own tests.has_pq_rerankprobe and_engine_version()), no skips.313f23e234with its checks passing..pyfiles compile.🤖 Generated with Claude Code