Skip to content

Flaky test: TestFindNearestApproximate first result can be y instead of x (PQ on three vectors) #149

Closed

Description

@tae898

tests/test_vector_index_introspection.py::TestFindNearestApproximate::test_returns_the_nearest_record_and_its_distance failed once on PR #148 (job https://github.com/ArcadeData/arcadedb-python/actions/runs/40999879904/job/123882125988, Test arcadedb (linux/amd64, Python 3.14)): assert record.get("name") == "x" got "y". The same test passed on every other leg of that run and on the last six runs of test-jvm.yml on main (scheduled run of 2026-10-30 03:04 UTC included), and PR #148 does not touch the PRODUCT path, so this looks like a flaky test rather than a regression.

Why it can flip: the fixture builds a PRODUCT index on three 3-dimensional vectors (PQ_VECTORS) with pq_clusters=4 and asks find_nearest_approximate([0.9, 0.1, 0.0], k=1). The codebook is trained on those few vectors, and the approximate answer is ranked on the PQ codes, so whether x or y comes first can depend on how the training lands. test_k_results_come_back_nearest_first asserts the same first name with k=3.

Possible fixes (to choose in the PR): rerank so the returned distance is exact (rerank_oversample, or a k large enough that the exact rerank covers every row), vectors far enough apart that any codebook of 4 clusters separates them, or an assertion on the set of the top k rather than the first name. A fix should be checked by repeating the test many times on the same leg, not by one green run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions