[Namrata Mohanty (cherryrhea)]

Summary

• expose a quantizer API that computes squared L2 distance from a candidate centroid to the original training centroid
• for squared-L2 and inner-product quantizers, recover the original training centroid by dividing the stored shift by pre_scale
• for cosine, compare the supplied centroid directly with the stored centroid without normalization or pre-scaling
• leave the supplied candidate centroid unscaled for every metric
• forward the API through the public Quantizer trait implementations
• validate dimensions and reject non-finite computed distances

Motivation

LibVector needs to compare a current data centroid with the original centroid persisted in a trained quantizer. Keeping this computation in diskann-quantization avoids reconstructing private quantizer state in an external FFI layer. Squared-L2 and inner-product quantizers store a pre-scaled shift, so the API removes that scale before comparison. Cosine training averages individually normalized vectors and does not pre-scale the stored centroid, so the supplied averaged centroid is compared directly without being normalized again.

Formula

Squared L2 / inner product:
    old_centroid = stored_shift / pre_scale
    result = sum((new_centroid - old_centroid)^2)

Cosine:
    old_centroid = stored_shift
    result = sum((new_centroid - old_centroid)^2)

Validation

• cargo test -p diskann-quantization
  • 414 unit and compile tests passed
  • 3 integration tests passed
  • 28 doc tests passed; 6 ignored
  • 0 failures
• added regression coverage verifying that squared-L2 and inner-product recover the unscaled stored centroid while leaving the new centroid unscaled
• retained cosine regression coverage using non-unit pre_scale to verify that cosine ignores scaling
• finite-input overflow is rejected through output validation
• cargo fmt --all -- --check passes
• cargo clippy --workspace --all-targets --config 'build.rustflags=["-Dwarnings"]' passes
• git diff --check passes

[Copilot]

🟡 Changes recommended

The public trait change is breaking, and cosine correction and overflow handling remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a metric-aware squared centroid-distance API to spherical quantizers.

Changes:

• Implements distance calculation and validation.
• Re-exports the new error type.
• Forwards the API through Quantizer implementations and adds tests.

File summaries

File	Summary
diskann-quantization/src/spherical/quantizer.rs	Implements centroid-distance logic, validation, and tests.
diskann-quantization/src/spherical/mod.rs	Re-exports the new error type.
diskann-quantization/src/spherical/iface.rs	Adds and forwards the public trait method.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Mark Hildebrand (hildebrandmw)]

Thanks - a few maintainability requests on my end with respect to testing. It's okay that this is technically an API breaking change.

Also feel free to ignore Copilot's claims about divergence in the cosine correction. There appears to be an existing bug with pre-scales not being correctly rejected when Cosine is used. As long as the implementation follows SphericalQuantizer::preprocess, it's fine.

[Mark Hildebrand (hildebrandmw)]

One request for documentation update, otherwise looks great to me. Thanks!

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 86.36364% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.06%. Comparing base (2739203) to head (405f6a3).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
diskann-quantization/src/spherical/iface.rs	0.00%	12 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1395      +/-   ##
==========================================
- Coverage   92.07%   92.06%   -0.01%     
==========================================
  Files         586      586              
  Lines      119997   120082      +85     
==========================================
+ Hits       110488   110557      +69     
- Misses       9509     9525      +16     

Flag	Coverage Δ
miri	92.06% <86.36%> (-0.01%)	⬇️
unittests	92.01% <86.36%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-quantization/src/spherical/quantizer.rs	98.28% <100.00%> (+0.08%)	⬆️
diskann-quantization/src/spherical/iface.rs	91.06% <0.00%> (-0.62%)	⬇️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Mark Hildebrand (hildebrandmw)]

Please open an issue for a follow-up to tighten the public API of the quantizer make providing pre-scaling with the Cosine metric impossible. Silently ignoring pre-scaling when provided is not behavior I want to keep long-term in this API.

[Namrata Mohanty (cherryrhea)]

Namrata Mohanty (Namrata Mohanty (@cherryrhea)) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree company="Microsoft"