Page Comparison

Current state: ["Under Discussion"]

...

This API does range search for index, it returns all unsorted results with distance "better than radius" (for IP: > radius; for others: < radius).

virtual DatasetPtr
QueryByRange(const DatasetPtr& dataset, const Config& config, const faiss::BitsetView bitset)

...

This API does range search for no-index dataset, it returns all unsorted results with distance "better than radius" (for IP: > radius; for others: < radius).

static DatasetPtr
RangeSearch(const DatasetPtr base_dataset,
            const DatasetPtr query_dataset,
            const Config& config,
            const faiss::BitsetView bitset);

...

Range search completely reuses the call stack from SDK to segcore.

How to get more than 16384 range search results ?

With above solution, user can get maximum 16384 range search results in one call.

If user wants to get more than 16384 range search results, they can do like this: call range search multiple times with different radius parameter (use L2 as an example)

• NQ = 1

1st call with (radius_low_bound = 0.0, radius_high_bound = inf), get result distances like this:

{d(0), d(1), d(2), ..., d(n-1)}

2nd call with (radius_low_bound = d(n-1), radius_high_bound = inf), get result distances like this:

{d(n), d(n+1), d(n+2), ..., d(2n-1)}

3rd call with (radius_low_bound = d(2n-1), radius_high_bound = inf), get result distances like this:

...

• NQ > 1 (suppose NQ = 2)

1st call with (radius_low_bound = 0.0, radius_high_bound = inf), get result distances like this: 

{d(0,0), d(0,1), d(0,2), ..., d(0,n-1), d(1,0), d(1,1), d(1,2), ..., d(1,n-1)}

2nd call with (radius_low_bound = min{d(0,n-1), d(1,n-1)}, radius_high_bound = inf), get result distances like this: 

{d(0,n), d(0,n+1), d(0,n+2), ..., d(0,2n-1), d(1,n), d(1,n+1), d(1,n+2), ..., d(1,2n-1)}

3rd call with (radius_low_bound = min{d(0,2n-1), d(1,2n-1)}, radius_high_bound = inf), get result distances like this:

...

If there is user request to get all range search result in one call, query node team will afford another solution to save knowhere's output range search output of knowhere to S3 directly.

References(optional)

...