MEP 4 -- Support Search by ID
- Yicheng Hu
- Yefu Chen
Current state: Under Discussion
ISSUE: #5215 Support using the ID search that already exists in the collection
PRs:
Keywords: Query / Search / ID
Released:
Summary
When the vector is stored in Milvus, the corresponding ID will be generated. If the vector to be queried already exists, it will be more efficient to query directly by ID. However, the current version does not support querying by ID. We can only use the ID to retrieve the vector and then use the vector for query, which reduces the efficiency.
This proposal is about
The API of the new function
The details on how to support search by ID
Motivation
Reduce complicated Operation: Without the new function, users need to call the get_entity_by_id() and search() to perform vector querying by id
Low querying efficiency: Calling functions get_entity_by_id() and search() both need to load data into memory, which consume time.
Design Details
A. What is the API of the new function?
Plan (Recommended): The API of the new function is the same as the search() function, except for one parameter: query_record is replaced by query_id
Pros: The PyMilvus repository is more valuable.
Cons: Not see any.
B. How to search by ID?
Plan B1：Directly use the existing functions search() and get_entity_by_id() to construct new function
Pros:
Reduce operations in querying by id. Only need to call the function once.
Cons:
The efficiency of the querying behavior is still low and has not changed.
Plan B2 (Recommended): The new function will :
1 Find the vector corresponding to specific id from the data that already in the memory
2 Use the vector to do vector querying with the data that has been read in the memory
Pros:
1 Reduce operations in querying by id. Only need to call the function once.
2 Improving the efficiency of the querying behavior by avoiding duplicated data loading.
Cons:
Not see any
Compatibility, Deprecation, and Migration Plan
Need to merge the new function into 2 python SDKs for Milvus, PyMilvus and PyMilvus-ORM
Test Plan
- need to pass the unittest
- need to pass the basic python test case