...
Design Details(required)
1. Current dependence
- Segcore depends on the query module, both of them depend on faiss lib.
- Except for NSG and SPTAG, other index implementations are based on VecIndex Interface.
- HNSWLib and NSG were implemented with fass lib.
- The thirdpart libs include HNSWLib、NGT and Annoy that depending on faiss.
2. Decoupling
- The distributed layer doesn't directly depend on faiss and should access the unified interface of knowhere.
- Each index implementation should be independent, there is an important thing that we keep maintenance thirdpart libs following the new source version.
- Move Knowhere to a new project.
- We'd better decouple libfiu from the key code path, which should only use within the test code.
2. Support compile on MAC
- Distributed layer
- Optimize compile phase make Query and segcore module depends on knowhere shared library.
- Reconstruct BinarySearchBruteForce and FloatSearchBruteForce functions within the query module by knowhere supported brute force search index.
- Support an alternative way to use libfiu which doesn't compile on Mac, or provide a optional an option to use libfiu when compile compiling on Mac.
- Engine Layer(knowhere)
- Support a brute force search which is a kind of vector index and compiled on MAC & Ubuntu.
- Upgrade faiss to the latest version, because the current version can not be compiled on Mac.
...