...
Proposal delete operation persistent
- DataNode subscribe the insert channel
- Proxy receives a delete request, parse `expr` to get primary keys, then split into insert channels by primary keys (done)
- DataNode receives a delete request from the insert channel, save it in buffer, and write it into the delta channel
- when DataNode start up
- load all segments info from Minio into memory (done)
- load all segments' deleted ids from Minio into in-memory structure delBuf
- save deleted id into structure map<segID, list(id, ts)>
- deleted ids are saved into a when DDNode receive DeleteMsg, save it into FlowGraphMsg structure, and send FlowGraphMsg to next node (done)
- when InsertBufferNode receive FlowGraphMsg, process InsertMsg in it, then wrapper DeleteMsg into another FlowGraphMsg and sent it to next node (done)
- add DeleteNode, when receive FlowGraphMsg, process DeleteMsg in it, save deleted ids and timestamps into sync.Map delBuf (done)
- add another flushChan for DeleteNode (doing)
- when DeleteNode receive flushMsg, save all data in delBuf into Minio
- deleted ids and timestamps are saved into two separated Minio file with name "/by-dev/deltalog/collectionID/partitionID/segmentID/xxx"
- when DataNode start up
- DataNode notifies IndexNode to building indexes
- finish
...