/
MEP 25 -- Collection freezing/archiving

MEP 25 -- Collection freezing/archiving

Current state: Under Discussion

ISSUE:

PRs: 

Keywords: collection, meta, export, backup

Released:

Summary

To add the ability to forbidding dm operation on collection.

Motivation

  • Freeze or Archive a collection if there is no need to change the data inside it to improve the Search performance
  • Keep the collection meta stable before exporting or backup it.
  • Add the ability to re-allocate dml/delta channel on demand.

Public Interfaces

Public API to freeze and unfreeze a collection is needed.

//TODO proto here

Design Details

  • Update meta in RootCoord and record the freezing timestamp
  • Write a FreezingCollection message into dml channel
  • If a datanode receives a FreezingCollection message, it shall flush all segments in the memory and close the flowgraph after notifying the DataCoord
  • If a querynode received a FreezingCollection message, its shall close the flowgraph and notify the QueryCoord. While the growing segment shall remains the same and waits for the hand-off.

Test Plan


Related content

MEP 32 -- Atomic switching for collection alias
MEP 32 -- Atomic switching for collection alias
More like this
MEP 10 -- Support Collection Alias
MEP 10 -- Support Collection Alias
More like this
MEP 17 -- Support handoff and load balance segment on query nodes
MEP 17 -- Support handoff and load balance segment on query nodes
More like this
MEP 5 -- Refactor query(search & retrieve) workflow
MEP 5 -- Refactor query(search & retrieve) workflow
More like this
MEP 9 -- Support delete entities by primary keys
MEP 9 -- Support delete entities by primary keys
More like this
MEP 23 -- Multiple memory replication design
MEP 23 -- Multiple memory replication design
More like this