/
MEP 25 -- Collection freezing/archiving

MEP 25 -- Collection freezing/archiving

Feb 16, 2022

Current state: Under Discussion

ISSUE:

PRs: 

Keywords: collection, meta, export, backup

Released:

Summary

To add the ability to forbidding dm operation on collection.

Motivation

  • Freeze or Archive a collection if there is no need to change the data inside it to improve the Search performance

  • Keep the collection meta stable before exporting or backup it.

  • Add the ability to re-allocate dml/delta channel on demand.

Public Interfaces

Public API to freeze and unfreeze a collection is needed.

//TODO proto here

Design Details

  • Update meta in RootCoord and record the freezing timestamp

  • Write a FreezingCollection message into dml channel

  • If a datanode receives a FreezingCollection message, it shall flush all segments in the memory and close the flowgraph after notifying the DataCoord

  • If a querynode received a FreezingCollection message, its shall close the flowgraph and notify the QueryCoord. While the growing segment shall remains the same and waits for the hand-off.

Test Plan