...
- As
CollectionAlias
works as an extra pointer to the existing collection in theRootCoordinator
, we can implement collection hot reloading at a low cost. - When
AddAlias
,DropAlias
andAlterAlias
, the change of relationship between alias and collection should not affect the operations online, i.e. these operations shoud use the relationship when the task begin to excute. - When
AddAlias
,DropAlias
andAlterAlias
, all proxy should receive the change in the same timestamp to ensure the atomicity between the proxy cache and the rootcoord data. - Users can't drop the
collection
if thecollection
is referenced by analias
, which can simplify the operation and make it easier to ensure the atomicity property about alias changes.
...
Design Details
...
Changes to
...
the MetaTable
Metatable
now addts2alias2name
andnewestAliasTs
member variables
Code Block |
---|
type immutablemap_string2string struct{ storemap map[string]string } func (imstr2str * immutablemap_string2string) get(key string) (string, error){ rstr,ok := imstr2str.storemap[key] if(!ok){ return nil"", fmt.Errorf("key not exist") } return rstr, nil } func (imstr2str * immutablemap_string2string) put(key string, val string) (string, error){ return nil"", fmt.Errorf("not allowed put in immutablemap") } type metatable struct{ ... ts2alias2name map[Timestamp]immutablemap_string2string newestAliasTs Timestamp ... } |
- Change to alias will add a new
alias2name
immutable map ints2alias2name
.
Code Block |
---|
func build(im typeutil.ImmutablemapString2string) *Builder { builder := &Builder{ reference: im, mutable: nil, } return builder } type Builder struct { reference typeutil.ImmutablemapString2string mutable map[string]string } func (bdr *Builder) maybeClone() { if !bdr.reference.IsEmpty() { bdr.mutable = make(map[string]string) for k, v := range bdr.reference.GetCopy() { bdr.mutable[k] = v } bdr.reference = typeutil.ImmutablemapString2string{} } else { bdr.mutable = make(map[string]string) } } func (bdr *Builder) getnamefromalias(key string) (string, bool) { bdr.maybeClone() rstr, rbool := bdr.mutable[key] return rstr, rbool } func (bdr *Builder) putalias2name(key string, val string) (string, bool) { bdr.maybeClone() var pre string v, ok := bdr.mutable[key] if ok { pre = v } else { pre = "" } bdr.mutable[key] = val return pre, ok } func (bdr *Builder) removealias2name(key string) (string, bool) { bdr.maybeClone() var pre string v, ok := bdr.mutable[key] if ok { pre = v delete(bdr.mutable, key) } else { pre = "" } return pre, ok } func (bdr *Builder) Build() typeutil.ImmutablemapString2string { if !bdr.reference.IsEmpty() { reference := bdr.reference bdr.reference = typeutil.ImmutablemapString2string{} return reference } mutable := bdr.mutable bdr.mutable = nil res := typeutil.NewImmutablemapString2string(mutable) return res } func (mt * metatable) addAlias(collectionAlias string, collectionName string) error{ mt.ddLock.Lock() defer mt.ddLock.Unlock() ... ts = getTimestamp() tspre = mt.newestAliasTs Bdr = build(mt.ts2alias2name[tspre]) _,ok := Bdr.putalias2name(collectionAlias, collectionName) if ok{ return fmt.Errorf("alias already exist when add alias") } mt.ts2alias2name[ts] = Bdr.build() mt.newestAliasTs = ts ... return nil } func (mt * metatable) dropAlias(collectionAlias string) error{ mt.ddLock.Lock() defer mt.ddLock.Unlock() ... ts = getTimestamp() tspre = mt.newestAliasTs Bdr = build(mt.ts2alias2name[tspre]) _, ok := Bdr.removealias2id(collectionAlias) if !ok{ return fmt.Errorf("alias not exist when drop alias") } mt.ts2alias2name[ts] = Bdr.build() mt.newestAliasTs = ts ... return nil } func (mt * metatable) alterAlias(collectionAlias string, collectionName string) error{ mt.ddLock.Lock() defer mt.ddLock.Unlock() ... ts = getTimestamp() tspre = mt.newestAliasTs Bdr = build(mt.ts2alias2name[tspre]) _,ok := Bdr.putalias2name(collectionAlias, collectionName) if !ok{ return fmt.Errorf("alias not exist when alter alias") } mt.ts2alias2name[ts] = Bdr.build() mt.newestAliasTs = ts ... return nil } |
GetCollectionByName(), IsAlias(), ListAlias() ...
which is related to alias will add alias timestamp as parameter to select whichalias2name
map will be used ints2alias2name
.- Users can't drop the
collection
if thecollection
is referenced by analias
Change to globalMetaCache
- Add a function
fetchAliasCache()
to fetch thenewestAliasTs
and the correspondingalias2name
from rootcoord into proxy ifnewestAlias
Ts is different with the stored timestamp inglobalMetaCache.
- When each proxy task begin to excute, running
fetchAliasCache()
to get thenewestAliasTs
and the correspondingalias2name
from globalMetaCache to task environment. - When proxy task need a collecition name in task request, first check if the name is alias. If yes, replace it with the corresponding collecition name use gotten
alias2name.
- If we do this, the cache in
globalMetaCache
will be the map with key as collection real name and value as collection information. - When proxy task refers to the rootcoord task, it's necessary to pass the cached
newestAliasTs
as the rootcoord task request in order to make the apis which is related with alias to select whichalias2name
they need to use.
Compatibility, Deprecation, and Migration Plan
Above changing plan will add a item in rootcoord task request, which may need some attention.
Test Plan
- Unit tests