2020-12-08 DM - Analytic Models and Code Efficiency

Date

Dec 8, 2020

Attendees

• @Mandy Chessell

Goals

• Analytical models review in Open Metadata/Egeria

• Optimizations/efficiency as part of the design for components

Discussion items

Notes

Analytic Models

• 0265 Analytics Assets

• 0280 Software Development Assets

• 0281 Software module is collection of components that make up software

• 0282 Release software component

  • source component relationship to asset

• 0285 Analytics Development Assets

  • AnalyticsModel (managedModel)

  • AnalyticsProject related to 0130 Projects

  • AnalyticsModelExperiment (modelProject)

  • ChampionRole in AnalyticsModuleExperiment

• 0525 Analytic variables

  • variables onto process

  • variables related to AnalyticModel

• Update model from @Mandy Chessell  and send out for review.

Code Efficiences

• @Ljupcho Palashevski & @Chris Grote (Deactivated) optimal way of asset/scanner pattern.

• Access services do initial load? still optimal?

• context needed in load use cases.

• caching to manage state/information in OMAS.

• creative ways to solve the paging problem?

  • knowing when you have the last piece vs return same multiple times.

• doing this at the enterprise services level.

  • more knowledge than the OMAS has.

  • index would be at that level.

• Needs more discussion/steps for reducing duplicate when/if possible.

• Asset lineage optimizations maybe in short term?

• How often do we do slow initial-load, but how often down the time line?

• Banking example: for 2500 cols, get table, schema, db, etc. 2500 times as example.

  • can we change that to make 1 call for table, schema, db instead of 2500

• 40 hours to build full lineage picture on initial, then add in processes non-optimal becoming a bigger issues.

  • optimize on sub-set of technical assets.

• Keep topics live!

• Let's build initial, and tackle maybe in Jan 2021 optimizations for this from @Maryna.Strelchuk@ing.com

Review Materials

Open

Action items