2020-12-08 DM - Analytic Models and Code Efficiency

Date

Attendees

Goals

  • Analytical models review in Open Metadata/Egeria

  • Optimizations/efficiency as part of the design for components

Discussion items

TimeItemWhoNotes
10minIntroductionsMaryna
 60 mins Analytic ModelsMandy
40minsCode efficienciesMaryna
10 minsWrap-upMaryna


Notes

Analytic Models

  • 0265 Analytics Assets
  • 0280 Software Development Assets
  • 0281 Software module is collection of components that make up software
  • 0282 Release software component
    • source component relationship to asset
  • 0285 Analytics Development Assets
    • AnalyticsModel (managedModel)
    • AnalyticsProject related to 0130 Projects
    • AnalyticsModelExperiment (modelProject)
    • ChampionRole in AnalyticsModuleExperiment
  • 0525 Analytic variables
    • variables onto process
    • variables related to AnalyticModel
  • Update model from Mandy Chessell  and send out for review.

Code Efficiences


  • Ljupcho Palashevski & Chris Grote (Deactivated) optimal way of asset/scanner pattern.
  • Access services do initial load? still optimal?
  • context needed in load use cases.
  • caching to manage state/information in OMAS.
  • creative ways to solve the paging problem?
    • knowing when you have the last piece vs return same multiple times.
  • doing this at the enterprise services level.
    • more knowledge than the OMAS has.
    • index would be at that level.
  • Needs more discussion/steps for reducing duplicate when/if possible.
  • Asset lineage optimizations maybe in short term?
  • How often do we do slow initial-load, but how often down the time line?
  • Banking example: for 2500 cols, get table, schema, db, etc. 2500 times as example.
    • can we change that to make 1 call for table, schema, db instead of 2500
  • 40 hours to build full lineage picture on initial, then add in processes non-optimal becoming a bigger issues.
    • optimize on sub-set of technical assets.
  • Keep topics live!
  • Let's build initial, and tackle maybe in Jan 2021 optimizations for this from Maryna.Strelchuk@ing.com


Review Materials

Action items

  •