Overview
During the march 11th GB F2F meeting, the board The LF AI & Data Governing Board recommended having a reference ML workflow / stack, and showcasing where current LF AI hosted projects fall. The idea is to have a reference architecture and to allow us to identify gaps in our project portfolio so we can focus on project recruitment in areas that we need most, leading us to provide a reference implementation using LF AI & Data hosted projects.
Mail Mailing List
Patrick Fu | Gemini Open Cloud | patfu2005@geminiopencloud.com | Edison Peng | Gemini Open Cloud | edison@geminiopencloud.com | Fitz Wang | Tencent | fitzwang@tencent.com | Clive Cox | Seldon | cc@seldon.io | Sophia Arakelyan | Simiotics | sophia@simiotics.com | Neeraj Kashyap | Simiotics | neeraj@simiotics.com | Next Meeting
Thursday October 3, 2019 at 8:00 am PT.
Please contact Ofer Hermoni or email info@lfai.foundation to be added to the call invitation.
Meetings
How to Join:
Contact Ofer to get invites
Meeting Content (minutes / recording / slides / other):
Date | MinutesPlease subscribe to the TAC ML Workflow mailing list: https://lists.lfailfaidata.foundation/g/tac-general
Participants
Working Group Lead: Ofer Hermoni <oferher@gmail.com>
Name | Affiliation | Email | LF ID |
---|
Ofer Hermoni | Amdocs | Ofer.Hermoni@amdocs.com | |
Ibrahim Haddad | Linux Foundation | ibrahim@linuxfoundation.org | |
Jim Spohrer | IBM | spohrer@us.ibm.com | Nat Subramanian | Tech Mahindra | natarajan.subramanian@techmahindra.com | |
Ian Lumb | Sylabs.io | ilumb@sylabs.io | Jamil Chawki | Orange | jamil.chawki@orange.com | Vishnu Ram OV | Member of ITU FG ML5G | vishnu.n@ieee.org | |
Animesh Singh | IBM | singhan@us.ibm.com | Vijay R Bommireddipalli | IBM | vijayrb@us.ibm.com | Ganesh Harinath | Verizon | ganesh.harinath@verizonmedia.com | Kiril Nejkov | IFC | knejkov@ifc.org | Gordon I. Myers | IFC | gmyers@ifc.org | Guangji Xue | IFC | gxue@ifc.org | Hussain A. Alkazemi | World Bank Group (WBG) | halkazemi@worldbankgroup.org | mlworkflow-committee
Meetings
Every 4 weeks on Thursday, 10am EST right after the TAC meeting.
To participate in meetings, please join the mailing list and subscribe to the group calendar: ML Workflow - Community Meetings & Calendar
Zoom:
https://zoom.us/j/9918615568?pwd=TTVaOXcyMXoxWWU4VEZWaTBPQnZIUT09
Slack Channel and Shared Document:
- Slack Channel
- Shared Document
Meetings -
Meeting Content (minutes / recording / slides / other):
Date | Minutes |
---|
27 August 2020 |
|
11 June 2020 | Call was recorded. View file |
---|
name | Recording Alejandro_Saucedo Interop_Seldon_Kubeflow 20200611.mp4 |
---|
height | 250 |
---|
|
|
04 June 2020 | - Start with Adlik project on interop
- document: https://docs.google.com/document/d/1_e7Ty9VvEXvgS-4YSuBbER8VA4_vnX6AtZE1bOw_Efc/edit?usp=sharing
- Map new project to ML workflow stack
|
07 May 2020 | - Kickstart of the interop work, identify top 1~3 use cases, champions that volunteer to lead the use case discussion, and milestones
- document: https://docs.google.com/document/d/1_e7Ty9VvEXvgS-4YSuBbER8VA4_vnX6AtZE1bOw_Efc/edit?usp=sharing
- Map new project to ML workflow stack
|
| Call was recorded, this is the record. And this is the chat. - We agreed to change the name of the sub committee from ML Workflow Committee to ML Workflow & InterOp Committee (MWI)
|
| - ML Workflow stack (updated deck):
- Ofer Hermoni shared the work was done off line with Marquez team to map it to the Data Consistency block
- Jnu Gu of Zilliz provided the mapping to the Serving box
- Interoperability proposal (see deck)
- Howard Huang of Huawei presented a proposal for interoperability between the different LFAI projects and also external projects
- We had a discussion if there is a need for a real interoperability effort or maybe a standard will be enough? We agreed that there is a need for a real interoperability effort
- Next steps:
- In the next ML Workflow meeting Howard will present a plan for the effort
- In one of the next TAC meetings Howard will share the proposal with the broader team
|
| Call was recorded, this is the record. And this is the chat. In the meeting we covered the following: - We reviewed the current version of the ML Workflow, and tried to identify the correct blocks for the two new projects (Milvus and Marquez)
- Jun of Zilliz will think about that for Milvus
- Ofer Hermoni will approach the WeWork team to identify the right location for Marquez
- Project integration:
- Natarajanc covered the status of the collaboration / integration between Acumos and Angel, and Acumos and Adlik. Due to the Corona epidemic everything is slowed down currently in China
- Natarajanc also mentioned the work Acumos is doing with AI360. Currently in discussions phase
- Jessica Kim shared the status of the integration infrastructure Huawei has in Hong Kong, and the plan to add more locations based on Tencent public cloud - both in China and Canada
- Jessica Kim suggested to leverage the integrations and the ML Workflow activity as part of the outreach activity
|
| In this meeting we had a team from RedHat to present ODH (Open Data Hub) - In this meeting we started recording our calls. The Zoom recording can be find here. The chat here
- Deck presented is available here
- Ofer Hermoni gave a background about the ML Workflow working group goals and activity
- Redhat team provided a presentation about ODH
- Apparently there are many overlaps between the two initiatives
- Next steps
- Invite the ODH team to present to the TAC
- A representative of the TAC will join one of the ODH meetings to present LFAI and ML Workflow
|
| Agenda for the meeting: 1. Update about the integration between Acumos and other LFAI projects - Angel and ONNX 2. Review the latest version of the ML Workflow slides and map 3. Discuss our focus going forward Agenda for the meeting: Minutes: - We have now a dedicated mailing list, please make sure you register so you can get all the emails and invites
- Ofer Hermoni shortly explained to goals of the committee
- We mapped the new project Sparklyr on the ML stack, and updated the deck
- Natarajanc and Fitz Wang updated regarding the integration between Angel and Acumos - the technical teams are working together on the integration
- Natarajanc updated re the status of the integration between Acumos and ONNX. Jim Spohrer said he will dedicate more resources to help with that
- Logistics:
- Everyone should register to the mailing list in order to get the invites
- Meetings will be scheduled for once in 4 weeks on Thursdays 10am EST (immediately after the TAC call)
- We discussed the goals of this committee:
- The integration between the different projects will be done by the technical teams of the projects. This committee will support that and identify potential collaboration opportunities and identify resource gaps
- Jim Spohrer suggested to work with the Red Hat team on the Open Data Hub initiative. He will invite Sherard Griffin to talk in the next meeting
- Jim Spohrer encourages collaboration and integration to IBM AI Fairness framework, it will help to convince IBM to contribute this framework of projects to LFAI
|
| - Goals defined for the meeting:
- Reevaluate the ML Workflow stack according to feedback we got
- Work on integration between different LFAI projects
- Ofer Hermoni shared that there is a lot of interest from the community (LFAI members and the broader community), and many joined this effort, we have 18 people representing 8 companies!
- evaluate the ML Workflow stack (see here updated deck):
- Patrick Fu (from Gemini Open Cloud) shared their view on the ML Pipeline (see slide 2 on the deck), main difference was the data consistency layer - we added it to our stack
- Ofer Hermoni mentioned that Fitz Wang showed in OSS that Angel supports Feature Engineering as well - we added that to the stack
- Next step - update existing projects on the new stack
- Integration between different projects:
- Natarajanc shared with the team the work we are doing regarding the integration between Acumos and Angel and between Acumos and Horovod
- Next step - set a "1:1" session dedicated to the integration between Angel and Acumos (Thursday September 19th)
- More next steps
|
| - We farther refined the presentation and discussed next steps for this group
- IBM presented some of the internal work they are doing around AI open source
|
| - We discussed the different open source projects and mapped them to our stack. See the results here
- Next week Animesh will present a framework IBM is building and we will discuss how it can be beneficial to create collaboration between different LFAI projects
- We will also prepare content for the next TAC meeting in which we will review our progress
|
| |
| - We defined our goal, and started to review the stack. This is the result
|
Participants (archive as we moved to use the mailing list)
Committee Lead: Zhipeng Howard Huang Zhipeng (Howard) Huang