Skip to end of metadata
Go to start of metadata
You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 40
Next »
Overview
During the march 11th GB F2F meeting, the board recommended having a reference ML workflow / stack, and showcasing where current LF AI hosted projects fall. The idea is to have a reference architecture and to allow us to identify gaps in our project portfolio so we can focus on project recruitment in areas that we need most, leading us to provide a reference implementation using LF AI hosted projects.
Mailing List
Please subscribe to the ML Workflow mailing list: https://lists.lfai.foundation/g/mlworkflow-committee
Meetings
Every 4 weeks on Thursday, 10am EST right after the TAC meeting.
To get invites, please join the mailing list
Zoom:https://zoom.us/j/9918615568
Agenda for 04/09/2020 meeting:
- Howard Huang of Huawei will present a plan for the interoperability effort
- Map new project to ML workflow stack
Past Meetings -
Meeting Content (minutes / recording / slides / other):
Date | Minutes |
---|
| Call was recorded, this is the record. And this is the chat. - We agreed to change the name of the sub committee from ML Workflow Committee to ML Workflow & InterOp Committee (MWI)
- Zhipeng (Howard) Huang shared the slides of the InterOp Proposal and we agreed to move forward with the plan
- The first use case we agreed on will be Federated Learning
- We will create a slack channel for the MWI Committee
- Zhipeng (Howard) Huang will lead the WMI Committee from now on as Ofer Hermoni is stepping down
|
| - ML Workflow stack (updated deck):
- Ofer Hermoni shared the work was done off line with Marquez team to map it to the Data Consistency block
- Jnu Gu of Zilliz provided the mapping to the Serving box
- Interoperability proposal (see deck)
- Howard Huang of Huawei presented a proposal for interoperability between the different LFAI projects and also external projects
- We had a discussion if there is a need for a real interoperability effort or maybe a standard will be enough? We agreed that there is a need for a real interoperability effort
- Next steps:
- In the next ML Workflow meeting Howard will present a plan for the effort
- In one of the next TAC meetings Howard will share the proposal with the broader team
|
| Call was recorded, this is the record. And this is the chat. In the meeting we covered the following: - We reviewed the current version of the ML Workflow, and tried to identify the correct blocks for the two new projects (Milvus and Marquez)
- Jun of Zilliz will think about that for Milvus
- Ofer Hermoni will approach the WeWork team to identify the right location for Marquez
- Project integration:
- Natarajanc covered the status of the collaboration / integration between Acumos and Angel, and Acumos and Adlik. Due to the Corona epidemic everything is slowed down currently in China
- Natarajanc also mentioned the work Acumos is doing with AI360. Currently in discussions phase
- Jessica Kim shared the status of the integration infrastructure Huawei has in Hong Kong, and the plan to add more locations based on Tencent public cloud - both in China and Canada
- Jessica Kim suggested to leverage the integrations and the ML Workflow activity as part of the outreach activity
|
| In this meeting we had a team from RedHat to present ODH (Open Data Hub) - In this meeting we started recording our calls. The Zoom recording can be find here. The chat here
- Deck presented is available here
- Ofer Hermoni gave a background about the ML Workflow working group goals and activity
- Redhat team provided a presentation about ODH
- Apparently there are many overlaps between the two initiatives
- Next steps
- Invite the ODH team to present to the TAC
- A representative of the TAC will join one of the ODH meetings to present LFAI and ML Workflow
|
| Agenda for the meeting: 1. Update about the integration between Acumos and other LFAI projects - Angel and ONNX 2. Review the latest version of the ML Workflow slides and map 3. Discuss our focus going forward Agenda for the meeting: Minutes: - We have now a dedicated mailing list, please make sure you register so you can get all the emails and invites
- Ofer Hermoni shortly explained to goals of the committee
- We mapped the new project Sparklyr on the ML stack, and updated the deck
- Natarajanc and Fitz Wang updated regarding the integration between Angel and Acumos - the technical teams are working together on the integration
- Natarajanc updated re the status of the integration between Acumos and ONNX. Jim Spohrer said he will dedicate more resources to help with that
- Logistics:
- Everyone should register to the mailing list in order to get the invites
- Meetings will be scheduled for once in 4 weeks on Thursdays 10am EST (immediately after the TAC call)
- We discussed the goals of this committee:
- The integration between the different projects will be done by the technical teams of the projects. This committee will support that and identify potential collaboration opportunities and identify resource gaps
- Jim Spohrer suggested to work with the Red Hat team on the Open Data Hub initiative. He will invite Sherard Griffin to talk in the next meeting
- Jim Spohrer encourages collaboration and integration to IBM AI Fairness framework, it will help to convince IBM to contribute this framework of projects to LFAI
|
| - Goals defined for the meeting:
- Reevaluate the ML Workflow stack according to feedback we got
- Work on integration between different LFAI projects
- Ofer Hermoni shared that there is a lot of interest from the community (LFAI members and the broader community), and many joined this effort, we have 18 people representing 8 companies!
- evaluate the ML Workflow stack (see here updated deck):
- Patrick Fu (from Gemini Open Cloud) shared their view on the ML Pipeline (see slide 2 on the deck), main difference was the data consistency layer - we added it to our stack
- Ofer Hermoni mentioned that Fitz Wang showed in OSS that Angel supports Feature Engineering as well - we added that to the stack
- Next step - update existing projects on the new stack
- Integration between different projects:
- Natarajanc shared with the team the work we are doing regarding the integration between Acumos and Angel and between Acumos and Horovod
- Next step - set a "1:1" session dedicated to the integration between Angel and Acumos (Thursday September 19th)
- More next steps
|
| - We farther refined the presentation and discussed next steps for this group
- IBM presented some of the internal work they are doing around AI open source
|
| - We discussed the different open source projects and mapped them to our stack. See the results here
- Next week Animesh will present a framework IBM is building and we will discuss how it can be beneficial to create collaboration between different LFAI projects
- We will also prepare content for the next TAC meeting in which we will review our progress
|
| |
| - We defined our goal, and started to review the stack. This is the result
|
Participants (archive as we moved to use the mailing list)
Committee Lead: Ofer Hermoni <oferher@gmail.com>