- OpenFL Governance Document
- TSC Members
- TSC Meeting Notes
03-21-2023
Agenda: https://docs.google.com/document/d/1bjcoDNk0pESBSEP2fD8SmqDZX5akxbWkwdG-EtZ9gJo/edit
Meeting recording: https://lists.lfaidata.foundation/g/openfl-tsc/files/OpenFL%20TSC%20Meeting%20Recordings/OpenFL_TSC_Meeting_3_21_2023.mp4
Meeting Notes:
Summary : Voted "Yes" by the 7 TSC members to add Open FL as a sub project. Slack to be the official channel for offline conversation for the TSC members. Offline conversation on how UPenn could communicate offline with the rest of the team.
Decisions:
- Voted "Yes" by the 7 TSC members to add Open FL as sub project
- Slack to be the official channel for Open FL Communications
AR:
- Discuss and get back to the TSC members on presenting FedLCM at the OpenFL Community Meeting - (Layne)
- Old Meeting links in the GitHub to be updated - (Olga) - done
- Send out additional procedure questions that couldn’t be covered in the TSC meeting- (Patrick) - done
- Investigate whether an exception can be made for Slack / Google Docs at UPenn (Spyros)
- Propose more detailed criteria for Governance document based on prior experience in this area (Eric)
- Create private slack channel for protected information - TSC - private (Prashant)
Resources:
Details:
- OpenFL Governance document review
- Set new opportunity for publicizing the work
- Creation of sub projects and sub committees
- An example is work FL standards. May be covered in larger TSC meeting, but could be moved to subcommittee
- Team went through the doc and reviewed and agreed.
- Responsibilities of TSC members:
- When TSC starts to grow, the topic which may not be in the larger group’s interest, we can choose to form sub committee
- TSC
- 7 members
- Initially chose an odd number for voting purpose
- Can vote and add new members
- Maintenance
- Currently 4 maintainers from intel
- Would love to have more contributors from different organizations (drives toward open source / contribution)
- Opens?
- Will be edited by the TSC
- Changes will be proposed by the committee and confirmed by voting
- Don’t have the detail spelled out yet in the governance document
- Can make proposal for the language
- Could use this meeting but since its monthly, we can propose it offline
- Eric would like to propose few things around processes
- Meeting are recorded, posted to both LF Wiki as well as the OpenFL TSC meeting doc
- Video will go to LF shared google drive [ update: this will actually be posted to the OpenFL LF&AI project wiki]
- Public will be allowed to join the meeting in future
- If any private messages, discussion could use private slack channels (to be created) for immediate response, or email for longer form discussions
- How will the Governance Document be reviewed and updated?
- Where do we maintain these processes for public to look at it?
- Sharing of the information? What about the meeting notes? What about the recording?
- Can slack be the common channel?
- No concern
- UPenn blocks slacks and google doc. Limited to contribution
- In Slack, we can have private channel
- Take it offline to discuss how Upenn can join Slack
Proposal: Creation of OpenFL Security subproject in github.com/securefederatedai repo
What is OpenFL Security [Patrick]:
- Extension to openFL
- Adds additional component called the governor that has the role of:
- Storing participant identity (certificate) information
- Metadata about data sets useful for other participants to search on
- Keeps a record of what state a given experiment is in
- Verifies that all participants are executing an experiment in a hardware enclave (when specified in the FL plan)
- Governor is built on Microsoft Confidential Consoritum Framework (CCF)
- Blockchain application so can record , what dataset was used, what model was used is stored in the governor, used by governor for long term quality of the dataset
- What is the project?
- New repo under the LF
- After the initial commit from Intel development will take place in open
- In the future, our goal is to explore expanding beyond support for OpenFL; support federated evaluation, other FL frameworks (Flower), base governance that can operate with any other use cases. Others can contribute support for other TEE’s in the future - not necessarily limited to SGX.
Vote:
Daniel: He voted from the meeting chat - Yes
Layne: Yes
Prashant: Yes
Eric: Yes
Spyridos Yes
Micah: Yes
Patrick: Yes
7 yes, 0 no
Proposal passed
April OpenFL community meeting agenda
- First OpenFL Community Meeting happened in December. We have an Asia Friendly and Europe friendly meeting every two months
- Agenda:
- TSC: Anyone interested in the above meetings?
- Last time, Intel presented the usage of OpenFL in other projects
- Intel Labs presented an example of how OpenFL was used for training a model used in 5G deployments
- Can have different company in different timezone
- These will be public meeting
- Doesn’t have to be from TSC, anyone from the company can participate
- For companies that don’t have active projects, TSC member involvement on the calls will help give a view into what the community is working on
- Layne (VMWare) - To discuss with internal team about FedLCM presentation and get back by end of week (3/24)
- Syros (UPenn) - interested in contributing to OpenFL Community meetings. Will discuss offline with Intel about whether April meeting, or future meeting would be best
- Micah (Intel) - would be interested in presenting related work in Medperf in future meeting (after April)
- We should share the meeting with wider (most from intel, good if we have diversity from different domain)
- Share the link
- Subscribe to the mail list
- Share your calendar invite
- Also have the link the GitHub Repo (ReadMe)
- Old links need to change - OlgaOpenFL 2023 Roadmap:
- TSC: Anyone interested in the above meetings?
- Europe- April 5th 8 am PST
- Asia: April 6th 6pm
- One agenda item is to present OpenFL Security
- As we are onboarding to LF, Intel would like to have other members (Leidos/UPenn/VMWare/Flower Labs) come and present related work in federated learning
- How to share the meeting?
High level goals for the year
- 1.6 OpenFL
- Example: horizontal federated learning, data preprocessed, doesn’t work for vertical federated learning
- Working internally to make this experience
- Added experimental support for 1.5, continue this work, expand from a simulation on a single node to a multi-node federated run time
- Possible that a UI can be built on this interface
- Aim to bridge the gap between existing interfaces
- Having cohesion between all these interface( high, middle, low)
- Long term
- Standard ML models
- Research community - want to support new research ideas easily
- Compatible with latest version of OpenFl
- Interfaces are tightly coupled to infrastructure
- We want to decouple the interface and have api expand across different infrastructures
- 1.6 new use cases:
- OpenFL documentation needs improvement
- OpenFL Security extension - early Q2
- Is there a strategy for enabling groups to pursue grant support to develop and/or contribute to OpenFL?
- How to broaden out to contribute more from academy plus commercial
- Do we have strategy around contribution to this
- Academic
- Separate discussion, new institution perhaps with NIH, use slack on using
- Various use case for deployment, in terms of researcher deployment not dependent on the platform
- Do we clarify how we have solutions for each of these?
- ROADMAP.md doc in the repo, explains the high level features planning to add in the next year
- Historically driven by Intel but we would like for the consortium to provide input here
- FederatedRuntime
- Opens? Any new features?
- Patrick to send what we didn’t get to discuss for offline discussion
Call for and discussion of additional features / integrations
04-18-2023
Agenda: https://docs.google.com/document/d/1bjcoDNk0pESBSEP2fD8SmqDZX5akxbWkwdG-EtZ9gJo/edit
Meeting recording:
Meeting Notes:
Attendees: Daniel Beutel, Layne Peng, Prashant Shah, Hasan Kassem (MLCommons), Patrick Foley, Sarita Regmi, Micah Sheller
Summary:
Meeting started with a high level project update. The TSC discussed the need for Federated Learning standardization, and called for a vote on a standards repo [offline vote]. Brainstormed how to better publicize OpenFL and bring new contributors and maintainers into the project.
ARs:
- TSC Members: Vote on creation of FL Standards repo in securefederatedai Github organization
- Micah to talk to reach out to Duke students about working on OpenFL
- Micah: to add OpenFL / MedPerf CI example to Medperf repo
- Patrick to read about baseline if we can adapt it back to OpenFL: https://flower.dev/docs/contributing-baselines.html.
Community Acknowledgements:
- Thank you to Layne for presenting FedLCM at the OpenFL Community meeting!
Project Update
- Intel Open Source Software Hackathon
- April 25-28th, 5-10 Intel teams contributing
- Part of our effort to extend our boundary beyond the core team
- This will be used as practice for hosting a true open source hackathon
- Improving accessibility in OpenFL
- Intel is collaborating with a company to support accessibility. Documentation is being reviewed by members of this external team. For example: Accessible fonts used will be visibility impaired. (Completing in May)
- Accessibility issues to be added to OpenFL backlog. Addressed on an ongoing basis
- OpenFL Website and logo are underway. These will likely be ready by the next TSC meeting in May.
- OpenFL Security trending for release in mid Q2
Federated Learning Standards
- There are several reason why standardized components would benefit the wider FL community
- Framework interoperability - i.e. pairing a FATE / Flower Client with an OpenFL server, or vice versa. Allows for larger heterogeneous federations that bring strengths from individual implementations (ex. TFLite support for Flower, TEE’s with OpenFL)
- Standardization of communication mechanisms is a key step in achieving hybrid deployments (along with weight setting / extraction, task representation)
- Aggregation algorithm / research verification - standardization could allow for portability of aggregation algorithm implementations across frameworks
- Micah: Flower baselines could be a good source of inspiration for this
- Could allow for higher level interfaces to be built on top of OpenFL / Flower / FATE backends
- Micah: Excited for when standards solve common problems, and various teams can focus on their areas of specialty and cover more of the FL problem space. This is so large that a single entity cannot take it on by themselves. Governance and the way that models get persisted would also be good candidates for standardization.
- Federation governance / TEE support could eventually be in scope for use not just in OpenFL Security, but also Flower / FATE
- Daniel: Secure aggregation / DP: they are difficult to get exactly right. If each group tries to reinvent, not the best way to build a reliable ecosystem
- Code review is essential, but problems are easy to miss
- Many implementations get to the point where they don’t find the bug, and leads to research that needs to be restarted
- Prashant: What about using OpenFL and Flower in a combined federation. Operational OpenFL federation, and operational Flower federation - combine to increase the data universe?
- Micah:Another way this could plan out is for multiple federations that have already vetted a model / plan, and they want to combine but don’t want to go through vetting again. Hierarchy of federations in medicine. Multiple partner groups are pulling data at different levels. End up with a tree of data ownership
- Micah: Can these topics live in a shared document so we can brainstorm ideas live?
- Live note during the meeting
- Google doc might be the platform [i.e. HERE]
- Meetings are recorded
Brainstorming how to grow the OpenFL Community?
- We have been increasing public facing blogs in the last 6 months
- These span release announcements to research that builds on OpenFL
- ML privacy meter was a highlight from the OpenFL 1.5 release
- EDEN compression algorithm (VMWare)
- Other ideas:
- Starting with GaNDLF (UPenn) and how to make these models run in a federated way with OpenFL
- Blog highlighting the need and goals for Federated Learning Standards
- Create pipeline for contributors to become eventual project maintainers (akin to a technical promotion path)
- For future TSC members:
- Have a recommendation about contributing code back to the framework
- Not strict but recommendation
- Being hands on with implementation will help in setting the technical direction
- Micah: Standardization will help bring in a new community (and combined PR people) pushing cross contributions between frameworks.
- Incentivizing research:
- Not just writing blog posts, but helping researchers with a well defined tutorial that accompanies the blog would be great for them (Andrew Trask (head of PySyft)) puts a lot of personal effort into tutorials because they are so important for community building).
- This is a role that Intel Labs has helped with. OpenFL maintainers should prioritize this to the extent that they can
- Daniel: We should move from discussing standards in google doc and have some version control. Some community examples that have a well defined standards proposal process. :
- Python PEP: https://peps.python.org/pep-0000/
- Kubernetes enhancement proposals: https://github.com/kubernetes/enhancements/blob/master/keps/README.md
- Flower Lab: Work has continued on Baselines
- FedAvg, FedProx was just merged
- There is some work in taking research code and bringing it to the format of baselines (this is expected)
- We could adopt this, It tied to flower, the workload, standardization could help us reuse components: https://flower.dev/docs/contributing-baselines.html
- Prashant: Tests could be added to show collaboration between OpenFL / GaNDLF / Standards, without requiring larger corporate reviews.
- This is underway between OpenFL / GaNDLF (both repositories include tests verifying interoperability with the other)
- Micah: I should add OpenFL / MedPerf CI example to MedPerf
- Layne: Should host more virtual events and invite members of the TSC. Should reach out to more universities to use OpenFL in their research and work.
- Prashant: Could ask Spyros to have students do sample projects
- Micah: Students already work with OpenFL at UPenn and know it quite well. Duke, who is working on the data science algorithm for the RANO federation, could be a good candidate for working on OpenFL (AR)