Architectural Patterns for the Design of Federated Learning Systems

• URL: http://arxiv.org/abs/2101.02373v1
• Date: Thu, 7 Jan 2021 05:11:09 GMT
• Title: Architectural Patterns for the Design of Federated Learning Systems
• Authors: Sin Kit Lo, Qinghua Lu, Liming Zhu, Hye-young Paik, Xiwei Xu, Chen Wang
• Abstract summary: Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. This paper presents a collection of architectural patterns to deal with the design challenges of federated learning systems.
• Score: 12.330671239159102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. A federated learning system can be viewed as a large-scale distributed system with different components and stakeholders as numerous client devices participate in federated learning. Designing a federated learning system requires software system design thinking apart from machine learning knowledge. Although much effort has been put into federated learning from the machine learning technique aspects, the software architecture design concerns in building federated learning systems have been largely ignored. Therefore, in this paper, we present a collection of architectural patterns to deal with the design challenges of federated learning systems. Architectural patterns present reusable solutions to a commonly occurring problem within a given context during software architecture design. The presented patterns are based on the results of a systematic literature review and include three client management patterns, four model management patterns, three model training patterns, and four model aggregation patterns. The patterns are associated to particular state transitions in a federated learning model lifecycle, serving as a guidance for effective use of the patterns in the design of federated learning systems.

Related papers

• Deep Learning and Machine Learning: Advancing Big Data Analytics and Management with Design Patterns [17.624263707781655]
  The book explores the application of classical software engineering patterns to optimize the development, maintenance, and scalability of big data analytics systems. Key design patterns such as Singleton, Factory, Observer, and Strategy are analyzed for their impact on model management, deployment strategies, and team collaboration. This volume is an essential resource for developers, researchers, and engineers aiming to enhance their technical expertise in both machine learning and software design.
  arXiv  Detail & Related papers  (2024-10-04T02:50:58Z)
• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systems [16.690434072032176]
  We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines on how a consistent framework can be built up with existing, or newly created, viewpoints.
  arXiv  Detail & Related papers  (2022-12-27T18:05:02Z)
• Decision Models for Selecting Federated Learning Architecture Patterns [7.468413169676602]
  We present a set of decision models for the selection of patterns for federated machine learning architecture design. Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns.
  arXiv  Detail & Related papers  (2022-04-28T05:07:26Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)
• FLRA: A Reference Architecture for Federated Learning Systems [8.180947044673639]
  Federated learning is an emerging machine learning paradigm that enables multiple devices to train models locally and formulate a global model, without sharing the clients' local data. We propose FLRA, a reference architecture for federated learning systems, which provides a template design for federated learning-based solutions.
  arXiv  Detail & Related papers  (2021-06-22T06:59:19Z)
• Federated Learning: A Signal Processing Perspective [144.63726413692876]
  Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. This article provides a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools.
  arXiv  Detail & Related papers  (2021-03-31T15:14:39Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)
• A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective [9.315446698757768]
  Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. We perform a systematic literature review from a software engineering perspective, based on 231 primary studies. Our data synthesis covers the lifecycle of federated learning system development that includes background understanding, requirement analysis, architecture design, implementation, and evaluation.
  arXiv  Detail & Related papers  (2020-07-22T11:59:54Z)
• Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems [71.14339738190202]
  democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this paper. The proposed algorithms demonstrate better results in the generalization performance of learning models in agents compared to the conventional FL algorithms.
  arXiv  Detail & Related papers  (2020-07-07T08:34:48Z)
• Distributed and Democratized Learning: Philosophy and Research Challenges [80.39805582015133]
  We propose a novel design philosophy called democratized learning (Dem-AI) Inspired by the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are self-organized in a hierarchical structure to collectively perform learning tasks more efficiently. We present a reference design as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields.
  arXiv  Detail & Related papers  (2020-03-18T08:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.