Decision Models for Selecting Federated Learning Architecture Patterns

• URL: http://arxiv.org/abs/2204.13291v3
• Date: Fri, 28 Apr 2023 00:30:38 GMT
• Title: Decision Models for Selecting Federated Learning Architecture Patterns
• Authors: Sin Kit Lo, Qinghua Lu, Hye-Young Paik, Liming Zhu
• Abstract summary: We present a set of decision models for the selection of patterns for federated machine learning architecture design. Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns.
• Score: 7.468413169676602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated machine learning is growing fast in academia and industries as a solution to solve data hungriness and privacy issues in machine learning. Being a widely distributed system, federated machine learning requires various system design thinking. To better design a federated machine learning system, researchers have introduced multiple patterns and tactics that cover various system design aspects. However, the multitude of patterns leaves the designers confused about when and which pattern to adopt. In this paper, we present a set of decision models for the selection of patterns for federated machine learning architecture design based on a systematic literature review on federated machine learning, to assist designers and architects who have limited knowledge of federated machine learning. Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns. We also clarify the drawbacks of the patterns. We evaluated the decision models by mapping the decision patterns to concrete federated machine learning architectures by big tech firms to assess the models' correctness and usefulness. The evaluation results indicate that the proposed decision models are able to bring structure to the federated machine learning architecture design process and help explicitly articulate the design rationale.

Related papers

• The Buffer Mechanism for Multi-Step Information Reasoning in Language Models [52.77133661679439]
  Investigating internal reasoning mechanisms of large language models can help us design better model architectures and training strategies. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75% reduction in the training time required for the GPT-2 model.
  arXiv  Detail & Related papers  (2024-05-24T07:41:26Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systems [16.690434072032176]
  We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines on how a consistent framework can be built up with existing, or newly created, viewpoints.
  arXiv  Detail & Related papers  (2022-12-27T18:05:02Z)
• Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks [53.09649785009528]
  In this paper, we explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we propose Model Disassembling and Assembling. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task.
  arXiv  Detail & Related papers  (2022-03-25T05:27:28Z)
• Artefact Retrieval: Overview of NLP Models with Knowledge Base Access [18.098224374478598]
  This paper systematically describes the typology of artefacts (items retrieved from a knowledge base), retrieval mechanisms and the way these artefacts are fused into the model. Most of the focus is given to language models, though we also show how question answering, fact-checking and dialogue models fit into this system as well.
  arXiv  Detail & Related papers  (2022-01-24T13:15:33Z)
• Panoramic Learning with A Standardized Machine Learning Formalism [116.34627789412102]
  This paper presents a standardized equation of the learning objective, that offers a unifying understanding of diverse ML algorithms. It also provides guidance for mechanic design of new ML solutions, and serves as a promising vehicle towards panoramic learning with all experiences.
  arXiv  Detail & Related papers  (2021-08-17T17:44:38Z)
• FLRA: A Reference Architecture for Federated Learning Systems [8.180947044673639]
  Federated learning is an emerging machine learning paradigm that enables multiple devices to train models locally and formulate a global model, without sharing the clients' local data. We propose FLRA, a reference architecture for federated learning systems, which provides a template design for federated learning-based solutions.
  arXiv  Detail & Related papers  (2021-06-22T06:59:19Z)
• Architectural Patterns for the Design of Federated Learning Systems [12.330671239159102]
  Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. This paper presents a collection of architectural patterns to deal with the design challenges of federated learning systems.
  arXiv  Detail & Related papers  (2021-01-07T05:11:09Z)
• A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective [9.315446698757768]
  Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. We perform a systematic literature review from a software engineering perspective, based on 231 primary studies. Our data synthesis covers the lifecycle of federated learning system development that includes background understanding, requirement analysis, architecture design, implementation, and evaluation.
  arXiv  Detail & Related papers  (2020-07-22T11:59:54Z)
• Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems [71.14339738190202]
  democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this paper. The proposed algorithms demonstrate better results in the generalization performance of learning models in agents compared to the conventional FL algorithms.
  arXiv  Detail & Related papers  (2020-07-07T08:34:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.