Decision Models for Selecting Federated Learning Architecture Patterns

• URL: http://arxiv.org/abs/2204.13291v3
• Date: Fri, 28 Apr 2023 00:30:38 GMT
• Title: Decision Models for Selecting Federated Learning Architecture Patterns
• Authors: Sin Kit Lo, Qinghua Lu, Hye-Young Paik, Liming Zhu
• Abstract summary: We present a set of decision models for the selection of patterns for federated machine learning architecture design. Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns.
• Score: 7.468413169676602
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated machine learning is growing fast in academia and industries as a solution to solve data hungriness and privacy issues in machine learning. Being a widely distributed system, federated machine learning requires various system design thinking. To better design a federated machine learning system, researchers have introduced multiple patterns and tactics that cover various system design aspects. However, the multitude of patterns leaves the designers confused about when and which pattern to adopt. In this paper, we present a set of decision models for the selection of patterns for federated machine learning architecture design based on a systematic literature review on federated machine learning, to assist designers and architects who have limited knowledge of federated machine learning. Each decision model maps functional and non-functional requirements of federated machine learning systems to a set of patterns. We also clarify the drawbacks of the patterns. We evaluated the decision models by mapping the decision patterns to concrete federated machine learning architectures by big tech firms to assess the models' correctness and usefulness. The evaluation results indicate that the proposed decision models are able to bring structure to the federated machine learning architecture design process and help explicitly articulate the design rationale.

Related papers

• A Survey of Model Architectures in Information Retrieval [64.75808744228067]
  We focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs) We conclude by discussing emerging challenges and future directions, including architectural optimizations for performance and scalability, handling of multimodal, multilingual data, and adaptation to novel application domains beyond traditional search paradigms.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• A quantitative framework for evaluating architectural patterns in ML systems [49.1574468325115]
  This study proposes a framework for quantitative assessment of architectural patterns in ML systems. We focus on scalability and performance metrics for cost-effective CPU-based inference.
  arXiv  Detail & Related papers  (2025-01-20T15:30:09Z)
• The Buffer Mechanism for Multi-Step Information Reasoning in Language Models [52.77133661679439]
  Investigating internal reasoning mechanisms of large language models can help us design better model architectures and training strategies. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75% reduction in the training time required for the GPT-2 model.
  arXiv  Detail & Related papers  (2024-05-24T07:41:26Z)
• Geometric Deep Learning for Computer-Aided Design: A Survey [85.79012726689511]
  This survey offers a comprehensive overview of learning-based methods in computer-aided design. It includes similarity analysis and retrieval, 2D and 3D CAD model synthesis, and CAD generation from point clouds. It provides a complete list of benchmark datasets and their characteristics, along with open-source codes that have propelled research in this domain.
  arXiv  Detail & Related papers  (2024-02-27T17:11:35Z)
• Compositional Generative Inverse Design [69.22782875567547]
  Inverse design, where we seek to design input variables in order to optimize an underlying objective function, is an important problem. We show that by instead optimizing over the learned energy function captured by the diffusion model, we can avoid such adversarial examples. In an N-body interaction task and a challenging 2D multi-airfoil design task, we demonstrate that by composing the learned diffusion model at test time, our method allows us to design initial states and boundary shapes.
  arXiv  Detail & Related papers  (2024-01-24T01:33:39Z)
• A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systems [16.690434072032176]
  We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines on how a consistent framework can be built up with existing, or newly created, viewpoints.
  arXiv  Detail & Related papers  (2022-12-27T18:05:02Z)
• Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks [53.09649785009528]
  In this paper, we explore a paradigm that does not require training to obtain new models. Similar to the birth of CNN inspired by receptive fields in the biological visual system, we propose Model Disassembling and Assembling. For model assembling, we present the alignment padding strategy and parameter scaling strategy to construct a new model tailored for a specific task.
  arXiv  Detail & Related papers  (2022-03-25T05:27:28Z)
• FLRA: A Reference Architecture for Federated Learning Systems [8.180947044673639]
  Federated learning is an emerging machine learning paradigm that enables multiple devices to train models locally and formulate a global model, without sharing the clients' local data. We propose FLRA, a reference architecture for federated learning systems, which provides a template design for federated learning-based solutions.
  arXiv  Detail & Related papers  (2021-06-22T06:59:19Z)
• Architectural Patterns for the Design of Federated Learning Systems [12.330671239159102]
  Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. This paper presents a collection of architectural patterns to deal with the design challenges of federated learning systems.
  arXiv  Detail & Related papers  (2021-01-07T05:11:09Z)
• A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective [9.315446698757768]
  Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. We perform a systematic literature review from a software engineering perspective, based on 231 primary studies. Our data synthesis covers the lifecycle of federated learning system development that includes background understanding, requirement analysis, architecture design, implementation, and evaluation.
  arXiv  Detail & Related papers  (2020-07-22T11:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.