Where is the Testbed for my Federated Learning Research?

• URL: http://arxiv.org/abs/2407.14154v1
• Date: Fri, 19 Jul 2024 09:34:04 GMT
• Title: Where is the Testbed for my Federated Learning Research?
• Authors: Janez Božič, Amândio R. Faustino, Boris Radovič, Marco Canini, Veljko Pejović,
• Abstract summary: We present CoLExT, a real-world testbed for federated learning (FL) research. CoLExT is designed to streamline experimentation with custom FL algorithms in a rich testbed configuration space. Through an initial investigation involving popular FL algorithms running on CoLExT, we reveal previously unknown trade-offs, inefficiencies, and programming bugs.
• Score: 3.910931245706272
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Progressing beyond centralized AI is of paramount importance, yet, distributed AI solutions, in particular various federated learning (FL) algorithms, are often not comprehensively assessed, which prevents the research community from identifying the most promising approaches and practitioners from being convinced that a certain solution is deployment-ready. The largest hurdle towards FL algorithm evaluation is the difficulty of conducting real-world experiments over a variety of FL client devices and different platforms, with different datasets and data distribution, all while assessing various dimensions of algorithm performance, such as inference accuracy, energy consumption, and time to convergence, to name a few. In this paper, we present CoLExT, a real-world testbed for FL research. CoLExT is designed to streamline experimentation with custom FL algorithms in a rich testbed configuration space, with a large number of heterogeneous edge devices, ranging from single-board computers to smartphones, and provides real-time collection and visualization of a variety of metrics through automatic instrumentation. According to our evaluation, porting FL algorithms to CoLExT requires minimal involvement from the developer, and the instrumentation introduces minimal resource usage overhead. Furthermore, through an initial investigation involving popular FL algorithms running on CoLExT, we reveal previously unknown trade-offs, inefficiencies, and programming bugs.

Related papers

• Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study [1.9265466185360185]
  Federated Learning (FL) emerged as a practical approach to training a model from decentralized data. To bridge this gap, we conduct extensive performance evaluation on several canonical FL algorithms. Our comprehensive measurement study reveals that no single algorithm works best across different performance metrics.
  arXiv  Detail & Related papers  (2024-03-26T00:33:49Z)
• FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
  Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data. There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
  arXiv  Detail & Related papers  (2023-03-23T15:37:17Z)
• FLAGS Framework for Comparative Analysis of Federated Learning Algorithms [0.0]
  This work consolidates the Federated Learning landscape and offers an objective analysis of the major FL algorithms. To enable a uniform assessment, a multi-FL framework named FLAGS: Federated Learning AlGorithms Simulation has been developed. Our experiments indicate that fully decentralized FL algorithms achieve comparable accuracy under multiple operating conditions.
  arXiv  Detail & Related papers  (2022-12-14T12:08:30Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• FL_PyTorch: optimization research simulator for federated learning [1.6114012813668934]
  Federated Learning (FL) has emerged as a promising technique for edge devices to collaboratively learn a shared machine learning model. FL_PyTorch is a suite of open-source software written in python that builds on top of one the most popular research Deep Learning (DL) framework PyTorch.
  arXiv  Detail & Related papers  (2022-02-07T12:18:28Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Fast Federated Learning in the Presence of Arbitrary Device Unavailability [26.368873771739715]
  Federated Learning (FL) coordinates heterogeneous devices to collaboratively train a shared model while preserving user privacy. One challenge arises when devices drop out of the training process beyond the central server. We propose Im Federated Apatientaging (MIFA) to solve this problem.
  arXiv  Detail & Related papers  (2021-06-08T07:46:31Z)
• Hybrid Federated Learning: Algorithms and Implementation [61.0640216394349]
  Federated learning (FL) is a recently proposed distributed machine learning paradigm dealing with distributed and private data sets. We propose a new model-matching-based problem formulation for hybrid FL. We then propose an efficient algorithm that can collaboratively train the global and local models to deal with full and partial featured data.
  arXiv  Detail & Related papers  (2020-12-22T23:56:03Z)
• FedML: A Research Library and Benchmark for Federated Machine Learning [55.09054608875831]
  Federated learning (FL) is a rapidly growing research field in machine learning. Existing FL libraries cannot adequately support diverse algorithmic development. We introduce FedML, an open research library and benchmark to facilitate FL algorithm development and fair performance comparison.
  arXiv  Detail & Related papers  (2020-07-27T13:02:08Z)
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data [59.50904660420082]
  Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model.
  arXiv  Detail & Related papers  (2020-05-22T23:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.