AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems

• URL: http://arxiv.org/abs/2311.13166v2
• Date: Tue, 9 Apr 2024 05:43:38 GMT
• Title: AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems
• Authors: Chentao Jia, Ming Hu, Zekai Chen, Yanxin Yang, Xiaofei Xie, Yang Liu, Mingsong Chen,
• Abstract summary: Federated Learning (FL) is promising to enable collaborative learning among Artificial Intelligence of Things (AIoT) devices. This paper introduces an effective FL approach named AdaptiveFL based on a novel fine-grained width-wise model pruning strategy. We show that AdaptiveFL can achieve up to 16.83% inference improvements for both IID and non-IID scenarios.
• Score: 25.0282475069725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although Federated Learning (FL) is promising to enable collaborative learning among Artificial Intelligence of Things (AIoT) devices, it suffers from the problem of low classification performance due to various heterogeneity factors (e.g., computing capacity, memory size) of devices and uncertain operating environments. To address these issues, this paper introduces an effective FL approach named AdaptiveFL based on a novel fine-grained width-wise model pruning strategy, which can generate various heterogeneous local models for heterogeneous AIoT devices. By using our proposed reinforcement learning-based device selection mechanism, AdaptiveFL can adaptively dispatch suitable heterogeneous models to corresponding AIoT devices on the fly based on their available resources for local training. Experimental results show that, compared to state-of-the-art methods, AdaptiveFL can achieve up to 16.83% inference improvements for both IID and non-IID scenarios.

Related papers

• Towards Energy-Aware Federated Learning via MARL: A Dual-Selection Approach for Model and Client [16.67119399590236]
  We propose an energy-aware Federated Learning (FL) framework named DR-FL. DR-FL considers the energy constraints in both clients and heterogeneous deep learning models to enable energy-efficient FL. Unlike Vanilla FL, DR-FL adopts our proposed Muti-Agents Reinforcement Learning (MARL)-based dual-selection method.
  arXiv  Detail & Related papers  (2024-05-13T21:02:31Z)
• Efficient Language Model Architectures for Differentially Private Federated Learning [21.280600854272716]
  Cross-device federated learning (FL) is a technique that trains a model on data distributed across typically millions of edge devices without data leaving the devices. In centralized training of language models, adaptives are preferred as they offer improved stability and performance. We propose a scale-in Coupled Input Forget Gate (SI CIFG) recurrent network by modifying the sigmoid and tanh activations in neural recurrent cell.
  arXiv  Detail & Related papers  (2024-03-12T22:21:48Z)
• AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems [24.013937378054074]
  Federated Learning (FL) enables collaborative learning of large-scale distributed clients without data sharing. Mobile computing systems can only use small low-performance models for collaborative learning. We use a model reassemble strategy to facilitate collaborative training of massive heterogeneous mobile devices adaptively.
  arXiv  Detail & Related papers  (2023-11-23T14:42:43Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy [27.443439653087662]
  This paper introduces AQUILA (adaptive quantization in device selection strategy), a novel adaptive framework devised to handle these issues. AQUILA integrates a sophisticated device selection method that prioritizes the quality and usefulness of device updates. Our experiments demonstrate that AQUILA significantly decreases communication costs compared to existing methods.
  arXiv  Detail & Related papers  (2023-08-01T03:41:47Z)
• FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
  Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data. There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
  arXiv  Detail & Related papers  (2023-03-23T15:37:17Z)
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence [52.83633954857744]
  We investigate real-time machine learning in a federated edge intelligence (FEI) system. FEI systems exhibit heterogenous communication and computational resource distribution. We propose a time-sensitive federated learning (TS-FL) framework to minimize the overall run-time for collaboratively training a shared ML model.
  arXiv  Detail & Related papers  (2023-01-26T08:13:22Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• FedGPO: Heterogeneity-Aware Global Parameter Optimization for Efficient Federated Learning [11.093360539563657]
  Federated learning (FL) has emerged as a solution to deal with the risk of privacy leaks in machine learning training. We propose FedGPO to optimize the energy-efficiency of FL use cases while guaranteeing model convergence. In our experiments, FedGPO improves the model convergence time by 2.4 times, and achieves 3.6 times higher energy efficiency over the baseline settings.
  arXiv  Detail & Related papers  (2022-11-30T01:22:57Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• Reconfigurable Intelligent Surface Assisted Mobile Edge Computing with Heterogeneous Learning Tasks [53.1636151439562]
  Mobile edge computing (MEC) provides a natural platform for AI applications. We present an infrastructure to perform machine learning tasks at an MEC with the assistance of a reconfigurable intelligent surface (RIS) Specifically, we minimize the learning error of all participating users by jointly optimizing transmit power of mobile users, beamforming vectors of the base station, and the phase-shift matrix of the RIS.
  arXiv  Detail & Related papers  (2020-12-25T07:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.