AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems

• URL: http://arxiv.org/abs/2311.13166v2
• Date: Tue, 9 Apr 2024 05:43:38 GMT
• Title: AdaptiveFL: Adaptive Heterogeneous Federated Learning for Resource-Constrained AIoT Systems
• Authors: Chentao Jia, Ming Hu, Zekai Chen, Yanxin Yang, Xiaofei Xie, Yang Liu, Mingsong Chen,
• Abstract summary: Federated Learning (FL) is promising to enable collaborative learning among Artificial Intelligence of Things (AIoT) devices. This paper introduces an effective FL approach named AdaptiveFL based on a novel fine-grained width-wise model pruning strategy. We show that AdaptiveFL can achieve up to 16.83% inference improvements for both IID and non-IID scenarios.
• Score: 25.0282475069725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although Federated Learning (FL) is promising to enable collaborative learning among Artificial Intelligence of Things (AIoT) devices, it suffers from the problem of low classification performance due to various heterogeneity factors (e.g., computing capacity, memory size) of devices and uncertain operating environments. To address these issues, this paper introduces an effective FL approach named AdaptiveFL based on a novel fine-grained width-wise model pruning strategy, which can generate various heterogeneous local models for heterogeneous AIoT devices. By using our proposed reinforcement learning-based device selection mechanism, AdaptiveFL can adaptively dispatch suitable heterogeneous models to corresponding AIoT devices on the fly based on their available resources for local training. Experimental results show that, compared to state-of-the-art methods, AdaptiveFL can achieve up to 16.83% inference improvements for both IID and non-IID scenarios.

Related papers

• Gen-DFL: Decision-Focused Generative Learning for Robust Decision Making [48.62706690668867]
  Decision-focused generative learning (Gen-DFL) is a novel framework that leverages generative models to adaptively model uncertainty and improve decision quality. The paper shows, theoretically, that Gen-DFL achieves improved worst-case performance bounds compared to traditional DFL.
  arXiv  Detail & Related papers  (2025-02-08T06:52:11Z)
• Over-the-Air Fair Federated Learning via Multi-Objective Optimization [52.295563400314094]
  We propose an over-the-air fair federated learning algorithm (OTA-FFL) to train fair FL models. Experiments demonstrate the superiority of OTA-FFL in achieving fairness and robust performance.
  arXiv  Detail & Related papers  (2025-01-06T21:16:51Z)
• Learnable Sparse Customization in Heterogeneous Edge Computing [27.201987866208484]
  We propose Learnable Personalized Sparsification for heterogeneous Federated learning (FedLPS) FedLPS learns the importance of model units on local data representation and derives an importance-based sparse pattern with minimals to accurately extract personalized data features. Experiments show that FedLPS outperforms status quo approaches in accuracy and training costs.
  arXiv  Detail & Related papers  (2024-12-10T06:14:31Z)
• Towards Energy-Aware Federated Learning via MARL: A Dual-Selection Approach for Model and Client [16.67119399590236]
  We propose an energy-aware Federated Learning (FL) framework named DR-FL. DR-FL considers the energy constraints in both clients and heterogeneous deep learning models to enable energy-efficient FL. Unlike Vanilla FL, DR-FL adopts our proposed Muti-Agents Reinforcement Learning (MARL)-based dual-selection method.
  arXiv  Detail & Related papers  (2024-05-13T21:02:31Z)
• Efficient Language Model Architectures for Differentially Private Federated Learning [21.280600854272716]
  Cross-device federated learning (FL) is a technique that trains a model on data distributed across typically millions of edge devices without data leaving the devices. In centralized training of language models, adaptives are preferred as they offer improved stability and performance. We propose a scale-in Coupled Input Forget Gate (SI CIFG) recurrent network by modifying the sigmoid and tanh activations in neural recurrent cell.
  arXiv  Detail & Related papers  (2024-03-12T22:21:48Z)
• AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems [24.013937378054074]
  Federated Learning (FL) enables collaborative learning of large-scale distributed clients without data sharing. Mobile computing systems can only use small low-performance models for collaborative learning. We use a model reassemble strategy to facilitate collaborative training of massive heterogeneous mobile devices adaptively.
  arXiv  Detail & Related papers  (2023-11-23T14:42:43Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy [27.443439653087662]
  This paper introduces AQUILA (adaptive quantization in device selection strategy), a novel adaptive framework devised to handle these issues. AQUILA integrates a sophisticated device selection method that prioritizes the quality and usefulness of device updates. Our experiments demonstrate that AQUILA significantly decreases communication costs compared to existing methods.
  arXiv  Detail & Related papers  (2023-08-01T03:41:47Z)
• Time-sensitive Learning for Heterogeneous Federated Edge Intelligence [52.83633954857744]
  We investigate real-time machine learning in a federated edge intelligence (FEI) system. FEI systems exhibit heterogenous communication and computational resource distribution. We propose a time-sensitive federated learning (TS-FL) framework to minimize the overall run-time for collaboratively training a shared ML model.
  arXiv  Detail & Related papers  (2023-01-26T08:13:22Z)
• Faster Adaptive Federated Learning [84.38913517122619]
  Federated learning has attracted increasing attention with the emergence of distributed data. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on momentum-based variance reduced technique in cross-silo FL.
  arXiv  Detail & Related papers  (2022-12-02T05:07:50Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.