FedX: Adaptive Model Decomposition and Quantization for IoT Federated Learning

• URL: http://arxiv.org/abs/2504.12849v2
• Date: Sat, 19 Apr 2025 15:18:12 GMT
• Title: FedX: Adaptive Model Decomposition and Quantization for IoT Federated Learning
• Authors: Phung Lai, Xiaopeng Jiang, Hai Phan, Cristian Borcea, Khang Tran, An Chen, Vijaya Datta Mayyuri, Ruoming Jin,
• Abstract summary: Federated Learning (FL) allows collaborative training among multiple devices without data sharing.<n>This paper proposes FedX, a novel adaptive model decomposition and quantization FL system for IoT.<n>We show that FedX significantly improves quantization times by up to 8.43X, on-device computation time by 1.5X, and total end-to-end training time by 1.36X, compared with baseline FL systems.
• Score: 12.271321978561183
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) allows collaborative training among multiple devices without data sharing, thus enabling privacy-sensitive applications on mobile or Internet of Things (IoT) devices, such as mobile health and asset tracking. However, designing an FL system with good model utility that works with low computation/communication overhead on heterogeneous, resource-constrained mobile/IoT devices is challenging. To address this problem, this paper proposes FedX, a novel adaptive model decomposition and quantization FL system for IoT. To balance utility with resource constraints on IoT devices, FedX decomposes a global FL model into different sub-networks with adaptive numbers of quantized bits for different devices. The key idea is that a device with fewer resources receives a smaller sub-network for lower overhead but utilizes a larger number of quantized bits for higher model utility, and vice versa. The quantization operations in FedX are done at the server to reduce the computational load on devices. FedX iteratively minimizes the losses in the devices' local data and in the server's public data using quantized sub-networks under a regularization term, and thus it maximizes the benefits of combining FL with model quantization through knowledge sharing among the server and devices in a cost-effective training process. Extensive experiments show that FedX significantly improves quantization times by up to 8.43X, on-device computation time by 1.5X, and total end-to-end training time by 1.36X, compared with baseline FL systems. We guarantee the global model convergence theoretically and validate local model convergence empirically, highlighting FedX's optimization efficiency.

Related papers

• Federated Split Learning with Model Pruning and Gradient Quantization in Wireless Networks [7.439160287320074]
  Federated split learning (FedSL) implements collaborative training across the edge devices and the server through model splitting.<n>We propose a lightweight FedSL scheme, that further alleviates the training burden on resource-constrained edge devices.<n>We conduct theoretical analysis to quantify the convergence performance of the proposed scheme.
  arXiv  Detail & Related papers  (2024-12-09T11:43:03Z)
• FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing [14.938531944702193]
  We propose Federated Learning with Local Heterogeneous Sharing (FedLPS) FedLPS uses transfer learning to facilitate the deployment of multiple tasks on a single device by dividing the local model into a shareable encoder and task-specific encoders. FedLPS significantly outperforms the state-of-the-art (SOTA) FL frameworks by up to 4.88% and reduces the computational resource consumption by 21.3%.
  arXiv  Detail & Related papers  (2024-02-13T16:30:30Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• Adaptive Federated Pruning in Hierarchical Wireless Networks [69.6417645730093]
  Federated Learning (FL) is a privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. In this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
  arXiv  Detail & Related papers  (2023-05-15T22:04:49Z)
• Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System Optimization [82.12796238714589]
  We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation. We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
  arXiv  Detail & Related papers  (2023-05-04T09:26:03Z)
• Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks [103.22651843174471]
  This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which aggregates them into a quantized global model and synchronizes the devices. We show that the FL training process can be described as a Markov decision process and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations.
  arXiv  Detail & Related papers  (2022-09-21T08:52:51Z)
• Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks [52.93232352968347]
  This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) FL preserves data privacy by exchanging the locally trained models of mobile devices. SNNs are however non-trivial, particularly under wireless connections with time-varying channel conditions. We propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models.
  arXiv  Detail & Related papers  (2021-12-05T11:17:17Z)
• Computational Intelligence and Deep Learning for Next-Generation Edge-Enabled Industrial IoT [51.68933585002123]
  We investigate how to deploy computational intelligence and deep learning (DL) in edge-enabled industrial IoT networks. In this paper, we propose a novel multi-exit-based federated edge learning (ME-FEEL) framework. In particular, the proposed ME-FEEL can achieve an accuracy gain up to 32.7% in the industrial IoT networks with the severely limited resources.
  arXiv  Detail & Related papers  (2021-10-28T08:14:57Z)
• Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices [40.69663094185572]
  Federated learning (FL) is a popular framework for training an AI model using distributed mobile data in a wireless network. One main challenge confronting practical FL is that resource constrained devices struggle with the computation intensive task of updating a deep-neural network model. To tackle the challenge, in this paper, a federated dropout (FedDrop) scheme is proposed building on the classic dropout scheme for random model pruning.
  arXiv  Detail & Related papers  (2021-09-30T16:52:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.