REFT: Resource-Efficient Federated Training Framework for Heterogeneous
and Resource-Constrained Environments
- URL: http://arxiv.org/abs/2308.13662v2
- Date: Thu, 7 Mar 2024 04:50:04 GMT
- Title: REFT: Resource-Efficient Federated Training Framework for Heterogeneous
and Resource-Constrained Environments
- Authors: Humaid Ahmed Desai, Amr Hilal, Hoda Eldardiry
- Abstract summary: Federated Learning (FL) plays a critical role in distributed systems.
FL emerges as a privacy-enforcing sub-domain of machine learning.
We propose "Resource-Efficient Federated Training Framework for Heterogeneous and Resource-Constrained Environments"
- Score: 2.117841684082203
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Federated Learning (FL) plays a critical role in distributed systems. In
these systems, data privacy and confidentiality hold paramount importance,
particularly within edge-based data processing systems such as IoT devices
deployed in smart homes. FL emerges as a privacy-enforcing sub-domain of
machine learning that enables model training on client devices, eliminating the
necessity to share private data with a central server. While existing research
has predominantly addressed challenges pertaining to data heterogeneity, there
remains a current gap in addressing issues such as varying device capabilities
and efficient communication. These unaddressed issues raise a number of
implications in resource-constrained environments. In particular, the practical
implementation of FL-based IoT or edge systems is extremely inefficient. In
this paper, we propose "Resource-Efficient Federated Training Framework for
Heterogeneous and Resource-Constrained Environments (REFT)," a novel approach
specifically devised to address these challenges in resource-limited devices.
Our proposed method uses Variable Pruning to optimize resource utilization by
adapting pruning strategies to the computational capabilities of each client.
Furthermore, our proposed REFT technique employs knowledge distillation to
minimize the need for continuous bidirectional client-server communication.
This achieves a significant reduction in communication bandwidth, thereby
enhancing the overall resource efficiency. We conduct experiments for an image
classification task, and the results demonstrate the effectiveness of our
approach in resource-limited settings. Our technique not only preserves data
privacy and performance standards but also accommodates heterogeneous model
architectures, facilitating the participation of a broader array of diverse
client devices in the training process, all while consuming minimal bandwidth.
Related papers
- Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth
and Data Heterogeneity [14.313847382199059]
Federated quantization-based self-supervised learning scheme (Fed-QSSL) designed to address heterogeneity in FL systems.
Fed-QSSL deploys de-quantization, weighted aggregation and re-quantization, ultimately creating models personalized to both data distribution and specific infrastructure of each client's device.
arXiv Detail & Related papers (2023-12-20T19:11:19Z) - Filling the Missing: Exploring Generative AI for Enhanced Federated
Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data.
Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
arXiv Detail & Related papers (2023-10-21T12:07:04Z) - Adaptive Resource Allocation for Virtualized Base Stations in O-RAN with
Online Learning [60.17407932691429]
Open Radio Access Network systems, with their base stations (vBSs), offer operators the benefits of increased flexibility, reduced costs, vendor diversity, and interoperability.
We propose an online learning algorithm that balances the effective throughput and vBS energy consumption, even under unforeseeable and "challenging'' environments.
We prove the proposed solutions achieve sub-linear regret, providing zero average optimality gap even in challenging environments.
arXiv Detail & Related papers (2023-09-04T17:30:21Z) - Analysis and Optimization of Wireless Federated Learning with Data
Heterogeneity [72.85248553787538]
This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation.
We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE)
Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
arXiv Detail & Related papers (2023-08-04T04:18:01Z) - RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learning [11.329273673732217]
Federated learning is an effective way to enable AI over massive distributed nodes with security.
It is challenging to ensure the privacy while maintain a coupled multi-task learning across multiple base stations (BSs) and terminals.
In this paper, inspired by the natural cloud-BS-terminal hierarchy of cellular works, we provide a viable resource-aware hierarchical federated MTL (RHFedMTL) solution.
arXiv Detail & Related papers (2023-06-01T13:49:55Z) - Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner.
Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server.
This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
arXiv Detail & Related papers (2023-02-24T08:41:19Z) - ON-DEMAND-FL: A Dynamic and Efficient Multi-Criteria Federated Learning
Client Deployment Scheme [37.099990745974196]
We introduce an On-Demand-FL, a client deployment approach for federated learning.
We make use of containerization technology such as Docker to build efficient environments.
The Genetic algorithm (GA) is used to solve the multi-objective optimization problem.
arXiv Detail & Related papers (2022-11-05T13:41:19Z) - Local Learning Matters: Rethinking Data Heterogeneity in Federated
Learning [61.488646649045215]
Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
arXiv Detail & Related papers (2021-11-28T19:03:39Z) - Federated Learning over Wireless IoT Networks with Optimized
Communication and Resources [98.18365881575805]
Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention.
It is of interest to investigate fast responding and accurate FL schemes over wireless systems.
We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
arXiv Detail & Related papers (2021-10-22T13:25:57Z) - Communication-Efficient Hierarchical Federated Learning for IoT
Heterogeneous Systems with Imbalanced Data [42.26599494940002]
Federated learning (FL) is a distributed learning methodology that allows multiple nodes to cooperatively train a deep learning model.
This paper studies the potential of hierarchical FL in IoT heterogeneous systems.
It proposes an optimized solution for user assignment and resource allocation on multiple edge nodes.
arXiv Detail & Related papers (2021-07-14T08:32:39Z) - Differentially Private Federated Learning for Resource-Constrained
Internet of Things [24.58409432248375]
Federated learning is capable of analyzing the large amount of data from a distributed set of smart devices without requiring them to upload their data to a central place.
This paper proposes a novel federated learning framework called DP-PASGD for training a machine learning model efficiently from the data stored across resource-constrained smart devices in IoT.
arXiv Detail & Related papers (2020-03-28T04:32:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.