UA-PDFL: A Personalized Approach for Decentralized Federated Learning

• URL: http://arxiv.org/abs/2412.11674v1
• Date: Mon, 16 Dec 2024 11:27:35 GMT
• Title: UA-PDFL: A Personalized Approach for Decentralized Federated Learning
• Authors: Hangyu Zhu, Yuxiang Fan, Zhenping Xie,
• Abstract summary: Federated learning (FL) is a privacy preserving machine learning paradigm designed to collaboratively learn a global model without data leakage. To mitigate this issue, decentralized federated learning (DFL) has been proposed, where all participating clients engage in peer-to-peer communication without a central server. We propose a novel unit representation aided personalized decentralized federated learning framework, named UA-PDFL, to deal with the non-IID challenge in DFL.
• Score: 5.065947993017158
• License:
• Abstract: Federated learning (FL) is a privacy preserving machine learning paradigm designed to collaboratively learn a global model without data leakage. Specifically, in a typical FL system, the central server solely functions as an coordinator to iteratively aggregate the collected local models trained by each client, potentially introducing single-point transmission bottleneck and security threats. To mitigate this issue, decentralized federated learning (DFL) has been proposed, where all participating clients engage in peer-to-peer communication without a central server. Nonetheless, DFL still suffers from training degradation as FL does due to the non-independent and identically distributed (non-IID) nature of client data. And incorporating personalization layers into DFL may be the most effective solutions to alleviate the side effects caused by non-IID data. Therefore, in this paper, we propose a novel unit representation aided personalized decentralized federated learning framework, named UA-PDFL, to deal with the non-IID challenge in DFL. By adaptively adjusting the level of personalization layers through the guidance of the unit representation, UA-PDFL is able to address the varying degrees of data skew. Based on this scheme, client-wise dropout and layer-wise personalization are proposed to further enhance the learning performance of DFL. Extensive experiments empirically prove the effectiveness of our proposed method.

Related papers

• De-VertiFL: A Solution for Decentralized Vertical Federated Learning [7.877130417748362]
  This work introduces De-VertiFL, a novel solution for training models in a decentralized VFL setting. De-VertiFL contributes by introducing a new network architecture distribution, an innovative knowledge exchange scheme, and a distributed federated training process. The results demonstrate that De-VertiFL generally surpasses state-of-the-art methods in F1-score performance, while maintaining a decentralized and privacy-preserving framework.
  arXiv  Detail & Related papers  (2024-10-08T15:31:10Z)
• FedMAP: Unlocking Potential in Personalized Federated Learning through Bi-Level MAP Optimization [11.040916982022978]
  Federated Learning (FL) enables collaborative training of machine learning models on decentralized data. Data across clients often differs significantly due to class imbalance, feature distribution skew, sample size imbalance, and other phenomena. We propose a novel Bayesian PFL framework using bi-level optimization to tackle the data heterogeneity challenges.
  arXiv  Detail & Related papers  (2024-05-29T11:28:06Z)
• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training [67.67045085186797]
  Almost all existing systems have to face large communication burdens if the central FL server fails. It personalizes the "right" in the deep models by alternately updating the shared and personal parameters. To further promote the shared parameters aggregation process, we propose DFed integrating the local Sharpness Miniization.
  arXiv  Detail & Related papers  (2023-05-24T13:52:18Z)
• Personalized Federated Learning with Local Attention [5.018560254008613]
  Federated Learning (FL) aims to learn a single global model that enables the central server to help the model training in local clients without accessing their local data. Key challenge of FL is the heterogeneous label distribution and feature shift, which could lead to significant performance degradation of the learned models. We propose a simple yet effective algorithm, namely textbfpersonalized textbfFederated learning with textbfLocal textbfAttention (pFedLA) Two modules are proposed in pFedLA, i.e., the personalized
  arXiv  Detail & Related papers  (2023-04-02T20:10:32Z)
• Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning [0.0]
  Federated learning (FL) is a promising decentralized deep learning (DL) framework that enables DL-based approaches trained collaboratively across clients without sharing private data. In this paper, we propose a novel framework, namely Personalized Privacy-Preserving Federated Learning (PPPFL) Our proposed framework outperforms multiple FL baselines on different datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100.
  arXiv  Detail & Related papers  (2023-02-22T07:24:08Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation [119.19061102064497]
  We propose a decentralized FL framework by integrating blockchain into FL, namely, blockchain assisted decentralized federated learning (BLADE-FL) In a round of the proposed BLADE-FL, each client broadcasts its trained model to other clients, competes to generate a block based on the received models, and then aggregates the models from the generated block before its local training of the next round. We explore the impact of lazy clients on the learning performance of BLADE-FL, and characterize the relationship among the optimal K, the learning parameters, and the proportion of lazy clients.
  arXiv  Detail & Related papers  (2021-01-18T07:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.