FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2511.22265v1
• Date: Thu, 27 Nov 2025 09:42:22 GMT
• Title: FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning
• Authors: Yuan Yao, Lixu Wang, Jiaqi Wu, Jin Song, Simin Chen, Zehua Wang, Zijian Tian, Wei Chen, Huixia Li, Xiaoxiao Li,
• Abstract summary: Federated Representation Entanglement (FedRE) is a framework built upon a novel form of client knowledge termed entangled representation.<n>FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead.
• Score: 46.7918908485135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) enables collaborative training across clients without compromising privacy. While most existing FL methods assume homogeneous model architectures, client heterogeneity in data and resources renders this assumption impractical, motivating model-heterogeneous FL. To address this problem, we propose Federated Representation Entanglement (FedRE), a framework built upon a novel form of client knowledge termed entangled representation. In FedRE, each client aggregates its local representations into a single entangled representation using normalized random weights and applies the same weights to integrate the corresponding one-hot label encodings into the entangled-label encoding. Those are then uploaded to the server to train a global classifier. During training, each entangled representation is supervised across categories via its entangled-label encoding, while random weights are resampled each round to introduce diversity, mitigating the global classifier's overconfidence and promoting smoother decision boundaries. Furthermore, each client uploads a single cross-category entangled representation along with its entangled-label encoding, mitigating the risk of representation inversion attacks and reducing communication overhead. Extensive experiments demonstrate that FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead. The codes are available at https://github.com/AIResearch-Group/FedRE.

Related papers

• FedAPT: Federated Adversarial Prompt Tuning for Vision-Language Models [97.35577473867296]
  Federated Adversarial Prompt Tuning (textbfFedAPT) is a novel method designed to enhance the adversarial robustness of FPT.<n>To address this issue, we propose a textbfclass-aware prompt generator that generates visual prompts from text prompts.<n>Experiments on multiple image classification datasets demonstrate the superiority of FedAPT in improving adversarial robustness.
  arXiv  Detail & Related papers  (2025-09-03T03:46:35Z)
• Recovering Global Data Distribution Locally in Federated Learning [7.885010255812708]
  Federated Learning (FL) is a distributed machine learning paradigm that enables collaboration among multiple clients. A major challenge in FL is the label imbalance, where clients may exclusively possess certain classes while having numerous minority and missing classes. We propose a novel approach ReGL to address this challenge, whose key idea is to Recover the Global data distribution Locally.
  arXiv  Detail & Related papers  (2024-09-21T08:35:04Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• FedCME: Client Matching and Classifier Exchanging to Handle Data Heterogeneity in Federated Learning [5.21877373352943]
  Data heterogeneity across clients is one of the key challenges in Federated Learning (FL) We propose a novel FL framework named FedCME by client matching and classifier exchanging. Experimental results demonstrate that FedCME performs better than FedAvg, FedProx, MOON and FedRS on popular federated learning benchmarks.
  arXiv  Detail & Related papers  (2023-07-17T15:40:45Z)
• No One Left Behind: Real-World Federated Class-Incremental Learning [111.77681016996202]
  Local-Global Anti-forgetting (LGA) model addresses local and global catastrophic forgetting. We develop a category-balanced gradient-adaptive compensation loss and a category gradient-induced semantic distillation loss. It augments perturbed prototype images of new categories collected from local clients via self-supervised prototype augmentation.
  arXiv  Detail & Related papers  (2023-02-02T06:41:02Z)
• Federated Learning with Intermediate Representation Regularization [14.01585596739954]
  Federated learning (FL) enables remote clients to collaboratively train a model without exposing their private data. Previous studies accomplish this by regularizing the distance between the representations learned by the local and global models. We introduce FedIntR, which provides a more fine-grained regularization by integrating the representations of intermediate layers into the local training process.
  arXiv  Detail & Related papers  (2022-10-28T01:43:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.