Clustered Federated Learning with Hierarchical Knowledge Distillation

• URL: http://arxiv.org/abs/2512.10443v1
• Date: Thu, 11 Dec 2025 09:08:35 GMT
• Title: Clustered Federated Learning with Hierarchical Knowledge Distillation
• Authors: Sabtain Ahmad, Meerzhan Kanatbekova, Ivona Brandic, Atakan Aral,
• Abstract summary: Clustered Federated Learning (CFL) has emerged as a powerful approach for addressing data heterogeneity and ensuring privacy in large distributed IoT environments.<n>This paper advocates a shift to hierarchical CFL, allowing bi-level aggregation to train cluster-specific models at the edge and a unified global model at the cloud.<n>We propose CFLHKD, a novel personalization scheme for integrating hierarchical cluster knowledge into CFL.
• Score: 1.587618661944613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clustered Federated Learning (CFL) has emerged as a powerful approach for addressing data heterogeneity and ensuring privacy in large distributed IoT environments. By clustering clients and training cluster-specific models, CFL enables personalized models tailored to groups of heterogeneous clients. However, conventional CFL approaches suffer from fragmented learning for training independent global models for each cluster and fail to take advantage of collective cluster insights. This paper advocates a shift to hierarchical CFL, allowing bi-level aggregation to train cluster-specific models at the edge and a unified global model at the cloud. This shift improves training efficiency yet might introduce communication challenges. To this end, we propose CFLHKD, a novel personalization scheme for integrating hierarchical cluster knowledge into CFL. Built upon multi-teacher knowledge distillation, CFLHKD enables inter-cluster knowledge sharing while preserving cluster-specific personalization. CFLHKD adopts a bi-level aggregation to bridge the gap between local and global learning. Extensive evaluations of standard benchmark datasets demonstrate that CFLHKD outperforms representative baselines in cluster-specific and global model accuracy and achieves a performance improvement of 3.32-7.57\%.

Related papers

• You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering [73.48306836608124]
  DCBoost is a parameter-free plug-in designed to enhance the global feature structures of current deep clustering models.<n>By harnessing reliable local structural cues, our method aims to elevate clustering performance effectively.
  arXiv  Detail & Related papers  (2025-11-26T09:16:36Z)
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddings [1.966764032092535]
  We propose CLoVE, a novel algorithm for Clustered Federated Learning (CFL)<n>CLoVE utilizes client embeddings derived from model losses on client data, and leverages the insight that clients in the same cluster share similar loss values.<n>CLoVE is able to iteratively identify and separate clients from different clusters and optimize cluster-specific models.
  arXiv  Detail & Related papers  (2025-06-27T17:52:16Z)
• Distilling A Universal Expert from Clustered Federated Learning [23.801864975543122]
  Clustered Federated Learning (CFL) addresses the challenges posed by non-IID data by training multiple group- or cluster-specific expert models.<n>This paper introduces a novel FL framework that distills a universal expert model from the knowledge of multiple clusters.
  arXiv  Detail & Related papers  (2025-06-25T09:44:39Z)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning [58.92159838586751]
  Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy.<n>We propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution.<n>Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy.
  arXiv  Detail & Related papers  (2025-02-05T16:33:36Z)
• LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data [21.341280782748278]
  Clustered federated learning (CFL) addresses the performance challenges posed by data heterogeneity in federated learning (FL)<n>Existing CFL approaches strictly limit knowledge sharing to within clusters, lacking the integration of global knowledge with intra-cluster training.<n>We propose LCFed, an efficient CFL framework to combat these challenges.
  arXiv  Detail & Related papers  (2025-01-03T14:59:48Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Contrastive encoder pre-training-based clustered federated learning for heterogeneous data [17.580390632874046]
  Federated learning (FL) enables distributed clients to collaboratively train a global model while preserving their data privacy. We propose contrastive pre-training-based clustered federated learning (CP-CFL) to improve the model convergence and overall performance of FL systems.
  arXiv  Detail & Related papers  (2023-11-28T05:44:26Z)
• Federated cINN Clustering for Accurate Clustered Federated Learning [33.72494731516968]
  Federated Learning (FL) presents an innovative approach to privacy-preserving distributed machine learning. We propose the Federated cINN Clustering Algorithm (FCCA) to robustly cluster clients into different groups.
  arXiv  Detail & Related papers  (2023-09-04T10:47:52Z)
• Federated Generalized Category Discovery [68.35420359523329]
  Generalized category discovery (GCD) aims at grouping unlabeled samples from known and unknown classes. To meet the recent decentralization trend in the community, we introduce a practical yet challenging task, namely Federated GCD (Fed-GCD) The goal of Fed-GCD is to train a generic GCD model by client collaboration under the privacy-protected constraint.
  arXiv  Detail & Related papers  (2023-05-23T14:27:41Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.