FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2604.27510v1
• Date: Thu, 30 Apr 2026 07:08:52 GMT
• Title: FMCL: Class-Aware Client Clustering with Foundation Model Representations for Heterogeneous Federated Learning
• Abstract summary: Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data.<n> Clustered Federated Learning addresses this challenge by grouping similar clients and training separate models.<n>We propose FMCL, a one-shot, class-aware client clustering framework.
• Score: 0.14323566945483493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its performance deteriorates under statistical heterogeneity. Clustered Federated Learning addresses this challenge by grouping similar clients and training separate models per cluster. However, existing clustering strategies often rely on raw data statistics, model parameters, or heuristic similarity measures that fail to capture class-level semantic structure across heterogeneous domains and frequently require iterative coordination. We propose FMCL, a one-shot, class-aware client clustering framework that leverages foundation model representations to construct semantic client signatures. Using a frozen foundation model, FMCL computes class-level embedding prototypes for each client and measures similarity via cosine distance between their class-aware representations. Clustering is performed once prior to training, introducing no additional communication during federated optimization and remaining agnostic to the downstream model architecture. Extensive experiments across heterogeneous benchmarks demonstrate that FMCL improves federated performance and yields more stable clustering behavior compared to existing clustering-based methods under non-identically distributed data partitioning.

Related papers

• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments [11.797290397638962]
  Federated Learning (FL) enables a group of clients to collaboratively train a model without sharing individual data, but its performance drops when client data are heterogeneous.<n>This paper introduces a clustered FL framework, FedDAG, that employs a weighted, class-wise similarity metric that integrates both data and information.<n> Experiments on diverse benchmarks and data settings show that FedDAG consistently outperforms state-of-the-art clustered FL baselines in accuracy.
  arXiv  Detail & Related papers  (2026-02-26T21:20:19Z)
• One-Shot Hierarchical Federated Clustering [51.490181220883905]
  This paper introduces an efficient one-shot hierarchical Federated Clustering framework.<n>It performs client-end distribution exploration and server-end distribution aggregation.<n>It turns out that the complex cluster distributions across clients can be efficiently explored.
  arXiv  Detail & Related papers  (2026-01-10T02:58:33Z)
• Federated Multi-Task Clustering [44.73672172790804]
  This paper proposes a novel framework named Federated Multi-Task Clustering (i.e.,FMTC)<n>It is composed of two main components: client-side personalized clustering module and server-side tensorial correlation module.<n>We derive an efficient, privacy-preserving distributed algorithm based on the Alternating Direction Method of Multipliers.
  arXiv  Detail & Related papers  (2025-12-28T12:02:32Z)
• DPMM-CFL: Clustered Federated Learning via Dirichlet Process Mixture Model Nonparametric Clustering [8.645893961456801]
  Clustered Federated Learning improves performance under non-IID client heterogeneity.<n>Most CFL methods require the number of clusters K to be fixed a priori.<n>We propose DPMM-CFL, a CFL algorithm that places a Dirichlet Process (DP) prior over the distribution of cluster parameters.
  arXiv  Detail & Related papers  (2025-10-08T15:27:08Z)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning [58.92159838586751]
  Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy.<n>We propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution.<n>Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy.
  arXiv  Detail & Related papers  (2025-02-05T16:33:36Z)
• A Bayesian Framework for Clustered Federated Learning [14.426129993432193]
  One of the main challenges of federated learning (FL) is handling non-independent and identically distributed (non-IID) client data. We present a unified Bayesian framework for clustered FL which associates clients to clusters. This work provides insights on client-cluster associations and enables client knowledge sharing in new ways.
  arXiv  Detail & Related papers  (2024-10-20T19:11:24Z)
• CCFC: Bridging Federated Clustering and Contrastive Learning [8.822947930471429]
  We propose a new federated clustering method named cluster-contrastive federated clustering (CCFC)<n>CCFC shows superior performance in handling device failures from a practical viewpoint.
  arXiv  Detail & Related papers  (2024-01-12T15:26:44Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)
• On the Convergence of Clustered Federated Learning [57.934295064030636]
  In a federated learning system, the clients, e.g. mobile devices and organization participants, usually have different personal preferences or behavior patterns. This paper proposes a novel weighted client-based clustered FL algorithm to leverage the client's group and each client in a unified optimization framework.
  arXiv  Detail & Related papers  (2022-02-13T02:39:19Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.