Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew

• URL: http://arxiv.org/abs/2509.12544v2
• Date: Tue, 30 Sep 2025 21:06:23 GMT
• Title: Neural Collapse-Inspired Multi-Label Federated Learning under Label-Distribution Skew
• Authors: Can Peng, Yuyuan Liu, Yingyu Yang, Pramit Saha, Qianye Yang, J. Alison Noble,
• Abstract summary: Federated Learning (FL) enables collaborative model training across distributed clients.<n>Many real-world applications, particularly in domains such as medical imaging, often involve multi-label settings.<n>We propose a method to align feature distributions across clients and to learn high-quality, well-clustered representations.
• Score: 13.59179877525954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) enables collaborative model training across distributed clients while preserving data privacy. However, the performance of deep learning often deteriorates in FL due to decentralized and heterogeneous data. This challenge is further amplified in multi-label scenarios, where data exhibit complex characteristics such as label co-occurrence, inter-label dependency, and discrepancies between local and global label relationships. While most existing FL research primarily focuses on single-label classification, many real-world applications, particularly in domains such as medical imaging, often involve multi-label settings. In this paper, we address this important yet underexplored scenario in FL, where clients hold multi-label data with skewed label distributions. Neural Collapse (NC) describes a geometric structure in the latent feature space where features of each class collapse to their class mean with vanishing intra-class variance, and the class means form a maximally separated configuration. Motivated by this theory, we propose a method to align feature distributions across clients and to learn high-quality, well-clustered representations. To make the NC-structure applicable to multi-label settings, where image-level features may contain multiple semantic concepts, we introduce a feature disentanglement module that extracts semantically specific features. The clustering of these disentangled class-wise features is guided by a predefined shared NC structure, which mitigates potential conflicts between client models due to diverse local data distributions. In addition, we design regularisation losses to encourage compact clustering in the latent feature space. Experiments conducted on four benchmark datasets across eight diverse settings demonstrate that our approach outperforms existing methods, validating its effectiveness in this challenging FL scenario.

Related papers

• FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Drift [29.2377620193847]
  Federated learning (FL) enables collaborative training of a global model in a centralized server with data from multiple parties.<n>We propose FedPall, an FL framework that utilizes prototype-based adversarial learning to unify feature spaces and collaborative learning to reinforce class information within the features.<n> evaluation results on three representative feature-drifted datasets demonstrate FedPall's consistently superior performance in classification with feature-drifted data in the FL scenario.
  arXiv  Detail & Related papers  (2025-07-07T08:58:39Z)
• Self-supervised Latent Space Optimization with Nebula Variational Coding [87.20343320266215]
  This paper proposes a variational inference model which leads to a clustered embedding.<n>We introduce additional variables in the latent space, called textbfnebula anchors, that guide the latent variables to form clusters during training.<n>Since each latent feature can be labeled with the closest anchor, we also propose to apply metric learning in a self-supervised way to make the separation between clusters more explicit.
  arXiv  Detail & Related papers  (2025-06-02T08:13:32Z)
• FedEPA: Enhancing Personalization and Modality Alignment in Multimodal Federated Learning [9.531634844824596]
  Federated Learning (FL) enables decentralized model training across multiple parties while preserving privacy.<n>Most FL systems assume clients hold only unimodal data, limiting their real-world applicability.<n>We propose FedEPA, a novel FL framework for multimodal learning.
  arXiv  Detail & Related papers  (2025-04-16T12:32:37Z)
• Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Need [83.10178754323955]
  Few-shot learning enables models to generalize from only a few labeled examples.<n>We propose the Unbiased Max-Min Embedding Classification (UMMEC) Method, which addresses the key challenges in few-shot learning.<n>Our method significantly improves classification performance with minimal labeled data, advancing the state-of-the-art in annotatedL.
  arXiv  Detail & Related papers  (2025-03-28T07:23:07Z)
• An Enhanced Classification Method Based on Adaptive Multi-Scale Fusion for Long-tailed Multispectral Point Clouds [67.96583737413296]
  We propose an enhanced classification method based on adaptive multi-scale fusion for MPCs with long-tailed distributions.<n>In the training set generation stage, a grid-balanced sampling strategy is designed to reliably generate training samples from sparse labeled datasets.<n>In the feature learning stage, a multi-scale feature fusion module is proposed to fuse shallow features of land-covers at different scales.
  arXiv  Detail & Related papers  (2024-12-16T03:21:20Z)
• Boosting Federated Learning with FedEntOpt: Mitigating Label Skew by Entropy-Based Client Selection [13.851391819710367]
  Deep learning domains typically require an extensive amount of data for optimal performance.<n>FedEntOpt is designed to mitigate performance issues caused by label distribution skew.<n>It exhibits robust and superior performance in scenarios with low participation rates and client dropout.
  arXiv  Detail & Related papers  (2024-11-02T13:31:36Z)
• Federated Contrastive Learning for Personalized Semantic Communication [55.46383524190467]
  We design a federated contrastive learning framework aimed at supporting personalized semantic communication. FedCL enables collaborative training of local semantic encoders across multiple clients and a global semantic decoder owned by the base station. To tackle the semantic imbalance issue arising from heterogeneous datasets across distributed clients, we employ contrastive learning to train a semantic centroid generator.
  arXiv  Detail & Related papers  (2024-06-13T14:45:35Z)
• Decoupled Subgraph Federated Learning [57.588938805581044]
  We address the challenge of federated learning on graph-structured data distributed across multiple clients.<n>We present a novel framework for this scenario, named FedStruct, that harnesses deep structural dependencies.<n>We validate the effectiveness of FedStruct through experimental results conducted on six datasets for semi-supervised node classification.
  arXiv  Detail & Related papers  (2024-02-29T13:47:23Z)
• FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients [19.3885479917635]
  Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices. We propose FedAnchor, an innovative FSSL method that introduces a unique double-head structure, called anchor head, paired with the classification head trained exclusively on labeled anchor data on the server. Our approach mitigates the confirmation bias and overfitting issues associated with pseudo-labeling techniques based on high-confidence model prediction samples.
  arXiv  Detail & Related papers  (2024-02-15T18:48:21Z)
• FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation [15.298650496155508]
  Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. Existing FL methods face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift. We propose a pioneering framework called textitFLea, incorporating the following key components.
  arXiv  Detail & Related papers  (2023-12-04T20:24:09Z)
• Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation [1.121358474059223]
  Federated learning (FL) enables multiple sites to collaboratively train powerful deep models without compromising data privacy and security. Weakly supervised segmentation, which uses sparsely-grained supervision, is increasingly being paid attention to due to its great potential of reducing annotation costs. We propose a novel personalized FL framework for medical image segmentation, named FedICRA, which uniformly leverages heterogeneous weak supervision.
  arXiv  Detail & Related papers  (2023-04-12T06:32:08Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.