Overcoming label shift with target-aware federated learning

• URL: http://arxiv.org/abs/2411.03799v2
• Date: Tue, 26 Aug 2025 10:07:02 GMT
• Title: Overcoming label shift with target-aware federated learning
• Authors: Edvin Listo Zec, Adam Breitholtz, Fredrik D. Johansson,
• Abstract summary: Federated learning enables multiple actors to collaboratively train models without sharing private data.<n>A common reason is label shift -- that the label distributions differ between clients and the target domain.<n>We propose FedPALS, a principled and practical model aggregation scheme that adapts to label shifts to improve performance in the target domain.
• Score: 10.355835466049092
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning enables multiple actors to collaboratively train models without sharing private data. Existing algorithms are successful and well-justified in this task when the intended target domain, where the trained model will be used, shares data distribution with the aggregate of clients, but this is often violated in practice. A common reason is label shift -- that the label distributions differ between clients and the target domain. We demonstrate empirically that this can significantly degrade performance. To address this problem, we propose FedPALS, a principled and practical model aggregation scheme that adapts to label shifts to improve performance in the target domain by leveraging knowledge of label distributions at the central server. Our approach ensures unbiased updates under federated stochastic gradient descent which yields robust generalization across clients with diverse, label-shifted data. Extensive experiments on image classification tasks demonstrate that FedPALS consistently outperforms baselines by aligning model aggregation with the target domain. Our findings reveal that conventional federated learning methods suffer severely in cases of extreme label sparsity on clients, highlighting the critical need for target-aware aggregation as offered by FedPALS.

Related papers

• FedFusion: Federated Learning with Diversity- and Cluster-Aware Encoders for Robust Adaptation under Label Scarcity [33.17279604575767]
  FedFusion is a federated transfer-learning framework that unifies domain adaptation and frugal labelling.<n>Labelled teacher clients guide learner clients via confidence-filtered pseudo-labels and domain-adaptive transfer.<n>FedFusion consistently outperforms state-of-the-art baselines in accuracy, robustness, and fairness.
  arXiv  Detail & Related papers  (2025-09-23T16:46:06Z)
• Whispers of Data: Unveiling Label Distributions in Federated Learning Through Virtual Client Simulation [4.81392127803963]
  Federated Learning enables collaborative training of a global model across multiple geographically dispersed clients without the need for data sharing. It is susceptible to inference attacks, particularly label inference attacks. We propose a novel label distribution inference attack that is stable and adaptable to various scenarios.
  arXiv  Detail & Related papers  (2025-04-30T08:51:06Z)
• Optimizing Federated Learning by Entropy-Based Client Selection [13.851391819710367]
  Deep learning domains typically require an extensive amount of data for optimal performance. FedOptEnt is designed to mitigate performance issues caused by label distribution skew. The proposed method outperforms several state-of-the-art algorithms by up to 6% in classification accuracy.
  arXiv  Detail & Related papers  (2024-11-02T13:31:36Z)
• (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning [4.803231218533992]
  Federated Learning (FL) is a distributed machine learning framework that trains accurate global models while preserving clients' privacy-sensitive data. Most FL approaches assume that clients possess labeled data, which is often not the case in practice. We propose $(FL)2$, a robust training method for unlabeled clients using sharpness-aware consistency regularization.
  arXiv  Detail & Related papers  (2024-10-30T17:15:02Z)
• Federated Learning with Label-Masking Distillation [33.80340338038264]
  Federated learning provides a privacy-preserving manner to collaboratively train models on data distributed over multiple local clients. Due to the different user behavior of the client, label distributions between different clients are significantly different. We propose a label-masking distillation approach termed FedLMD to facilitate federated learning via perceiving the various label distributions of each client.
  arXiv  Detail & Related papers  (2024-09-20T00:46:04Z)
• Federated Learning with Only Positive Labels by Exploring Label Correlations [78.59613150221597]
  Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting. We propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC)
  arXiv  Detail & Related papers  (2024-04-24T02:22:50Z)
• FedAnchor: Enhancing Federated Semi-Supervised Learning with Label Contrastive Loss for Unlabeled Clients [19.3885479917635]
  Federated learning (FL) is a distributed learning paradigm that facilitates collaborative training of a shared global model across devices. We propose FedAnchor, an innovative FSSL method that introduces a unique double-head structure, called anchor head, paired with the classification head trained exclusively on labeled anchor data on the server. Our approach mitigates the confirmation bias and overfitting issues associated with pseudo-labeling techniques based on high-confidence model prediction samples.
  arXiv  Detail & Related papers  (2024-02-15T18:48:21Z)
• Probabilistic Test-Time Generalization by Variational Neighbor-Labeling [62.158807685159736]
  This paper strives for domain generalization, where models are trained exclusively on source domains before being deployed on unseen target domains. Probability pseudo-labeling of target samples to generalize the source-trained model to the target domain at test time. Variational neighbor labels that incorporate the information of neighboring target samples to generate more robust pseudo labels.
  arXiv  Detail & Related papers  (2023-07-08T18:58:08Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
  Clustered learning has been shown to produce promising results by grouping clients into clusters. Existing FL algorithms are essentially trying to group clients together with similar distributions. Prior FL algorithms attempt similarities indirectly during training.
  arXiv  Detail & Related papers  (2022-09-21T17:37:54Z)
• Anomaly Detection through Unsupervised Federated Learning [0.0]
  Federated learning is proving to be one of the most promising paradigms for leveraging distributed resources. We propose a novel method in which, through a preprocessing phase, clients are grouped into communities. The resulting anomaly detection model is then shared and used to detect anomalies within the clients of the same community.
  arXiv  Detail & Related papers  (2022-09-09T08:45:47Z)
• FedAvg with Fine Tuning: Local Updates Lead to Representation Learning [54.65133770989836]
  Federated Averaging (FedAvg) algorithm consists of alternating between a few local gradient updates at client nodes, followed by a model averaging update at the server. We show that the reason behind generalizability of the FedAvg's output is its power in learning the common data representation among the clients' tasks. We also provide empirical evidence demonstrating FedAvg's representation learning ability in federated image classification with heterogeneous data.
  arXiv  Detail & Related papers  (2022-05-27T00:55:24Z)
• Federated Multi-Target Domain Adaptation [99.93375364579484]
  Federated learning methods enable us to train machine learning models on distributed user data while preserving its privacy. We consider a more practical scenario where the distributed client data is unlabeled, and a centralized labeled dataset is available on the server. We propose an effective DualAdapt method to address the new challenges.
  arXiv  Detail & Related papers  (2021-08-17T17:53:05Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Your Classifier can Secretly Suffice Multi-Source Domain Adaptation [72.47706604261992]
  Multi-Source Domain Adaptation (MSDA) deals with the transfer of task knowledge from multiple labeled source domains to an unlabeled target domain. We present a different perspective to MSDA wherein deep models are observed to implicitly align the domains under label supervision.
  arXiv  Detail & Related papers  (2021-03-20T12:44:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.