FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning

• URL: http://arxiv.org/abs/2403.16561v1
• Date: Mon, 25 Mar 2024 09:24:05 GMT
• Title: FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning
• Authors: Xinyuan Ji, Zhaowei Zhu, Wei Xi, Olga Gadyatskaya, Zilong Song, Yong Cai, Yang Liu,
• Abstract summary: Federated Learning (FL) heavily depends on label quality for its performance. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples. We propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples.
• Score: 15.382625332503125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) heavily depends on label quality for its performance. However, the label distribution among individual clients is always both noisy and heterogeneous. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples, impacting the effectiveness of existing label noise learning approaches. To tackle this issue, we propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples. In the dual models, updating the personalized model solely at a local level can lead to overfitting on noisy data due to limited samples, consequently affecting both the local and global models' performance. To mitigate overfitting, we address this concern from two perspectives. Firstly, we employ a confidence regularizer to alleviate the impact of unconfident predictions caused by label noise. Secondly, a distance regularizer is implemented to constrain the disparity between the personalized and global models. We validate the effectiveness of FedFixer through extensive experiments on benchmark datasets. The results demonstrate that FedFixer can perform well in filtering noisy label samples on different clients, especially in highly heterogeneous label noise scenarios.

Related papers

• Tackling Noisy Clients in Federated Learning with End-to-end Label Correction [20.64304520865249]
  We propose a two-stage framework FedELC to tackle this complicated label noise issue. The first stage aims to guide the detection of noisy clients with higher label noise. The second stage aims to correct the labels of noisy clients' data via an end-to-end label correction framework.
  arXiv  Detail & Related papers  (2024-08-08T08:35:32Z)
• Federated Learning with Extremely Noisy Clients via Negative Distillation [70.13920804879312]
  Federated learning (FL) has shown remarkable success in cooperatively training deep models, while struggling with noisy labels. We propose a novel approach, called negative distillation (FedNed) to leverage models trained on noisy clients. FedNed first identifies noisy clients and employs rather than discards the noisy clients in a knowledge distillation manner.
  arXiv  Detail & Related papers  (2023-12-20T01:59:48Z)
• FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels [99.70895640578816]
  Federated learning with noisy labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning. We present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter.
  arXiv  Detail & Related papers  (2023-12-19T15:46:47Z)
• Neighborhood Collective Estimation for Noisy Label Identification and Correction [92.20697827784426]
  Learning with noisy labels (LNL) aims at designing strategies to improve model performance and generalization by mitigating the effects of model overfitting to noisy labels. Recent advances employ the predicted label distributions of individual samples to perform noise verification and noisy label correction, easily giving rise to confirmation bias. We propose Neighborhood Collective Estimation, in which the predictive reliability of a candidate sample is re-estimated by contrasting it against its feature-space nearest neighbors.
  arXiv  Detail & Related papers  (2022-08-05T14:47:22Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• FedCorr: Multi-Stage Federated Learning for Label Noise Correction [80.9366438220228]
  Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables clients to jointly train a global model. We propose $textttFedCorr$, a general multi-stage framework to tackle heterogeneous label noise in FL. Experiments conducted on CIFAR-10/100 with federated synthetic label noise, and on a real-world noisy dataset, Clothing1M, demonstrate that $textttFedCorr$ is robust to label noise.
  arXiv  Detail & Related papers  (2022-04-10T12:51:18Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.