FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity

• URL: http://arxiv.org/abs/2305.05230v2
• Date: Tue, 1 Aug 2023 10:18:08 GMT
• Title: FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneity
• Authors: Nannan Wu, Li Yu, Xuefeng Jiang, Kwang-Ting Cheng, Zengqiang Yan
• Abstract summary: Federated noisy label learning (FNLL) is emerging as a promising tool for privacy-preserving decentralized learning. We first formulate a new and more realistic federated label noise problem where global data is class-imbalanced and label noise is heterogeneous. We propose a two-stage framework named FedNoRo for noise-robust federated learning.
• Score: 29.68112244504151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated noisy label learning (FNLL) is emerging as a promising tool for privacy-preserving multi-source decentralized learning. Existing research, relying on the assumption of class-balanced global data, might be incapable to model complicated label noise, especially in medical scenarios. In this paper, we first formulate a new and more realistic federated label noise problem where global data is class-imbalanced and label noise is heterogeneous, and then propose a two-stage framework named FedNoRo for noise-robust federated learning. Specifically, in the first stage of FedNoRo, per-class loss indicators followed by Gaussian Mixture Model are deployed for noisy client identification. In the second stage, knowledge distillation and a distance-aware aggregation function are jointly adopted for noise-robust federated model updating. Experimental results on the widely-used ICH and ISIC2019 datasets demonstrate the superiority of FedNoRo against the state-of-the-art FNLL methods for addressing class imbalance and label noise heterogeneity in real-world FL scenarios.

Related papers

• FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels [99.70895640578816]
  Federated learning with noisy labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning. We present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter.
  arXiv  Detail & Related papers  (2023-12-19T15:46:47Z)
• Federated Learning with Instance-Dependent Noisy Label [6.093214616626228]
  FedBeat aims to build a global statistically consistent classifier using the IDN transition matrix (IDNTM) Experiments conducted on CIFAR-10 and SVHN verify that the proposed method significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-16T05:08:02Z)
• FedNoisy: Federated Noisy Label Learning Benchmark [53.73816587601204]
  Federated learning has gained popularity for distributed learning without aggregating sensitive data from clients. The distributed and isolated nature of data isolation may be complicated by data quality, making it more vulnerable to noisy labels. We serve the first standardized benchmark that can help researchers fully explore potential federated noisy settings.
  arXiv  Detail & Related papers  (2023-06-20T16:18:14Z)
• Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients [4.782145666637457]
  Federated learning (FL) is a distributed framework for collaboratively training with privacy guarantees. In real-world scenarios, clients may have Non-IID data (local class imbalance) with poor annotation quality (label noise) We propose FedCNI without using an additional clean proxy dataset. It includes a noise-resilient local solver and a robust global aggregator.
  arXiv  Detail & Related papers  (2023-04-06T06:47:14Z)
• Quantifying the Impact of Label Noise on Federated Learning [7.531486350989069]
  Federated Learning (FL) is a distributed machine learning paradigm where clients collaboratively train a model using their local (human-generated) datasets. This paper provides a quantitative study on the impact of label noise on FL. Our empirical results show that the global model accuracy linearly decreases as the noise level increases.
  arXiv  Detail & Related papers  (2022-11-15T00:40:55Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations [54.400167806154535]
  Existing research on learning with noisy labels mainly focuses on synthetic label noise. This work presents two new benchmark datasets (CIFAR-10N, CIFAR-100N) We show that real-world noisy labels follow an instance-dependent pattern rather than the classically adopted class-dependent ones.
  arXiv  Detail & Related papers  (2021-10-22T22:42:11Z)
• Training Classifiers that are Universally Robust to All Label Noise Levels [91.13870793906968]
  Deep neural networks are prone to overfitting in the presence of label noise. We propose a distillation-based framework that incorporates a new subcategory of Positive-Unlabeled learning. Our framework generally outperforms at medium to high noise levels.
  arXiv  Detail & Related papers  (2021-05-27T13:49:31Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.