Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels

• URL: http://arxiv.org/abs/2412.00452v1
• Date: Sat, 30 Nov 2024 11:57:26 GMT
• Title: Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels
• Authors: Yuxin Tian, Mouxing Yang, Yuhao Zhou, Jian Wang, Qing Ye, Tongliang Liu, Gang Niu, Jiancheng Lv,
• Abstract summary: We propose a novel approach dubbed Global Reviser for Federated Learning with Noisy Labels (FedGR) FedGR employs three novel modules to achieve noisy label sniffing and refining, local knowledge revising, and local model regularization. In brief, FedGR employs three novel modules to achieve noisy label sniffing and refining, local knowledge revising, and local model regularization.
• Score: 68.25846387483178
• License:
• Abstract: The success of most federated learning (FL) methods heavily depends on label quality, which is often inaccessible in real-world scenarios, such as medicine, leading to the federated label-noise (F-LN) problem. In this study, we observe that the global model of FL memorizes the noisy labels slowly. Based on the observations, we propose a novel approach dubbed Global Reviser for Federated Learning with Noisy Labels (FedGR) to enhance the label-noise robustness of FL. In brief, FedGR employs three novel modules to achieve noisy label sniffing and refining, local knowledge revising, and local model regularization. Specifically, the global model is adopted to infer local data proxies for global sample selection and refine incorrect labels. To maximize the utilization of local knowledge, we leverage the global model to revise the local exponential moving average (EMA) model of each client and distill it into the clients' models. Additionally, we introduce a global-to-local representation regularization to mitigate the overfitting of noisy labels. Extensive experiments on three F-LNL benchmarks against seven baseline methods demonstrate the effectiveness of the proposed FedGR.

Related papers

• Revisiting Early-Learning Regularization When Federated Learning Meets Noisy Labels [27.777781072683986]
  This paper revisits early-learning regularization, introducing an innovative strategy, Federated Label-mixture Regularization (FLR) FLR adeptly adapts to FL's complexities by generating new pseudo labels, blending local and global model predictions.
  arXiv  Detail & Related papers  (2024-02-08T02:21:33Z)
• FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels [99.70895640578816]
  Federated learning with noisy labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning. We present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter.
  arXiv  Detail & Related papers  (2023-12-19T15:46:47Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• The Role of Global Labels in Few-Shot Classification and How to Infer Them [55.64429518100676]
  Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks. We propose Meta Label Learning (MeLa), a novel algorithm that infers global labels and obtains robust few-shot models via standard classification.
  arXiv  Detail & Related papers  (2021-08-09T14:07:46Z)
• Preservation of the Global Knowledge by Not-True Self Knowledge Distillation in Federated Learning [8.474470736998136]
  In Federated Learning (FL), a strong global model is collaboratively learned by aggregating the clients' locally trained models. We observe that fitting on biased local distribution shifts the feature on global distribution and results in forgetting of global knowledge. We propose a simple yet effective framework Federated Local Self-Distillation (FedLSD), which utilizes the global knowledge on locally available data.
  arXiv  Detail & Related papers  (2021-06-06T11:51:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.