ErrorEraser: Unlearning Data Bias for Improved Continual Learning

• URL: http://arxiv.org/abs/2506.09347v1
• Date: Wed, 11 Jun 2025 02:54:29 GMT
• Title: ErrorEraser: Unlearning Data Bias for Improved Continual Learning
• Authors: Xuemei Cao, Hanlin Gu, Xin Yang, Bingjun Wei, Haoyang Liang, Xiangkun Wang, Tianrui Li,
• Abstract summary: Continual Learning primarily aims to retain knowledge to prevent catastrophic forgetting and transfer knowledge to facilitate learning new tasks.<n>We propose ErrorEraser, a universal plugin that removes erroneous memories caused by biases in CL.<n>ErrorEraser significantly mitigates the negative impact of data biases, achieving higher accuracy and lower forgetting rates across three types of CL methods.
• Score: 7.812604562865828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual Learning (CL) primarily aims to retain knowledge to prevent catastrophic forgetting and transfer knowledge to facilitate learning new tasks. Unlike traditional methods, we propose a novel perspective: CL not only needs to prevent forgetting, but also requires intentional forgetting.This arises from existing CL methods ignoring biases in real-world data, leading the model to learn spurious correlations that transfer and amplify across tasks. From feature extraction and prediction results, we find that data biases simultaneously reduce CL's ability to retain and transfer knowledge. To address this, we propose ErrorEraser, a universal plugin that removes erroneous memories caused by biases in CL, enhancing performance in both new and old tasks. ErrorEraser consists of two modules: Error Identification and Error Erasure. The former learns the probability density distribution of task data in the feature space without prior knowledge, enabling accurate identification of potentially biased samples. The latter ensures only erroneous knowledge is erased by shifting the decision space of representative outlier samples. Additionally, an incremental feature distribution learning strategy is designed to reduce the resource overhead during error identification in downstream tasks. Extensive experimental results show that ErrorEraser significantly mitigates the negative impact of data biases, achieving higher accuracy and lower forgetting rates across three types of CL methods. The code is available at https://github.com/diadai/ErrorEraser.

Related papers

• Accurate Forgetting for Heterogeneous Federated Continual Learning [89.08735771893608]
  We propose a new concept accurate forgetting (AF) and develop a novel generative-replay methodMethodwhich selectively utilizes previous knowledge in federated networks.<n>We employ a probabilistic framework based on a normalizing flow model to quantify the credibility of previous knowledge.
  arXiv  Detail & Related papers  (2025-02-20T02:35:17Z)
• Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning [51.177789437682954]
  Class-incremental learning (CIL) seeks to enable a model to sequentially learn new classes while retaining knowledge of previously learned ones.<n> Balancing flexibility and stability remains a significant challenge, particularly when the task ID is unknown.<n>We propose a novel semantic drift calibration method that incorporates mean shift compensation and covariance calibration.
  arXiv  Detail & Related papers  (2025-02-11T13:57:30Z)
• BiasPruner: Debiased Continual Learning for Medical Image Classification [20.6029805375464]
  We present BiasPruner, a CL framework that intentionally forgets spurious correlations in the training data that could lead to shortcut learning. During inference, BiasPruner employs a simple task-agnostic approach to select the best debiased subnetwork for predictions. We conduct experiments on three medical datasets for skin lesion classification and chest X-Ray classification and demonstrate that BiasPruner consistently outperforms SOTA CL methods in terms of classification performance and fairness.
  arXiv  Detail & Related papers  (2024-07-11T15:45:57Z)
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning [93.90047628101155]
  Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks.<n>To address this, some methods propose replaying data from previous tasks during new task learning.<n>However, it is not expected in practice due to memory constraints and data privacy issues.
  arXiv  Detail & Related papers  (2024-01-12T12:51:12Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification.<n>XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations.<n>Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• Continual Learning in the Presence of Spurious Correlation [23.999136417157597]
  We show that standard continual learning algorithms can transfer biases from one task to another, both forward and backward. We propose a plug-in method for debiasing-aware continual learning, dubbed as Group-class Balanced Greedy Sampling (BGS)
  arXiv  Detail & Related papers  (2023-03-21T14:06:12Z)
• Agree to Disagree: Diversity through Disagreement for Better Transferability [54.308327969778155]
  We propose D-BAT (Diversity-By-disAgreement Training), which enforces agreement among the models on the training data. We show how D-BAT naturally emerges from the notion of generalized discrepancy.
  arXiv  Detail & Related papers  (2022-02-09T12:03:02Z)
• Prototypes-Guided Memory Replay for Continual Learning [13.459792148030717]
  Continual learning (CL) refers to a machine learning paradigm that using only a small account of training samples and previously learned knowledge to enhance learning performance. The major difficulty in CL is catastrophic forgetting of previously learned tasks, caused by shifts in data distributions. We propose a memory-efficient CL method, incorporating it into an online meta-learning model.
  arXiv  Detail & Related papers  (2021-08-28T13:00:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.