Data-Free Generative Replay for Class-Incremental Learning on Imbalanced Data

• URL: http://arxiv.org/abs/2406.09052v1
• Date: Fri, 7 Jun 2024 17:51:27 GMT
• Title: Data-Free Generative Replay for Class-Incremental Learning on Imbalanced Data
• Authors: Sohaib Younis, Bernhard Seeger,
• Abstract summary: Continual learning is a challenging problem in machine learning, especially for image classification tasks with imbalanced datasets. This paper proposes Data-Free Generative Replay (DFGR) for class incremental learning, where the generator is trained without access to real data. DFGR achieves up to 88.5% and 46.6% accuracy on MNIST and FashionMNIST datasets, respectively.
• Score: 0.7366405857677227
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning is a challenging problem in machine learning, especially for image classification tasks with imbalanced datasets. It becomes even more challenging when it involves learning new classes incrementally. One method for incremental class learning, addressing dataset imbalance, is rehearsal using previously stored data. In rehearsal-based methods, access to previous data is required for either training the classifier or the generator, but it may not be feasible due to storage, legal, or data access constraints. Although there are many rehearsal-free alternatives for class incremental learning, such as parameter or loss regularization, knowledge distillation, and dynamic architectures, they do not consistently achieve good results, especially on imbalanced data. This paper proposes a new approach called Data-Free Generative Replay (DFGR) for class incremental learning, where the generator is trained without access to real data. In addition, DFGR also addresses dataset imbalance in continual learning of an image classifier. Instead of using training data, DFGR trains a generator using mean and variance statistics of batch-norm and feature maps derived from a pre-trained classification model. The results of our experiments demonstrate that DFGR performs significantly better than other data-free methods and reveal the performance impact of specific parameter settings. DFGR achieves up to 88.5% and 46.6% accuracy on MNIST and FashionMNIST datasets, respectively. Our code is available at https://github.com/2younis/DFGR

Related papers

• Statistically Testing Training Data for Unwanted Error Patterns using Rule-Oriented Regression [0.5831737970661137]
  We provide a method to test training data for flaws, to establish a trustworthy ground-truth for a subsequent training of machine learning models. Our approach extends the abilities of conventional statistical testing by letting the test-condition'' be any condition to describe a pattern in the data. We provide an open source implementation for demonstration and experiments.
  arXiv  Detail & Related papers  (2025-03-24T09:52:36Z)
• Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AI [17.242331892899543]
  Learning performance data describe correct and incorrect answers or problem-solving attempts in adaptive learning. Learning performance data tend to be highly sparse (80%(sim)90% missing observations) in most real-world applications due to adaptive item selection. This article proposes a systematic framework for augmenting learner data to address data sparsity in learning performance data.
  arXiv  Detail & Related papers  (2024-09-24T00:25:07Z)
• GENIU: A Restricted Data Access Unlearning for Imbalanced Data [7.854651232997996]
  Class unlearning involves enabling a trained model to forget data belonging to a specific class learned before. GENIU is the first practical framework for class unlearning imbalanced data settings and restricted data access.
  arXiv  Detail & Related papers  (2024-06-12T05:24:53Z)
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning [100.7407460674153]
  Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks. To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks. However, it is not expected in practice considering the memory constraint or data privacy issue. As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
  arXiv  Detail & Related papers  (2024-01-12T12:51:12Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• Adjusting Logit in Gaussian Form for Long-Tailed Visual Recognition [37.62659619941791]
  We study the problem of long-tailed visual recognition from the perspective of feature level. Two novel logit adjustment methods are proposed to improve model performance at a modest computational overhead. Experiments conducted on benchmark datasets demonstrate the superior performance of the proposed method over the state-of-the-art ones.
  arXiv  Detail & Related papers  (2023-05-18T02:06:06Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Reproducible, incremental representation learning with Rosetta VAE [0.0]
  Variational autoencoders are among the most popular methods for distilling low-dimensional structure from high-dimensional data. We introduce the Rosetta VAE, a method of distilling previously learned representations and retraining new models to reproduce and build on prior results. We demonstrate that the R-VAE reconstructs data as well as the VAE and $beta$-VAE, outperforms both methods in recovery of a target latent space in a sequential training setting.
  arXiv  Detail & Related papers  (2022-01-13T20:45:35Z)
• Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding [60.226644697970116]
  Domain classification is the fundamental task in natural language understanding (NLU) Most existing continual learning approaches suffer from low accuracy and performance fluctuation. We propose a hyper parameter-free continual learning model for text data that can stably produce high performance under various environments.
  arXiv  Detail & Related papers  (2022-01-05T02:46:16Z)
• Data Aggregation for Reducing Training Data in Symbolic Regression [0.0]
  This work discusses methods to reduce the training data and thereby also the runtime of genetic programming. K-means clustering and data binning is used for data aggregation and compared with random sampling as the simplest data reduction method. The performance of genetic programming is compared with random forests and linear regression.
  arXiv  Detail & Related papers  (2021-08-24T11:58:17Z)
• Class Balancing GAN with a Classifier in the Loop [58.29090045399214]
  We introduce a novel theoretically motivated Class Balancing regularizer for training GANs. Our regularizer makes use of the knowledge from a pre-trained classifier to ensure balanced learning of all the classes in the dataset. We demonstrate the utility of our regularizer in learning representations for long-tailed distributions via achieving better performance than existing approaches over multiple datasets.
  arXiv  Detail & Related papers  (2021-06-17T11:41:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.