Non-Uniform Memory Sampling in Experience Replay

• URL: http://arxiv.org/abs/2502.11305v1
• Date: Sun, 16 Feb 2025 23:04:16 GMT
• Title: Non-Uniform Memory Sampling in Experience Replay
• Authors: Andrii Krutsylo,
• Abstract summary: A popular strategy to alleviate catastrophic forgetting is experience replay. Most approaches assume that sampling from this buffer is uniform by default. We generate 50 different non-uniform sampling probability weights for each trial and compare their final accuracy to the uniform sampling baseline.
• Score: 1.9580473532948401
• License:
• Abstract: Continual learning is the process of training machine learning models on a sequence of tasks where data distributions change over time. A well-known obstacle in this setting is catastrophic forgetting, a phenomenon in which a model drastically loses performance on previously learned tasks when learning new ones. A popular strategy to alleviate this problem is experience replay, in which a subset of old samples is stored in a memory buffer and replayed with new data. Despite continual learning advances focusing on which examples to store and how to incorporate them into the training loss, most approaches assume that sampling from this buffer is uniform by default. We challenge the assumption that uniform sampling is necessarily optimal. We conduct an experiment in which the memory buffer updates the same way in every trial, but the replay probability of each stored sample changes between trials based on different random weight distributions. Specifically, we generate 50 different non-uniform sampling probability weights for each trial and compare their final accuracy to the uniform sampling baseline. We find that there is always at least one distribution that significantly outperforms the baseline across multiple buffer sizes, models, and datasets. These results suggest that more principled adaptive replay policies could yield further gains. We discuss how exploiting this insight could inspire new research on non-uniform memory sampling in continual learning to better mitigate catastrophic forgetting. The code supporting this study is available at $\href{https://github.com/DentonJC/memory-sampling}{https://github.com/DentonJC/memory-sampling}$.

Related papers

• STAMP: Outlier-Aware Test-Time Adaptation with Stable Memory Replay [76.06127233986663]
  Test-time adaptation (TTA) aims to address the distribution shift between the training and test data with only unlabeled data at test time. This paper pays attention to the problem that conducts both sample recognition and outlier rejection during inference while outliers exist. We propose a new approach called STAble Memory rePlay (STAMP), which performs optimization over a stable memory bank instead of the risky mini-batch.
  arXiv  Detail & Related papers  (2024-07-22T16:25:41Z)
• Watch Your Step: Optimal Retrieval for Continual Learning at Scale [1.7265013728931]
  In continual learning, a model learns incrementally over time while minimizing interference between old and new tasks. One of the most widely used approaches in continual learning is referred to as replay. We propose a framework for evaluating selective retrieval strategies, categorized by simple, independent class- and sample-selective primitives. We propose a set of strategies to prevent duplicate replays and explore whether new samples with low loss values can be learned without replay.
  arXiv  Detail & Related papers  (2024-04-16T17:35:35Z)
• Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning [100.7407460674153]
  Deep learning systems are prone to catastrophic forgetting when learning from a sequence of tasks. To mitigate the problem, a line of methods propose to replay the data of experienced tasks when learning new tasks. However, it is not expected in practice considering the memory constraint or data privacy issue. As a replacement, data-free data replay methods are proposed by inverting samples from the classification model.
  arXiv  Detail & Related papers  (2024-01-12T12:51:12Z)
• Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learning [1.2891210250935146]
  Continual learning seeks to enable deep learners to train on a series of tasks of unknown length without suffering from the catastrophic forgetting of previous tasks. One effective solution is replay, which involves storing few previous experiences in memory and replaying them when learning the current task. This study aims to address these issues with a novel comparison of the commonly used reservoir sampling to various alternative population strategies.
  arXiv  Detail & Related papers  (2023-08-03T17:46:27Z)
• Reducing Training Sample Memorization in GANs by Training with Memorization Rejection [80.0916819303573]
  We propose rejection memorization, a training scheme that rejects generated samples that are near-duplicates of training samples during training. Our scheme is simple, generic and can be directly applied to any GAN architecture.
  arXiv  Detail & Related papers  (2022-10-21T20:17:50Z)
• Bias Mimicking: A Simple Sampling Approach for Bias Mitigation [57.17709477668213]
  We introduce a new class-conditioned sampling method: Bias Mimicking. Bias Mimicking improves underrepresented groups' accuracy of sampling methods by 3% over four benchmarks.
  arXiv  Detail & Related papers  (2022-09-30T17:33:00Z)
• Prioritizing Samples in Reinforcement Learning with Reducible Loss [5.901819658403315]
  We propose a method to prioritize samples based on how much we can learn from a sample. We develop an algorithm to prioritize samples with high learn-ability, while assigning lower priority to those that are hard-to-learn.
  arXiv  Detail & Related papers  (2022-08-22T17:55:43Z)
• Memory Population in Continual Learning via Outlier Elimination [25.511380924335207]
  Catastrophic forgetting, the phenomenon of forgetting previously learned tasks when learning a new one, is a major hurdle in developing continual learning algorithms. A popular method to alleviate forgetting is to use a memory buffer, which stores a subset of previously learned task examples for use during training on new tasks. This paper introduces Memory Outlier Elimination (MOE), a method for identifying and eliminating outliers in the memory buffer by choosing samples from label-homogeneous subpopulations.
  arXiv  Detail & Related papers  (2022-07-04T00:09:33Z)
• Memory Replay with Data Compression for Continual Learning [80.95444077825852]
  We propose memory replay with data compression to reduce the storage cost of old training samples. We extensively validate this across several benchmarks of class-incremental learning and in a realistic scenario of object detection for autonomous driving.
  arXiv  Detail & Related papers  (2022-02-14T10:26:23Z)
• Learning a Unified Sample Weighting Network for Object Detection [113.98404690619982]
  Region sampling or weighting is significantly important to the success of modern region-based object detectors. We argue that sample weighting should be data-dependent and task-dependent. We propose a unified sample weighting network to predict a sample's task weights.
  arXiv  Detail & Related papers  (2020-06-11T16:19:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.