Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment

• URL: http://arxiv.org/abs/2507.02310v1
• Date: Thu, 03 Jul 2025 04:41:20 GMT
• Title: Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment
• Authors: Alif Ashrafee, Jedrzej Kozal, Michal Wozniak, Bartosz Krawczyk,
• Abstract summary: We introduce a holistic framework for continual learning under concept drift.<n>We propose Adaptive Memory Realignment (AMR), a lightweight alternative that equips rehearsal-based learners with a drift-aware adaptation mechanism.<n>AMR consistently counters concept drift, maintaining high accuracy with minimal overhead.
• Score: 6.0897744845912865
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traditional continual learning methods prioritize knowledge retention and focus primarily on mitigating catastrophic forgetting, implicitly assuming that the data distribution of previously learned tasks remains static. This overlooks the dynamic nature of real-world data streams, where concept drift permanently alters previously seen data and demands both stability and rapid adaptation. We introduce a holistic framework for continual learning under concept drift that simulates realistic scenarios by evolving task distributions. As a baseline, we consider Full Relearning (FR), in which the model is retrained from scratch on newly labeled samples from the drifted distribution. While effective, this approach incurs substantial annotation and computational overhead. To address these limitations, we propose Adaptive Memory Realignment (AMR), a lightweight alternative that equips rehearsal-based learners with a drift-aware adaptation mechanism. AMR selectively removes outdated samples of drifted classes from the replay buffer and repopulates it with a small number of up-to-date instances, effectively realigning memory with the new distribution. This targeted resampling matches the performance of FR while reducing the need for labeled data and computation by orders of magnitude. To enable reproducible evaluation, we introduce four concept-drift variants of standard vision benchmarks: Fashion-MNIST-CD, CIFAR10-CD, CIFAR100-CD, and Tiny-ImageNet-CD, where previously seen classes reappear with shifted representations. Comprehensive experiments on these datasets using several rehearsal-based baselines show that AMR consistently counters concept drift, maintaining high accuracy with minimal overhead. These results position AMR as a scalable solution that reconciles stability and plasticity in non-stationary continual learning environments.

Related papers

• Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
  arXiv  Detail & Related papers  (2025-05-19T17:21:03Z)
• PreAdaptFWI: Pretrained-Based Adaptive Residual Learning for Full-Waveform Inversion Without Dataset Dependency [8.719356558714246]
  Full-waveform inversion (FWI) is a method that utilizes seismic data to invert the physical parameters of subsurface media.<n>Due to its ill-posed nature, FWI is susceptible to getting trapped in local minima.<n>Various research efforts have attempted to combine neural networks with FWI to stabilize the inversion process.
  arXiv  Detail & Related papers  (2025-02-17T15:30:17Z)
• Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction [48.30341580103962]
  We propose a novel re-visible dual-domain self-supervised deep unfolding network to address these issues.<n>We design a deep unfolding network based on Chambolle and Pock Proximal Point Algorithm (DUN-CP-PPA) to achieve end-to-end reconstruction.<n> Experiments conducted on the fastMRI and IXI datasets demonstrate that our method significantly outperforms state-of-the-art approaches in terms of reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-07T12:29:32Z)
• Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset [98.52916361979503]
  We introduce a novel learning approach that automatically models and adapts to non-stationarity. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.
  arXiv  Detail & Related papers  (2024-11-06T16:32:40Z)
• Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond [13.867793835583463]
  We propose an uncertainty-aware memory-based approach to solve catastrophic forgetting. We retrieve samples with specific characteristics, and - by retraining the model on such samples - we demonstrate the potential of this approach.
  arXiv  Detail & Related papers  (2024-05-29T09:29:39Z)
• Adaptive Cross Batch Normalization for Metric Learning [75.91093210956116]
  Metric learning is a fundamental problem in computer vision. We show that it is equally important to ensure that the accumulated embeddings are up to date. In particular, it is necessary to circumvent the representational drift between the accumulated embeddings and the feature embeddings at the current training iteration.
  arXiv  Detail & Related papers  (2023-03-30T03:22:52Z)
• A theoretical framework for self-supervised MR image reconstruction using sub-sampling via variable density Noisier2Noise [0.0]
  We use the Noisier2Noise framework to analytically explain the performance of Self-samplingd Learning via Data UnderSupervise. We propose partitioning the sampling set so that the subsets have the same type of distribution as the original sampling mask.
  arXiv  Detail & Related papers  (2022-05-20T16:19:23Z)
• Employing chunk size adaptation to overcome concept drift [2.277447144331876]
  We propose a new Chunk Adaptive Restoration framework that can be adapted to any block-based data stream classification algorithm. The proposed algorithm adjusts the data chunk size in the case of concept drift detection to minimize the impact of the change on the predictive performance of the used model.
  arXiv  Detail & Related papers  (2021-10-25T12:36:22Z)
• Self-Damaging Contrastive Learning [92.34124578823977]
  Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution. This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes. Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
  arXiv  Detail & Related papers  (2021-06-06T00:04:49Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.