Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence

• URL: http://arxiv.org/abs/2603.08600v1
• Date: Mon, 09 Mar 2026 16:49:04 GMT
• Title: Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence
• Abstract summary: We introduce MAGIC Net, a novel Streaming Continual Learning (SCL) approach that integrates architectural strategies with recurrent neural networks to tame temporal dependence.<n> MAGIC Net continuously learns, looks back at past knowledge by applying learnable masks over frozen weights, and expands its architecture when necessary.<n>Experiments on synthetic and real-world streams show that it improves adaptation to new concepts, limits memory usage, and mitigates forgetting.
• Score: 1.2191215203827384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Concept drift, temporal dependence, and catastrophic forgetting represent major challenges when learning from data streams. While Streaming Machine Learning and Continual Learning (CL) address these issues separately, recent efforts in Streaming Continual Learning (SCL) aim to unify them. In this work, we introduce MAGIC Net, a novel SCL approach that integrates CL-inspired architectural strategies with recurrent neural networks to tame temporal dependence. MAGIC Net continuously learns, looks back at past knowledge by applying learnable masks over frozen weights, and expands its architecture when necessary. It performs all operations online, ensuring inference availability at all times. Experiments on synthetic and real-world streams show that it improves adaptation to new concepts, limits memory usage, and mitigates forgetting.

Related papers

• A Practical Guide to Streaming Continual Learning [53.995807801604506]
  Continual Learning (CL) and Streaming Machine Learning () study the ability of agents to learn from a stream of non-stationary data.<n>Despite sharing some similarities, they address different and complementary challenges.<n>We discuss Streaming Continual Learning (SCL), an emerging paradigm providing a unifying solution to real-world problems.
  arXiv  Detail & Related papers  (2026-03-02T10:06:34Z)
• Bridging Streaming Continual Learning via In-Context Large Tabular Models [37.26465083968656]
  We argue that large in-context models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL)<n>In our view, streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs.<n>We show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity and stability.
  arXiv  Detail & Related papers  (2025-12-12T15:47:26Z)
• Towards Better De-raining Generalization via Rainy Characteristics Memorization and Replay [74.54047495424618]
  Current image de-raining methods primarily learn from a limited dataset.<n>We introduce a new framework that enables networks to progressively expand their de-raining knowledge base.
  arXiv  Detail & Related papers  (2025-06-03T05:50:00Z)
• Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps [56.827895559823126]
  Self-organizing map (SOM) is a neural model often used in clustering and dimensionality reduction. We propose a generalization of the SOM, the continual SOM, which is capable of online unsupervised learning under a low memory budget. Our results, on benchmarks including MNIST, Kuzushiji-MNIST, and Fashion-MNIST, show almost a two times increase in accuracy.
  arXiv  Detail & Related papers  (2024-02-19T19:11:22Z)
• Metalearning Continual Learning Algorithms [42.710124929514066]
  We propose Automated Continual Learning (ACL) to train self-referential neural networks to continual (meta)learning algorithms.<n>ACL encodes continual learning (CL) desiderata -- good performance on both old and new tasks -- into its metalearning objectives.<n>Our experiments demonstrate that ACL effectively resolves "in-context catastrophic forgetting," a problem that naive in-context learning algorithms suffer from.
  arXiv  Detail & Related papers  (2023-12-01T01:25:04Z)
• Learning Fast and Slow for Online Time Series Forecasting [76.50127663309604]
  Fast and Slow learning Networks (FSNet) is a holistic framework for online time-series forecasting. FSNet balances fast adaptation to recent changes and retrieving similar old knowledge. Our code will be made publicly available.
  arXiv  Detail & Related papers  (2022-02-23T18:23:07Z)
• Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning [54.7584721943286]
  Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. We propose to only activate and select sparse neurons for learning current and past tasks at any stage.
  arXiv  Detail & Related papers  (2022-02-21T13:25:03Z)
• Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System [13.041607703862724]
  We propose CLS-ER, a novel dual memory experience replay (ER) method. New knowledge is acquired while aligning the decision boundaries with the semantic memories. Our approach achieves state-of-the-art performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-01-29T15:15:23Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• Online Continual Learning on Sequences [9.603184477806954]
  Online continual learning (OCL) refers to the ability of a system to learn over time from a continuous stream of data without having to revisit previously encountered training samples. Machine learning models that address OCL must alleviate textitcatastrophic forgetting in which hidden representations are disrupted or completely overwritten when learning from streams of novel input.
  arXiv  Detail & Related papers  (2020-03-20T05:49:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.