Online Continual Learning on Sequences

• URL: http://arxiv.org/abs/2003.09114v1
• Date: Fri, 20 Mar 2020 05:49:31 GMT
• Title: Online Continual Learning on Sequences
• Abstract summary: Online continual learning (OCL) refers to the ability of a system to learn over time from a continuous stream of data without having to revisit previously encountered training samples. Machine learning models that address OCL must alleviate textitcatastrophic forgetting in which hidden representations are disrupted or completely overwritten when learning from streams of novel input.
• Score: 9.603184477806954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online continual learning (OCL) refers to the ability of a system to learn over time from a continuous stream of data without having to revisit previously encountered training samples. Learning continually in a single data pass is crucial for agents and robots operating in changing environments and required to acquire, fine-tune, and transfer increasingly complex representations from non-i.i.d. input distributions. Machine learning models that address OCL must alleviate \textit{catastrophic forgetting} in which hidden representations are disrupted or completely overwritten when learning from streams of novel input. In this chapter, we summarize and discuss recent deep learning models that address OCL on sequential input through the use (and combination) of synaptic regularization, structural plasticity, and experience replay. Different implementations of replay have been proposed that alleviate catastrophic forgetting in connectionists architectures via the re-occurrence of (latent representations of) input sequences and that functionally resemble mechanisms of hippocampal replay in the mammalian brain. Empirical evidence shows that architectures endowed with experience replay typically outperform architectures without in (online) incremental learning tasks.

Related papers

• Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures [50.668639270656115]
  ReSYNC is a robot that learns skills and concepts and refines its abstraction library to support abstract planning.<n>We show that ReSYNC's ability to continually expand and refine its abstraction library allows it to solve long-horizon, previously unseen problems.<n>Overall, ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure-aware planning.
  arXiv  Detail & Related papers  (2026-06-16T17:59:57Z)
• Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index [51.56484100374058]
  We propose an online, topology-aware monitor for evolving neural representations that couples Modular Morse Homology Maintenance (MMHM) with a composite Collapse Index (CI)<n>Instead of rebuilding complexes each epoch, we apply sparse edits at a fixed scale and maintain a discrete Morse matching, yielding fast, incremental updates.
  arXiv  Detail & Related papers  (2026-04-28T19:21:21Z)
• Abstraction as a Memory-Efficient Inductive Bias for Continual Learning [2.8602273445853967]
  Abstraction-Augmented Training (AAT) is a loss-level modification encouraging models to capture the latent relational structure shared across examples.<n>AAT stabilizes learning in strictly online data streams, eliminating the need for a replay buffer.<n>Our results show that AAT achieves performance comparable to or exceeding strong experience replay (ER) baselines.
  arXiv  Detail & Related papers  (2026-03-17T22:59:13Z)
• Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence [1.2191215203827384]
  We introduce MAGIC Net, a novel Streaming Continual Learning (SCL) approach that integrates architectural strategies with recurrent neural networks to tame temporal dependence.<n> MAGIC Net continuously learns, looks back at past knowledge by applying learnable masks over frozen weights, and expands its architecture when necessary.<n>Experiments on synthetic and real-world streams show that it improves adaptation to new concepts, limits memory usage, and mitigates forgetting.
  arXiv  Detail & Related papers  (2026-03-09T16:49:04Z)
• From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning [59.88543114325153]
  We introduce the Seeing-to-Experiencing framework to scale the capability of navigation foundation models with reinforcement learning.<n>S2E combines the strengths of pre-training on videos and post-training through RL.<n>We establish a comprehensive end-to-end evaluation benchmark, NavBench-GS, built on photorealistic 3DGS reconstructions of real-world scenes.
  arXiv  Detail & Related papers  (2025-07-29T17:26:10Z)
• Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning [13.836798036474143]
  Key challenge in Federated Class Continual Learning is catastrophic forgetting. We propose a novel method of data replay based on diffusion models. Our method significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2024-09-02T10:07:24Z)
• Learning Object-Centric Representation via Reverse Hierarchy Guidance [73.05170419085796]
  Object-Centric Learning (OCL) seeks to enable Neural Networks to identify individual objects in visual scenes. RHGNet introduces a top-down pathway that works in different ways in the training and inference processes. Our model achieves SOTA performance on several commonly used datasets.
  arXiv  Detail & Related papers  (2024-05-17T07:48:27Z)
• Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training [24.719121340143978]
  We explore the training dynamics of neural networks in a structured non-IID setting where documents are presented cyclically in a fixed, repeated sequence. We find that over-parametrized neural networks can recover from catastrophic interference.
  arXiv  Detail & Related papers  (2024-03-14T17:51:54Z)
• Self-Regulated Neurogenesis for Online Data-Incremental Learning [9.254419196812233]
  SERENA encodes each concept in a specialized network path called 'concept cell'<n>Once a concept is learned, its corresponding concept cell is frozen, effectively preventing the forgetting of previously acquired information.<n> Experimental results show that our method not only establishes new state-of-the-art results across ten benchmarks but also remarkably surpasses offline supervised batch learning performance.
  arXiv  Detail & Related papers  (2024-03-13T13:51:12Z)
• REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation [61.7171775202833]
  We introduce an efficient system for learning dexterous manipulation skills withReinforcement learning. The main idea of our approach is the integration of recent advances in sample-efficient RL and replay buffer bootstrapping. Our system completes the real-world training cycle by incorporating learned resets via an imitation-based pickup policy.
  arXiv  Detail & Related papers  (2023-09-06T19:05:31Z)
• OER: Offline Experience Replay for Continual Offline Reinforcement Learning [25.985985377992034]
  Continuously learning new skills via a sequence of pre-collected offline datasets is desired for an agent. In this paper, we formulate a new setting, continual offline reinforcement learning (CORL), where an agent learns a sequence of offline reinforcement learning tasks. We propose a new model-based experience selection scheme to build the replay buffer, where a transition model is learned to approximate the state distribution.
  arXiv  Detail & Related papers  (2023-05-23T08:16:44Z)
• Learning an evolved mixture model for task-free continual learning [11.540150938141034]
  We address the Task-Free Continual Learning (TFCL) in which a model is trained on non-stationary data streams with no explicit task information. We introduce two simple dropout mechanisms to selectively remove stored examples in order to avoid memory overload.
  arXiv  Detail & Related papers  (2022-07-11T16:01:27Z)
• Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning [113.58691755215663]
  We develop RetroPrompt to help a model strike a balance between generalization and memorization. In contrast with vanilla prompt learning, RetroPrompt constructs an open-book knowledge-store from training instances. Extensive experiments demonstrate that RetroPrompt can obtain better performance in both few-shot and zero-shot settings.
  arXiv  Detail & Related papers  (2022-05-29T16:07:30Z)
• Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay [67.50637511633212]
  A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day.
  arXiv  Detail & Related papers  (2021-12-09T07:11:14Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams [42.525141660788]
  We introduce a system to enable learning and prediction at any point in time. In contrast to the major body of work in continual learning, data streams are processed in an online fashion. We obtain state-of-the-art performance by a significant margin on eight benchmarks, including three highly imbalanced data streams.
  arXiv  Detail & Related papers  (2020-09-02T09:39:26Z)
• Incremental Training of a Recurrent Neural Network Exploiting a Multi-Scale Dynamic Memory [79.42778415729475]
  We propose a novel incrementally trained recurrent architecture targeting explicitly multi-scale learning. We show how to extend the architecture of a simple RNN by separating its hidden state into different modules. We discuss a training algorithm where new modules are iteratively added to the model to learn progressively longer dependencies.
  arXiv  Detail & Related papers  (2020-06-29T08:35:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.