Bridging Streaming Continual Learning via In-Context Large Tabular Models

• URL: http://arxiv.org/abs/2512.11668v1
• Date: Fri, 12 Dec 2025 15:47:26 GMT
• Title: Bridging Streaming Continual Learning via In-Context Large Tabular Models
• Authors: Afonso Lourenço, João Gama, Eric P. Xing, Goreti Marreiros,
• Abstract summary: We argue that large in-context models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL)<n>In our view, streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs.<n>We show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity and stability.
• Score: 37.26465083968656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In streaming scenarios, models must learn continuously, adapting to concept drifts without erasing previously acquired knowledge. However, existing research communities address these challenges in isolation. Continual Learning (CL) focuses on long-term retention and mitigating catastrophic forgetting, often without strict real-time constraints. Stream Learning (SL) emphasizes rapid, efficient adaptation to high-frequency data streams, but typically neglects forgetting. Recent efforts have tried to combine these paradigms, yet no clear algorithmic overlap exists. We argue that large in-context tabular models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL). In our view, unbounded streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs. This recovers the classical SL motivation of compressing massive streams with fixed-size guarantees, while simultaneously aligning with the experience-replay desiderata of CL. To clarify this bridge, we show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity (performing well on the current distribution) and stability (retaining past knowledge), while also imposing a minimal complexity constraint that motivates diversification (avoiding redundancy in what is stored) and retrieval (re-prioritizing past information when needed). Within this perspective, we propose structuring SCL with LTMs around two core principles of data selection for in-context learning: (1) distribution matching, which balances plasticity and stability, and (2) distribution compression, which controls memory size through diversification and retrieval mechanisms.

Related papers

• Streaming Continual Learning for Unified Adaptive Intelligence in Dynamic Environments [5.713812353511933]
  Continual Learning (CL) and Streaming Machine Learning () are two research areas that tackle this arduous task.<n>We put forward a unified setting that harnesses the benefits of both CL and Streaming Continual Learning.
  arXiv  Detail & Related papers  (2026-03-02T10:24:37Z)
• Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning [62.680551162054975]
  We introduce an end-to-end framework where LLMs learn to self-regulate the granularity of the reasoning steps through dynamic summarization.<n>We apply reinforcement learning to incentivize this capability further, uncovering a critical insight: the accuracy gap between the highly efficient Fold mode and the exhaustive Unfold mode progressively narrows.<n>Our Accordion-Thinker demonstrates that with learned self-compression, LLMs can tackle complex reasoning tasks with minimal dependency token overhead.
  arXiv  Detail & Related papers  (2026-02-03T08:34:20Z)
• LibContinual: A Comprehensive Library towards Realistic Continual Learning [62.34449396069085]
  A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones.<n>We propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL.
  arXiv  Detail & Related papers  (2025-12-26T13:59:13Z)
• In-Context Learning can Perform Continual Learning Like Humans [12.499724976235534]
  Large language models (LLMs) can adapt to new tasks via in-context learning (ICL) without parameter updates.<n>We investigate the retention characteristics of ICL in multitask settings and extend it to in-context continual learning (ICCL)<n>ICCL benefits from distributed practice in a manner analogous to humans, consistently revealing a spacing "sweet spot" for retention.
  arXiv  Detail & Related papers  (2025-09-26T15:08:06Z)
• Adapt before Continual Learning [9.477667054965782]
  Adapting PTMs before the core CL process (ACL) is a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task.<n>ACL significantly improves CL performance across benchmarks and integrated methods.
  arXiv  Detail & Related papers  (2025-06-04T13:46:33Z)
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning [79.44594332189018]
  Class-Incremental Learning (CIL) seeks to develop models that continuously learn new action categories over time without previously acquired knowledge.<n>Existing approaches either rely on forgetting, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling.<n>We propose a unified and exemplar-free VCIL framework that explicitly disentangles and preserves information.
  arXiv  Detail & Related papers  (2025-05-20T06:46:51Z)
• StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation [55.75008325187133]
  Reinforcement learning (RL) has become the core post-training technique for large language models (LLMs)<n>StreamRL is designed with disaggregation from first principles to address two types of performance bottlenecks.<n> Experiments show that StreamRL improves throughput by up to 2.66x compared to existing state-of-the-art systems.
  arXiv  Detail & Related papers  (2025-04-22T14:19:06Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially.<n>Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks.<n>However, such methods lack theoretical guarantees, making them prone to unexpected failures.<n>We aim to bridge this gap by designing a simple CL method that is theoretically sound and highly performant.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning [8.191971407001034]
  We propose a novel Prior-free Balanced Replay (PBR) framework to learn from long-tailed data stream with less forgetting. We incorporate two prior-free components to further reduce the forgetting issue. Our approach is evaluated on three standard long-tailed benchmarks.
  arXiv  Detail & Related papers  (2024-08-27T11:38:01Z)
• Learning an evolved mixture model for task-free continual learning [11.540150938141034]
  We address the Task-Free Continual Learning (TFCL) in which a model is trained on non-stationary data streams with no explicit task information. We introduce two simple dropout mechanisms to selectively remove stored examples in order to avoid memory overload.
  arXiv  Detail & Related papers  (2022-07-11T16:01:27Z)
• Generalized Variational Continual Learning [33.194866396158005]
  Two main approaches to continuous learning are Online Elastic Weight Consolidation and Variational Continual Learning. We show that applying this modification to mitigate Online EWC as a limiting case, allowing baselines between the two approaches. In order to the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, mitigate neural networks with task-specific FiLM layers.
  arXiv  Detail & Related papers  (2020-11-24T19:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.