Bridging Streaming Continual Learning via In-Context Large Tabular Models

• URL: http://arxiv.org/abs/2512.11668v1
• Date: Fri, 12 Dec 2025 15:47:26 GMT
• Title: Bridging Streaming Continual Learning via In-Context Large Tabular Models
• Abstract summary: We argue that large in-context models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL)<n>In our view, streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs.<n>We show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity and stability.
• Score: 37.26465083968656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In streaming scenarios, models must learn continuously, adapting to concept drifts without erasing previously acquired knowledge. However, existing research communities address these challenges in isolation. Continual Learning (CL) focuses on long-term retention and mitigating catastrophic forgetting, often without strict real-time constraints. Stream Learning (SL) emphasizes rapid, efficient adaptation to high-frequency data streams, but typically neglects forgetting. Recent efforts have tried to combine these paradigms, yet no clear algorithmic overlap exists. We argue that large in-context tabular models (LTMs) provide a natural bridge for Streaming Continual Learning (SCL). In our view, unbounded streams should be summarized on-the-fly into compact sketches that can be consumed by LTMs. This recovers the classical SL motivation of compressing massive streams with fixed-size guarantees, while simultaneously aligning with the experience-replay desiderata of CL. To clarify this bridge, we show how the SL and CL communities implicitly adopt a divide-to-conquer strategy to manage the tension between plasticity (performing well on the current distribution) and stability (retaining past knowledge), while also imposing a minimal complexity constraint that motivates diversification (avoiding redundancy in what is stored) and retrieval (re-prioritizing past information when needed). Within this perspective, we propose structuring SCL with LTMs around two core principles of data selection for in-context learning: (1) distribution matching, which balances plasticity and stability, and (2) distribution compression, which controls memory size through diversification and retrieval mechanisms.

Related papers

• The Forgetting-Retention Dilemma: Certified Unlearning Theory in Continual Learning [23.209639058960477]
  Machine unlearning aims to eliminate influence of specific data from models to safeguard privacy.<n>Current certified unlearning algorithms fail to account for cumulative model evolution inherent to continuous learning.<n>We formulate CL's unlearning objective as the minimization of post-unlearning excess risk, which decomposes into CL excess risk and unlearning loss.
  arXiv  Detail & Related papers  (2026-06-29T06:18:51Z)
• Theoretical Foundations of Continual Learning via Drift-Plus-Penalty [6.614755043607776]
  Continual learning (CL) addresses this challenge by incorporating new tasks while mitigating catastrophic forgetting.<n>We introduce a control-theoretic perspective on CL that explicitly regulates the evolution of forgetting.<n>We propose COntinual Learning with Drift-PlusPenalty (COLD), a continual framework based on the Drift-PlusPenalty principle from optimization.
  arXiv  Detail & Related papers  (2026-06-07T04:51:32Z)
• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Reinforcing Structured Chain-of-Thought for Video Understanding [49.3086326382764]
  We introduce Summary-Driven Reinforcement Learning (SDRL)<n>SDRL is a single-stage RL framework that obviates the need forSupervised Fine-Tuning (SFT)<n>Our method achieves state-of-the-art performance on seven public VideoQA datasets.
  arXiv  Detail & Related papers  (2026-03-26T22:11:14Z)
• Streaming Continual Learning for Unified Adaptive Intelligence in Dynamic Environments [5.713812353511933]
  Continual Learning (CL) and Streaming Machine Learning () are two research areas that tackle this arduous task.<n>We put forward a unified setting that harnesses the benefits of both CL and Streaming Continual Learning.
  arXiv  Detail & Related papers  (2026-03-02T10:24:37Z)
• Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning [62.680551162054975]
  We introduce an end-to-end framework where LLMs learn to self-regulate the granularity of the reasoning steps through dynamic summarization.<n>We apply reinforcement learning to incentivize this capability further, uncovering a critical insight: the accuracy gap between the highly efficient Fold mode and the exhaustive Unfold mode progressively narrows.<n>Our Accordion-Thinker demonstrates that with learned self-compression, LLMs can tackle complex reasoning tasks with minimal dependency token overhead.
  arXiv  Detail & Related papers  (2026-02-03T08:34:20Z)
• LibContinual: A Comprehensive Library towards Realistic Continual Learning [62.34449396069085]
  A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones.<n>We propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL.
  arXiv  Detail & Related papers  (2025-12-26T13:59:13Z)
• In-Context Learning can Perform Continual Learning Like Humans [12.499724976235534]
  Large language models (LLMs) can adapt to new tasks via in-context learning (ICL) without parameter updates.<n>We investigate the retention characteristics of ICL in multitask settings and extend it to in-context continual learning (ICCL)<n>ICCL benefits from distributed practice in a manner analogous to humans, consistently revealing a spacing "sweet spot" for retention.
  arXiv  Detail & Related papers  (2025-09-26T15:08:06Z)
• Adapt before Continual Learning [9.477667054965782]
  Adapting PTMs before the core CL process (ACL) is a novel framework that introduces a plug-and-play adaptation phase prior to learning each new task.<n>ACL significantly improves CL performance across benchmarks and integrated methods.
  arXiv  Detail & Related papers  (2025-06-04T13:46:33Z)
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning [79.44594332189018]
  Class-Incremental Learning (CIL) seeks to develop models that continuously learn new action categories over time without previously acquired knowledge.<n>Existing approaches either rely on forgetting, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling.<n>We propose a unified and exemplar-free VCIL framework that explicitly disentangles and preserves information.
  arXiv  Detail & Related papers  (2025-05-20T06:46:51Z)
• StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation [55.75008325187133]
  Reinforcement learning (RL) has become the core post-training technique for large language models (LLMs)<n>StreamRL is designed with disaggregation from first principles to address two types of performance bottlenecks.<n> Experiments show that StreamRL improves throughput by up to 2.66x compared to existing state-of-the-art systems.
  arXiv  Detail & Related papers  (2025-04-22T14:19:06Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially.<n>Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks.<n>However, such methods lack theoretical guarantees, making them prone to unexpected failures.<n>We aim to bridge this gap by designing a simple CL method that is theoretically sound and highly performant.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning [8.191971407001034]
  We propose a novel Prior-free Balanced Replay (PBR) framework to learn from long-tailed data stream with less forgetting. We incorporate two prior-free components to further reduce the forgetting issue. Our approach is evaluated on three standard long-tailed benchmarks.
  arXiv  Detail & Related papers  (2024-08-27T11:38:01Z)
• Learning an evolved mixture model for task-free continual learning [11.540150938141034]
  We address the Task-Free Continual Learning (TFCL) in which a model is trained on non-stationary data streams with no explicit task information. We introduce two simple dropout mechanisms to selectively remove stored examples in order to avoid memory overload.
  arXiv  Detail & Related papers  (2022-07-11T16:01:27Z)
• Generalized Variational Continual Learning [33.194866396158005]
  Two main approaches to continuous learning are Online Elastic Weight Consolidation and Variational Continual Learning. We show that applying this modification to mitigate Online EWC as a limiting case, allowing baselines between the two approaches. In order to the observed overpruning effect of VI, we take inspiration from a common multi-task architecture, mitigate neural networks with task-specific FiLM layers.
  arXiv  Detail & Related papers  (2020-11-24T19:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.