CL-VISTA: Benchmarking Continual Learning in Video Large Language Models

• URL: http://arxiv.org/abs/2604.00677v1
• Date: Wed, 01 Apr 2026 09:30:47 GMT
• Title: CL-VISTA: Benchmarking Continual Learning in Video Large Language Models
• Abstract summary: Video Large Language Models (Video-LLMs) require continual learning to adapt to non-stationary real-world data.<n>Existing benchmarks fall short of evaluating modern foundation models.<n>We propose CL-VISTA, a benchmark tailored for continual video understanding of Video-LLMs.
• Score: 43.746369203878004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Large Language Models (Video-LLMs) require continual learning to adapt to non-stationary real-world data. However, existing benchmarks fall short of evaluating modern foundation models: many still rely on models without large-scale pre-training, and prevailing benchmarks typically partition a single dataset into sub-tasks, resulting in high task redundancy and negligible forgetting on pre-trained Video-LLMs. To address these limitations, we propose CL-VISTA, a benchmark tailored for continual video understanding of Video-LLMs. By curating 8 diverse tasks spanning perception, understanding, and reasoning, CL-VISTA induces substantial distribution shifts that effectively expose catastrophic forgetting. To systematically assess CL methods, we establish a comprehensive evaluation framework comprising 6 distinct protocols across 3 critical dimensions: performance, computational efficiency, and memory footprint. Notably, the performance dimension incorporates a general video understanding assessment to assess whether CL methods genuinely enhance foundational intelligence or merely induce task-specific overfitting. Extensive benchmarking of 10 mainstream CL methods reveals a fundamental trade-off: no single approach achieves universal superiority across all dimensions. Methods that successfully mitigate catastrophic forgetting tend to compromise generalization or incur prohibitive computational and memory overheads. We hope CL-VISTA provides critical insights for advancing continual learning in multimodal foundation models.

Related papers

• LibContinual: A Comprehensive Library towards Realistic Continual Learning [62.34449396069085]
  A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones.<n>We propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL.
  arXiv  Detail & Related papers  (2025-12-26T13:59:13Z)
• Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting [70.83781268763215]
  Vision-language models (VLMs) have achieved impressive performance across diverse multimodal tasks by leveraging large-scale pre-training.<n>VLMs face unique challenges such as cross-modal feature drift, parameter interference due to shared architectures, and zero-shot capability erosion.<n>This survey aims to serve as a comprehensive and diagnostic reference for researchers developing lifelong vision-language systems.
  arXiv  Detail & Related papers  (2025-08-06T09:03:10Z)
• LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially.<n>Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks.<n>However, such methods lack theoretical guarantees, making them prone to unexpected failures.<n>We aim to bridge this gap by designing a simple CL method that is theoretically sound and highly performant.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models [15.847302755988506]
  We address the Continual Learning problem, wherein a model must learn a sequence of tasks from non-stationary distributions. We propose LayUP, a new prototype-based approach to CL that leverages second-order feature statistics from multiple intermediate layers of a pre-trained network. Our results demonstrate that fully exhausting the representational capacities of pre-trained models in CL goes well beyond their final embeddings.
  arXiv  Detail & Related papers  (2023-12-13T13:11:44Z)
• Real-Time Evaluation in Online Continual Learning: A New Hope [104.53052316526546]
  We evaluate current Continual Learning (CL) methods with respect to their computational costs. A simple baseline outperforms state-of-the-art CL methods under this evaluation. This surprisingly suggests that the majority of existing CL literature is tailored to a specific class of streams that is not practical.
  arXiv  Detail & Related papers  (2023-02-02T12:21:10Z)
• vCLIMB: A Novel Video Class Incremental Learning Benchmark [53.90485760679411]
  We introduce vCLIMB, a novel video continual learning benchmark. vCLIMB is a standardized test-bed to analyze catastrophic forgetting of deep models in video continual learning. We propose a temporal consistency regularization that can be applied on top of memory-based continual learning methods.
  arXiv  Detail & Related papers  (2022-01-23T22:14:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.