UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding

• URL: http://arxiv.org/abs/2604.08522v1
• Date: Thu, 09 Apr 2026 17:57:09 GMT
• Title: UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding
• Abstract summary: Video temporal grounding (VTG) is typically tackled with dataset-specific models that transfer poorly across domains and query styles.<n>We present UniversalVTG, a single VTG model trained with large-scale cross-dataset pretraining.
• Score: 46.36397337493086
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video temporal grounding (VTG) is typically tackled with dataset-specific models that transfer poorly across domains and query styles. Recent efforts to overcome this limitation have adapted large multimodal language models (MLLMs) to VTG, but their high compute cost and limited video context still hinder long-video grounding. We instead scale unified supervision while keeping the model lightweight. We present UniversalVTG, a single VTG model trained with large-scale cross-dataset pretraining. An offline Query Unifier canonicalizes heterogeneous query formats into a shared declarative space, reducing linguistic mismatch and preventing the negative transfer observed under naïve joint training. Combined with an efficient grounding head, UniversalVTG scales to long, untrimmed videos. Across diverse benchmarks-GoalStep-StepGrounding, Ego4D-NLQ, TACoS, Charades-STA, and ActivityNet-Captions-one UniversalVTG checkpoint achieves state-of-the-art performance versus dedicated VTG models. Moreover, despite being $>100\times$ smaller than recent MLLM-based approaches, UniversalVTG matches or exceeds their accuracy on multiple benchmarks, offering a practical alternative to parameter-heavy MLLMs.

Related papers

• OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding [55.29748680163419]
  Video Temporal Grounding (VTG) struggles in open-world settings due to limited dataset scale and semantic diversity.<n>We introduce OmniVTG, a new large-scale dataset for open-world VTG.<n>We train the MLLM to first predict, then use its understanding capabilities to reflect on and refine its own predictions.
  arXiv  Detail & Related papers  (2026-04-28T06:34:19Z)
• SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding [5.9095311593289575]
  We propose SlotVTG, a framework that steers MLLMs toward object-centric, input-grounded visual reasoning at minimal cost.<n>Our approach significantly improves OOD robustness while maintaining competitive In-Domain (ID) performance with minimal overhead.
  arXiv  Detail & Related papers  (2026-03-26T17:59:31Z)
• TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs [81.78017865436816]
  We present TimeLens, a systematic investigation into building MLLMs with strong video temporal grounding ability.<n>We first expose critical quality issues in existing VTG benchmarks and introduce TimeLens-Bench, comprising meticulously re-annotated versions of three popular benchmarks.<n>We also address noisy training data through an automated re-annotation pipeline, yielding TimeLens-100K, a large-scale, high-quality training dataset.
  arXiv  Detail & Related papers  (2025-12-16T18:59:58Z)
• Aligning Effective Tokens with Video Anomaly in Large Language Models [42.99603812716817]
  We propose VA-GPT, a novel MLLM designed for summarizing and localizing abnormal events in various videos.<n>Our approach efficiently aligns effective tokens between visual encoders and LLMs through two key proposed modules.<n>We construct an instruction-following dataset specifically for fine-tuning video-anomaly-aware MLLMs.
  arXiv  Detail & Related papers  (2025-08-08T14:30:05Z)
• A Survey on Video Temporal Grounding with Multimodal Large Language Model [107.24431595873808]
  Recent advancement in temporal grounding (VTG) has significantly enhanced fine-grained video understanding.<n>With superior multimodal comprehension and reasoning abilities, VTG approaches based on MLLMs (VTG-MLLMs) are gradually surpassing traditional fine-tuned methods.<n>Despite extensive surveys on general video-language understanding, comprehensive reviews specifically addressing VTG-MLLMs remain scarce.
  arXiv  Detail & Related papers  (2025-08-07T08:52:11Z)
• MLVTG: Mamba-Based Feature Alignment and LLM-Driven Purification for Multi-Modal Video Temporal Grounding [13.025856914576673]
  Video Temporal Grounding aims to localize video clips corresponding to natural language queries.<n>Existing Transformer-based methods often suffer from redundant attention and suboptimal multi-modal alignment.<n>We propose MLVTG, a novel framework that integrates two key modules: MambaAligner and LLMRefiner.
  arXiv  Detail & Related papers  (2025-06-10T07:20:12Z)
• SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses [58.488812405557]
  Video grounding aims to localize specific natural language queries in an untrimmed video. We present a large-scale video grounding dataset named SynopGround. We introduce a more complex setting of video grounding dubbed Multi-Paragraph Video Grounding (MPVG)
  arXiv  Detail & Related papers  (2024-08-03T05:35:13Z)
• GiT: Towards Generalist Vision Transformer through Universal Language Interface [94.33443158125186]
  This paper proposes a simple, yet effective framework, called GiT, simultaneously applicable for various vision tasks only with a vanilla ViT. GiT is a multi-task visual model, jointly trained across five representative benchmarks without task-specific fine-tuning.
  arXiv  Detail & Related papers  (2024-03-14T13:47:41Z)
• MatFormer: Nested Transformer for Elastic Inference [91.45687988953435]
  MatFormer is a novel Transformer architecture designed to provide elastic inference across diverse deployment constraints.<n>MatFormer achieves this by incorporating a nested Feed Forward Network (FFN) block structure within a standard Transformer model.<n>We show that a 850M decoder-only MatFormer language model (MatLM) allows us to extract multiple smaller models spanning from 582M to 850M parameters.
  arXiv  Detail & Related papers  (2023-10-11T17:57:14Z)
• MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval [60.454321238910474]
  State-of-the-art video-text retrieval methods typically involve fully fine-tuning a pre-trained model on specific datasets. We present our pioneering work that enables parameter-efficient VTR using a pre-trained model. We propose a new method dubbed Multimodal Video Adapter (MV-Adapter) for efficiently transferring the knowledge in the pre-trained CLIP from image-text to video-text.
  arXiv  Detail & Related papers  (2023-01-19T03:42:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.