ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits

• URL: http://arxiv.org/abs/2609.00505v1
• Date: Tue, 01 Sep 2026 00:13:44 GMT
• Title: ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits
• Abstract summary: Video-text models frequently exhibit temporal blindness, the inability to perceive fundamental cues like order, direction, and motion dynamics.<n>We propose Cross-Modal Temporal Edits (XTE), a self-supervised framework that injects precise temporal supervision.<n>We instantiate this with ViTAL-X, a lightweight model that equips frozen image-text backbones with temporal awareness.
• Score: 50.455758151353926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video-text models adapted from image-text architectures (e.g., CLIP) frequently exhibit temporal blindness, the inability to perceive fundamental cues like order, direction, and motion dynamics. Standard datasets mask this limitation by enabling models to exploit static spatial shortcuts. To systematically evaluate this, we introduce XTE-Bench, a diagnostic probe revealing that even large-scale video-language models struggle with basic temporal reasoning, indicating that parameter scaling alone is insufficient to resolve this flaw. To address this, we propose Cross-Modal Temporal Edits (XTE), a self-supervised framework that injects precise temporal supervision. By performing synchronized video-text transformations, XTE generates hard temporal negatives without manual annotation. We instantiate this with ViTAL-X, a lightweight model that equips frozen image-text backbones with temporal awareness while preserving their foundational spatial knowledge. Across six temporal benchmarks, ViTAL-X achieves state-of-the-art performance. Utilizing only 0.4B parameters and 1M training clips, ViTAL-X outperforms 7B-parameter models and surpasses baselines trained on 600x more data. These results demonstrate that targeted, high-quality temporal alignment provides a highly efficient alternative to pure scaling.

Related papers

• TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval [1.4874449172133888]
  Text-Video Retrieval (TVR) retrieves videos that match a natural-language query.<n>TVR is limited by the lack of temporal modeling.<n>We propose Temporal-Aware Mixture-of-Experts for Text-Video Retrieval (TAME)
  arXiv  Detail & Related papers  (2026-09-02T07:12:50Z)
• TV-RAG: A Temporal-aware and Semantic Entropy-Weighted Framework for Long Video Retrieval and Understanding [14.570869250170139]
  TV-RAG is a training-free architecture that couples temporal alignment with entropy-guided semantics to improve long-video reasoning.<n>By weaving these temporal and semantic signals together, TV-RAG realises a dual-level reasoning routine that can be grafted onto any LVLM without re-training or fine-tuning.
  arXiv  Detail & Related papers  (2025-12-29T14:10:22Z)
• Harnessing Synthetic Preference Data for Enhancing Temporal Understanding of Video-LLMs [54.502280390499756]
  We propose TimeWarp to create a targeted synthetic temporal dataset to fine-tune the model's responses to encourage it to focus on the given input video.<n>We demonstrate that when our method is applied to existing models, it significantly improves performance on temporal understanding benchmarks.
  arXiv  Detail & Related papers  (2025-10-04T21:48:40Z)
• DyST-XL: Dynamic Layout Planning and Content Control for Compositional Text-to-Video Generation [14.34140569565309]
  DyST-XL is a framework that enhances off-the-shelf text-to-video models through frame-aware control.<n>The code is released in https://github.com/XiaoBuL/DyST-XL.
  arXiv  Detail & Related papers  (2025-04-21T11:41:22Z)
• Temporal Reasoning Transfer from Text to Video [51.68487044397409]
  Video Large Language Models (Video LLMs) struggle with tracking temporal changes and reasoning about temporal relationships. We introduce the Textual Temporal reasoning Transfer (T3) to transfer temporal reasoning abilities from text to video domains. LongVA-7B model achieves competitive performance on comprehensive video benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T16:10:29Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• Towards Consistent Video Editing with Text-to-Image Diffusion Models [10.340371518799444]
  Existing works have advanced Text-to-Image (TTI) diffusion models for video editing in a one-shot learning manner. These methods might produce results of unsatisfied consistency with text prompt as well as temporal sequence. We propose a novel EI$2$ model towards textbfEnhancing vtextbfIdeo textbfEditing constextbfIstency of TTI-based frameworks.
  arXiv  Detail & Related papers  (2023-05-27T10:03:36Z)
• Implicit Temporal Modeling with Learnable Alignment for Video Recognition [95.82093301212964]
  We propose a novel Implicit Learnable Alignment (ILA) method, which minimizes the temporal modeling effort while achieving incredibly high performance. ILA achieves a top-1 accuracy of 88.7% on Kinetics-400 with much fewer FLOPs compared with Swin-L and ViViT-H.
  arXiv  Detail & Related papers  (2023-04-20T17:11:01Z)
• Distortion-Aware Network Pruning and Feature Reuse for Real-time Video Segmentation [49.17930380106643]
  We propose a novel framework to speed up any architecture with skip-connections for real-time vision tasks. Specifically, at the arrival of each frame, we transform the features from the previous frame to reuse them at specific spatial bins. We then perform partial computation of the backbone network on the regions of the current frame that captures temporal differences between the current and previous frame.
  arXiv  Detail & Related papers  (2022-06-20T07:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.