From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA

• URL: http://arxiv.org/abs/2606.30220v1
• Date: Mon, 29 Jun 2026 12:33:15 GMT
• Title: From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA
• Abstract summary: High benchmark accuracy does not guarantee genuine use of visual evidence.<n>We study this problem in traffic accident Video Question Answering (VideoQA)<n>We propose an instance-level Shortcut Score that combines text-only confidence with visual necessity signals.
• Score: 44.46419663487345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High benchmark accuracy does not guarantee genuine use of visual evidence. We study this problem in traffic accident Video Question Answering (VideoQA), where correct answers should depend on scene-specific visual evidence but may instead be inferred from textual shortcuts. Through an audit of four public benchmarks, we find that several recent open-weight Vision-Language Models (VLMs) perform competitively, and sometimes better, without video input. On the MM-AU benchmark, removing video consistently improves accuracy, and adding more frames further degrades performance. To quantify visual dependence, we introduce two dataset-level diagnostics: Blind Gap, measuring above-chance text-only performance, and Visual Gain, measuring the marginal benefit of adding video. We further propose an instance-level Shortcut Score that combines text-only confidence with visual necessity signals, enabling continuous, training-free filtering of shortcut-prone questions. The resulting subsets reduce shortcut bias and improve visual grounding. Our findings reveal large differences in grounding quality across benchmarks and show that visually grounded evaluation, not just high accuracy, is essential in safety-critical VideoQA.

Related papers

• VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models [30.792102059830274]
  Video-Adaptive Debiasing via Evidence Reweighting (VADER) is a training-free framework with two complementary modules.<n>Visual Focus Reallocation (VFR) automatically instantiates an intervention policy for each video-question input.<n> Selective Evidence Erasure (SEE) independently masks high-importance visual tokens in every frame.
  arXiv  Detail & Related papers  (2026-08-09T10:09:11Z)
• VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA [42.454642585543446]
  Video text-based visual question answering (Video TextVQA) aims to answer questions by reasoning over visual textual content appearing in videos.<n>We propose a question-guided agent framework that explicitly anchors the relevants before answering.<n>Our study underscores the critical role of explicit anchoring for advancing Video TextVQA.
  arXiv  Detail & Related papers  (2026-05-06T13:01:52Z)
• How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos [41.46982669196867]
  Video-based quality assurance (QA) for long-form gameplay is labor-intensive and error-prone.<n>Recent benchmarks suggest that vision language models (VLMs) can achieve promising results in detecting visual glitches on curated datasets.
  arXiv  Detail & Related papers  (2026-03-24T01:59:06Z)
• Reducing Text Bias in Synthetically Generated MCQAs for VLMs in Autonomous Driving [1.6039614357284375]
  Multiple Choice Question Answering (MCQA) benchmarks are an established standard for measuring Vision Language Model (VLM) performance in driving tasks.<n>We show that a VLM fine-tuned on such data can achieve accuracy comparable to human-validated benchmarks even without visual input.
  arXiv  Detail & Related papers  (2026-01-28T20:30:26Z)
• A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs [19.46311809796145]
  This paper introduces a simple shortcut-aware video QA benchmark for assessing the physical understanding of video language models.<n>The benchmark is comprised of 55K high-quality multiple-choice video QA examples.<n>Human performance on MVP is 92.9%, while the best open-source state-of-the-art video-language model achieves 40.2%.
  arXiv  Detail & Related papers  (2025-06-11T17:57:32Z)
• Video SimpleQA: Towards Factuality Evaluation in Large Video Language Models [77.96693360763925]
  Video SimpleQA is the first comprehensive benchmark tailored for factuality evaluation in video contexts.<n>Our work differs from existing video benchmarks through the following key features: Knowledge required: demanding integration of external knowledge beyond the video's explicit narrative.<n>Short-form definitive answer: Answers are crafted as unambiguous and definitively correct in a short format with minimal scoring variance.
  arXiv  Detail & Related papers  (2025-03-24T17:46:09Z)
• VaQuitA: Enhancing Alignment in LLM-Assisted Video Understanding [63.075626670943116]
  We introduce a cutting-edge framework, VaQuitA, designed to refine the synergy between video and textual information. At the data level, instead of sampling frames uniformly, we implement a sampling method guided by CLIP-score rankings. At the feature level, we integrate a trainable Video Perceiver alongside a Visual-Query Transformer.
  arXiv  Detail & Related papers  (2023-12-04T19:48:02Z)
• Characterizing Video Question Answering with Sparsified Inputs [55.7455981156755]
  We characterize a task with different input sparsity and provide a tool for doing that. Specifically, we use a Gumbel-based learnable selection module to adaptively select the best inputs for the final task. From our experiments, we have observed only 5.2%-5.8% loss of performance with only 10% of video lengths.
  arXiv  Detail & Related papers  (2023-11-27T21:00:20Z)
• Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model [70.97446870672069]
  Video anomaly detection (VAD) has been paid increasing attention due to its potential applications. Video Anomaly Retrieval ( VAR) aims to pragmatically retrieve relevant anomalous videos by cross-modalities. We present two benchmarks, UCFCrime-AR and XD-Violence, constructed on top of prevalent anomaly datasets.
  arXiv  Detail & Related papers  (2023-07-24T06:22:37Z)
• ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency [62.38914747727636]
  We study self-supervised video representation learning, which is a challenging task due to 1) a lack of labels for explicit supervision and 2) unstructured and noisy visual information. Existing methods mainly use contrastive loss with video clips as the instances and learn visual representation by discriminating instances from each other. In this paper, we observe that the consistency between positive samples is the key to learn robust video representations.
  arXiv  Detail & Related papers  (2021-06-04T08:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.