POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency

• URL: http://arxiv.org/abs/2510.01009v1
• Date: Wed, 01 Oct 2025 15:15:36 GMT
• Title: POVQA: Preference-Optimized Video Question Answering with Rationales for Data Efficiency
• Abstract summary: Recent advancements in large context/long video question answering have allowed VQA tasks to have context window of 1500+ frames.<n>We introduce POVQA, a data-efficient pipeline that compresses each second of video into a single temporally pooled image.
• Score: 3.4998703934432682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video Question Answering (VQA) with Large Vision Language Models (LVLMs) has gained significant traction in research ever since the Flamingo was introduced by Deepmind. Recent advancements in large context/long video question answering have allowed VQA tasks to have context window of 1500+ frames. However, this only leads to 50 seconds of video footage without losing any significant information. We introduce POVQA, a data-efficient pipeline that compresses each second of video into a single temporally pooled image (via motion blur and weighted averaging variants) and then align LVLMs with lightweight supervision. Concretely, we build 1 fps input sources using Blend Blur with Last Frame, Weighted Average, Exponential and Ramp pooling and fine-tune QWEN-2.5-VL 7B with supervised two turn target including reasoning and final answer. We apply Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO) on our novel dataset ReasonVQA consisting of 12 movies with 239 human annotated question-answer with reasoning prompts. On our ReasonVQA dataset, this method dramatically improves performance over pooled baselines: F1 score improves from 0.212 to 0.543, BLEU-4 from 0.031 to 0.291, and ROUGE-L from 0.196 to 0.528. Rationale quality also significantly increases. Cross-evaluation of SFT + DPO on various pooling functions show that the gains persist regardless of the pooling scheme used at train or test time, indicating strong robustness on summarization of temporal evidence. Similar observations were made on zero-shot in TVQA.

Related papers

• LongVU-TTT: Causal Test-Time Training for Visual Resampling in Long Video Understanding [30.902165004701683]
  LongVU-TTT inserts a convolutional Test-Time Training (TTT) resampler with causal fast-weight updates between the vision encoder and the LLM.<n>Its grouped 2D fast weights adapt to each video and contextualize frame features before compression, while a hybrid uniform-and-change-aware selector retains explicit visual evidence for downstream reasoning.<n>LongVU-TTT processes up to 512 frames before reducing them to 128 LLM frames and achieves competitive performance across five video understanding benchmarks.
  arXiv  Detail & Related papers  (2026-08-26T12:43:43Z)
• Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model [60.85752511708308]
  We present Mage-VL, an efficient streaming foundation model for real-time multimodal understanding and interaction.<n>At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions.<n>We establish AI4AI data pipelines encompassing prompt-code joint optimization for captioning and AI-driven performance diagnosis.
  arXiv  Detail & Related papers  (2026-07-27T17:59:53Z)
• Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks [2.0789144738685956]
  We introduce the Visual Dependency Gap (VDG), the difference in per-question correctness between original-video and black-screen conditions.<n>We show that accuracy and visual dependency are separable.<n>These results motivate VDG as a standard audit for whether video benchmarks measure visually grounded capability.
  arXiv  Detail & Related papers  (2026-07-14T22:15:50Z)
• Evidence-Backed Video Question Answering [90.93225758130426]
  Existing explainability efforts rely on textual rationales or sparse bounding boxes.<n>We propose Evidence-Backed Video Question Answering (E-VQA)<n>To support this, we introduce ST-Evidence, the first human-verified benchmark for both discriminative and generative pixel-level grounding.
  arXiv  Detail & Related papers  (2026-07-13T17:49:10Z)
• VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA [42.454642585543446]
  Video text-based visual question answering (Video TextVQA) aims to answer questions by reasoning over visual textual content appearing in videos.<n>We propose a question-guided agent framework that explicitly anchors the relevants before answering.<n>Our study underscores the critical role of explicit anchoring for advancing Video TextVQA.
  arXiv  Detail & Related papers  (2026-05-06T13:01:52Z)
• How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos [41.46982669196867]
  Video-based quality assurance (QA) for long-form gameplay is labor-intensive and error-prone.<n>Recent benchmarks suggest that vision language models (VLMs) can achieve promising results in detecting visual glitches on curated datasets.
  arXiv  Detail & Related papers  (2026-03-24T01:59:06Z)
• VideoBrain: Learning Adaptive Frame Sampling for Long Video Understanding [9.415923244280542]
  VideoBrain is an end-to-end framework that enables Vision-Language Models to adaptively acquire visual information through learned sampling policies.<n>Our approach features dual complementary agents: a CLIP-based agent for semantic retrieval across the video and a Uniform agent for dense temporal sampling within intervals.
  arXiv  Detail & Related papers  (2026-02-04T00:08:35Z)
• LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling [87.98096428508181]
  LongVT is an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought.<n>We exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip and resample finer-grained video frames.<n>Our training dataset consists of 247.9K samples for tool-integrated cold-start supervised fine-tuning, 1.6K samples for agentic reinforcement learning, and 15.4K samples for agentic reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2025-11-25T19:22:48Z)
• LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning [73.90466023069125]
  We propose LOVE-R1, a model that can adaptively zoom in on a video clip.<n>The model is first provided with densely sampled frames but in a small resolution.<n>If some spatial details are needed, the model can zoom in on a clip of interest with a large frame resolution.
  arXiv  Detail & Related papers  (2025-09-29T13:43:55Z)
• Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames [70.93346841539626]
  We present Temporal Chain of Thought, an inference strategy for video question-answering.<n>We use the VLM itself to iteratively identify and extract the most relevant frames from the video.<n>We demonstrate how leveraging more computation at inference-time to select the most relevant context leads to improvements in accuracy.
  arXiv  Detail & Related papers  (2025-07-01T18:39:26Z)
• T*: Re-thinking Temporal Search for Long-Form Video Understanding [66.72243342954823]
  Current temporal search methods only achieve 2.1% temporal F1 score on the Longvideobench subset.<n>Inspired by visual search in images, we propose a lightweight temporal search framework, T* that reframes costly temporal search as spatial search.<n>Extensive experiments show that integrating T* with existing methods significantly improves SOTA long-form video understanding.
  arXiv  Detail & Related papers  (2025-04-03T04:03:10Z)
• BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding [51.49345400300556]
  Large video-language models (VLMs) have demonstrated promising progress in various video understanding tasks.<n>Traditional approaches, such as uniform frame sampling, often inevitably allocate resources to irrelevant content.<n>We introduce BOLT, a method to BOost Large VLMs without additional Training through a comprehensive study of frame selection strategies.
  arXiv  Detail & Related papers  (2025-03-27T13:18:40Z)
• FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Grounding [25.21011724370177]
  Text-guided Video Temporal Grounding (VTG) aims to localize relevant segments in untrimmed videos based on textual descriptions.<n>We introduce FlashVTG, a framework featuring a Temporal Feature Layering (TFL) module and an Adaptive Score Refinement (ASR) module.<n>FlashVTG achieves state-of-the-art performance on four widely adopted datasets in both Moment Retrieval (MR) and Highlight Detection (HD)
  arXiv  Detail & Related papers  (2024-12-18T02:23:33Z)
• A Simple LLM Framework for Long-Range Video Question-Answering [63.50439701867275]
  We present LLoVi, a language-based framework for long-range video question-answering (LVQA) Our approach uses a frame/clip-level visual captioner coupled with a Large Language Model (GPT-3.5, GPT-4) Our method achieves 50.3% accuracy, outperforming the previous best-performing approach by 18.1% (absolute gain)
  arXiv  Detail & Related papers  (2023-12-28T18:58:01Z)
• Temporal Bilinear Encoding Network of Audio-Visual Features at Low Sampling Rates [7.1273332508471725]
  This paper aims to exploit audio-visual information in video classification with a 1 frame per second sampling rate. We propose Temporal Bilinear Networks (TBEN) for encoding both audio and visual long range temporal information.
  arXiv  Detail & Related papers  (2020-12-18T14:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.