Ground, Cover, and Refine: Evidence-Centric Frame Selection for Long-Video Question Answering

• URL: http://arxiv.org/abs/2608.01660v1
• Date: Mon, 03 Aug 2026 03:50:22 GMT
• Title: Ground, Cover, and Refine: Evidence-Centric Frame Selection for Long-Video Question Answering
• Abstract summary: Long-video question answering requires identifying sparse yet critical evidence from videos under a constrained visual-token budget.<n>Existing methods either select query-aware frames in a single pass or rely on timestamped text solely as retrieval guidance.<n>We propose GCR, a training-free framework that casts fixed-budget frame selection as a joint evidence curation problem.
• Score: 9.937860041147625
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-video question answering requires identifying sparse yet critical evidence from videos containing thousands of frames under a constrained visual-token budget. Existing methods either select query-aware frames in a single pass or rely on timestamped text solely as retrieval guidance, leading to two key limitations. First, selected frames tend to cluster around local relevance peaks, and once the budget is exhausted, omitted evidence cannot be recovered. Second, textual and visual evidence remain weakly aligned. We propose GCR, a training-free framework that casts fixed-budget frame selection as a joint evidence curation problem. Ground converts timestamped text into temporal events, selects query-relevant real frame anchors, and renders each event text onto its temporally aligned frame. Cover supplements grounded events with direct visual anchors for complementary visual evidence and applies global maximal marginal relevance to preserve diverse context. Refine revisits omitted temporal regions and replaces the weakest revisable context frame with a real-frame medoid---but only when the medoid offers greater evidence value. GCR maintains a fixed number of chronologically ordered frames and requires no VLM training or architectural modification. Experiments on LongVideoBench and Video-MME, across three 7B backbones and frame budgets of 8, 32, and 64, demonstrate consistent improvements in long-video QA. With the 7B LLaVA-OV backbone and 32 frames, GCR achieves 64.25% and 62.15% on the two benchmarks, outperforming the strongest reproduced baselines by 2.54 and 1.93 percentage points, respectively.

Related papers

• Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding [59.773752487686444]
  Long-video understanding on edge devices must reason over hours of content under tight compute and bandwidth budgets.<n>We propose Caption-once, Frames-onDemand, a budget-aware edge-cloud agentic framework.<n> Experiments on long-video benchmarks demonstrate strong accuracy-efficiency trade-offs while substantially reducing online visual processing.
  arXiv  Detail & Related papers  (2026-09-10T17:53:59Z)
• Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval [40.983482307220676]
  PRVR aims to retrieve untrimmed videos that contain moments relevant to a text query.<n>We propose an Intrinsic Temporal Adaptation (ITA) framework for PRVR.<n>Our method achieves state-of-the-art performance on PRVR benchmarks.
  arXiv  Detail & Related papers  (2026-09-04T06:58:59Z)
• RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding [55.75126327155788]
  RIDGE is a frame selection framework that reads the frame-query similarity curve as a temporal signal.<n>It partitions the timeline into structural regions and applies region-specific selection to preserve event cores, transitions, buildup, aftermath, and contextual frames under a fixed budget.<n>It achieves the best performance in most settings and remains competitive in the others.
  arXiv  Detail & Related papers  (2026-08-30T18:40:51Z)
• One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding [43.953366792790625]
  We introduce Matryoshka Evidence-to-Context (MEC) Frame Selection, a training-free framework that builds a reusable sparse video index.<n>MEC improves average accuracy over uniform sampling by 3.77 percentage points, matches strong state-of-the-art selectors, and reduces end-to-end selection latency by 47.37-51.19%.
  arXiv  Detail & Related papers  (2026-08-06T07:47:15Z)
• When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understanding [17.062781672214104]
  We propose EcoFrame, a framework for low-overhead query-adaptive visual evidence scheduling.<n>Experiments on Video-MME, LongVideoBench, and MLVU demonstrate that EcoFrame achieves a better accuracy--efficiency trade-off across multiple VLM backbones.
  arXiv  Detail & Related papers  (2026-08-04T16:49:53Z)
• Coverage-Driven Adaptive Keyframe Selection for Video Understanding [12.078063042214573]
  We propose CSES, a training-free semantic selector that adaptively determines the numbers of frames to score and select.<n> CSES estimates the prominence of the frame-query to guide active acquisition and adapt the temporal coverage of each input.<n>It then formulates selection relevance as a coverage problem that jointly accounts for semantic relevance, temporal redundancy, and visual redundancy.
  arXiv  Detail & Related papers  (2026-08-01T15:24:06Z)
• Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding [17.716106078756066]
  Long-video understanding commonly compresses videos into a small set of frames or visual tokens for answer generation.<n>Our key idea is to organize selected frames into query-relevant cross-frame evidence before generation.<n>Across four benchmarks and two Video-MLLM backbones, GenEvA consistently improves matched-frame baselines.
  arXiv  Detail & Related papers  (2026-07-30T16:55:00Z)
• QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding [78.36929439152566]
  We propose a Query- and Content-Aware (QCA) selection framework that can select a compact yet information-rich set of frames from long videos.<n>QCA anchors on the most query-relevant frame and iteratively incorporates additional frames to maximize diversity while maintaining high semantic relevance to the query.<n>Our method requires no additional training and can be seamlessly integrated into existing Video-LLMs.
  arXiv  Detail & Related papers  (2026-07-01T14:19:24Z)
• ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocation [69.45329019089041]
  Multi-Shot Video Extrapolation (MSVE) is a task that extends an observed frame or clip into a sequence of cinematically structured shots.<n>MSVE reveals that long-video failure is not merely a limitation of context length, but a failure of context allocation.<n>We propose Recursive Context Allocation (ReCA), an inference-time framework that allocates context hierarchically across planning and generation.
  arXiv  Detail & Related papers  (2026-05-26T04:16:56Z)
• ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting [17.594941754364484]
  Video Large Language Models (VideoLLMs) have enabled strong performance across diverse multimodal video tasks.<n>To reduce the high computational cost of processing dense video frames, efficiency-oriented methods such as frame selection have been widely adopted.<n>We introduce ViKey, a training-free framework that combines VP with a lightweight Keyword-Frame Mapping (KFM) module.
  arXiv  Detail & Related papers  (2026-03-24T13:32:52Z)
• Adaptive Greedy Frame Selection for Long Video Understanding [11.923839324117674]
  Large vision--language models (VLMs) are increasingly applied to long-video question answering.<n>Inference is often bottlenecked by the number of input frames and resulting visual tokens.<n>We propose a question-adaptive greedy frame selection method that jointly optimize query relevance and semantic representativeness under a fixed frame budget.
  arXiv  Detail & Related papers  (2026-03-20T17:55:32Z)
• An Empirical Study of Frame Selection for Text-to-Video Retrieval [62.28080029331507]
  Text-to-video retrieval (TVR) aims to find the most relevant video in a large video gallery given a query text. Existing methods typically select a subset of frames within a video to represent the video content for TVR. In this paper, we make the first empirical study of frame selection for TVR.
  arXiv  Detail & Related papers  (2023-11-01T05:03:48Z)
• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)
• Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding [64.99924160432144]
  Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. We propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames.
  arXiv  Detail & Related papers  (2023-01-02T03:38:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.