RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding

• URL: http://arxiv.org/abs/2608.29958v1
• Date: Sun, 30 Aug 2026 18:40:51 GMT
• Title: RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understanding
• Abstract summary: RIDGE is a frame selection framework that reads the frame-query similarity curve as a temporal signal.<n>It partitions the timeline into structural regions and applies region-specific selection to preserve event cores, transitions, buildup, aftermath, and contextual frames under a fixed budget.<n>It achieves the best performance in most settings and remains competitive in the others.
• Score: 55.75126327155788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long videos contain far more visual content than Large Vision-Language Models (LVLMs) can process under a fixed visual-token budget, making frame selection essential. Existing query-aware selectors usually estimate frame-query relevance and build a compact subset from high-scoring frames. Although their mechanisms differ, the similarity sequence is still often treated primarily as values to rank or sample from, rather than as an ordered signal whose shape reflects how query-relevant evidence emerges, peaks, and fades over time. This can obscure frames that explain, contextualize, or follow an event, because such evidence may lie on the rising or falling sides of a nearby relevance peak and receive lower absolute scores. We propose RIDGE, a frame selection framework that reads the frame-query similarity curve as a temporal signal. By using local changes and curvature, RIDGE partitions the timeline into structural regions and applies region-specific selection to preserve event cores, transitions, buildup, aftermath, and contextual frames under a fixed budget. It is a lightweight post-processing step on precomputed frame-query scores and requires neither training nor iterative LVLM calls. Across four long-video benchmarks and three backbones, RIDGE achieves the best performance in most settings and remains competitive in the others.

Related papers

• MEDR: Query-Independent Frame Selection via Multi-Signal Event Modeling and Dynamic Rescoring [17.900940738333407]
  Frame selection is a fundamental component of multimodal large language models.<n>Uniform sampling preserves temporal coverage but may miss informative content that appears only briefly.<n>We propose a training-free and query-independent frame selection method.
  arXiv  Detail & Related papers  (2026-08-15T06:10:43Z)
• One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding [43.953366792790625]
  We introduce Matryoshka Evidence-to-Context (MEC) Frame Selection, a training-free framework that builds a reusable sparse video index.<n>MEC improves average accuracy over uniform sampling by 3.77 percentage points, matches strong state-of-the-art selectors, and reduces end-to-end selection latency by 47.37-51.19%.
  arXiv  Detail & Related papers  (2026-08-06T07:47:15Z)
• Ground, Cover, and Refine: Evidence-Centric Frame Selection for Long-Video Question Answering [9.937860041147625]
  Long-video question answering requires identifying sparse yet critical evidence from videos under a constrained visual-token budget.<n>Existing methods either select query-aware frames in a single pass or rely on timestamped text solely as retrieval guidance.<n>We propose GCR, a training-free framework that casts fixed-budget frame selection as a joint evidence curation problem.
  arXiv  Detail & Related papers  (2026-08-03T03:50:22Z)
• VisualRouter: Query-Grounded Visual Sampling for Long Video Understanding [60.161767365040554]
  We propose a training-free and plug-and-play framework for query-grounded visual sampling.<n>Visual first classifies each query as either global or local and then applies the corresponding sampling strategy.<n>Experiments show that Visual consistently improves multiple LVLMs over uniform sampling, achieving gains of 5.2%, 7.7%, and 11.6% on Video-MME, LongVideoBench, and MLVU with Qwen2.5-VL-7B, and outperforming existing training-free visual sampling methods under the same setting.
  arXiv  Detail & Related papers  (2026-07-30T16:23:10Z)
• Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors [69.75895255395702]
  Cross-modal attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation.<n>A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer attention into relevance scores through query-conditioned aggregation.<n>Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on Video-MME and outperforms prior training-based selectors under matched frame budgets.
  arXiv  Detail & Related papers  (2026-07-17T07:04:31Z)
• QCA: Query- and Content-Aware Keyframe Selection for Long Video Understanding [78.36929439152566]
  We propose a Query- and Content-Aware (QCA) selection framework that can select a compact yet information-rich set of frames from long videos.<n>QCA anchors on the most query-relevant frame and iteratively incorporates additional frames to maximize diversity while maintaining high semantic relevance to the query.<n>Our method requires no additional training and can be seamlessly integrated into existing Video-LLMs.
  arXiv  Detail & Related papers  (2026-07-01T14:19:24Z)
• Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs [81.04673240949074]
  Reinforcement learning has improved the reasoning ability of large language models.<n>Applying outcome-only rewards to video multimodal large language models provides limited guidance on which visual evidence should support the answer.<n>We introduce Consensus Frame GRPO, a temporal-annotation-free process-level reward framework for evidence-aware video reasoning.
  arXiv  Detail & Related papers  (2026-06-16T19:42:54Z)
• Adaptive Greedy Frame Selection for Long Video Understanding [11.923839324117674]
  Large vision--language models (VLMs) are increasingly applied to long-video question answering.<n>Inference is often bottlenecked by the number of input frames and resulting visual tokens.<n>We propose a question-adaptive greedy frame selection method that jointly optimize query relevance and semantic representativeness under a fixed frame budget.
  arXiv  Detail & Related papers  (2026-03-20T17:55:32Z)
• Event-Anchored Frame Selection for Effective Long-Video Understanding [67.56884568828508]
  Event-Anchored Frame Selection (EFS) is a hierarchical, event-aware pipeline.<n>As a training-free, plug-and-play module, EFS can be seamlessly integrated into off-the-shelf LVLMs.
  arXiv  Detail & Related papers  (2026-03-01T08:25:37Z)
• Wavelet-based Frame Selection by Detecting Semantic Boundary for Long Video Understanding [43.587729230845525]
  Current methods typically select frames with high relevance to a given query.<n>We introduce Wavelet-based Frame Selection by Detecting Semantic Boundary (WFS-SB), a training-free framework.<n>WFS-SB significantly boosts LVLM performance, improving accuracy by 5.5% on VideoMME, 9.5% on MLVU, and 6.2% on LongVideoBench.
  arXiv  Detail & Related papers  (2026-02-28T07:18:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.