Video Detective: Seek Critical Clues Recurrently to Answer Question from Long Videos

• URL: http://arxiv.org/abs/2512.17229v1
• Date: Fri, 19 Dec 2025 04:29:07 GMT
• Title: Video Detective: Seek Critical Clues Recurrently to Answer Question from Long Videos
• Abstract summary: Long Video Question-Answering (LVQA) presents a significant challenge for Multi-modal Large Language Models (MLLMs)<n>We propose an efficient question-aware memory mechanism, enabling MLLMs to recurrently seek these critical clues.<n>Our method enables MLLMs with limited context length of 32K to efficiently process 100K tokens (3600 frames, an hour-long video sampled at 1fps), requiring only 2 minutes and 37GB GPU memory usage.
• Score: 48.34455169866522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long Video Question-Answering (LVQA) presents a significant challenge for Multi-modal Large Language Models (MLLMs) due to immense context and overloaded information, which could also lead to prohibitive memory consumption. While existing methods attempt to address these issues by reducing visual tokens or extending model's context length, they may miss useful information or take considerable computation. In fact, when answering given questions, only a small amount of crucial information is required. Therefore, we propose an efficient question-aware memory mechanism, enabling MLLMs to recurrently seek these critical clues. Our approach, named VideoDetective, simplifies this task by iteratively processing video sub-segments. For each sub-segment, a question-aware compression strategy is employed by introducing a few special memory tokens to achieve purposefully compression. This allows models to effectively seek critical clues while reducing visual tokens. Then, due to history context could have a significant impact, we recurrently aggregate and store these memory tokens to update history context, which would be reused for subsequent sub-segments. Furthermore, to more effectively measure model's long video understanding ability, we introduce GLVC (Grounding Long Video Clues), a long video question-answering dataset, which features grounding critical and concrete clues scattered throughout entire videos. Experimental results demonstrate our method enables MLLMs with limited context length of 32K to efficiently process 100K tokens (3600 frames, an hour-long video sampled at 1fps), requiring only 2 minutes and 37GB GPU memory usage. Evaluation results across multiple long video benchmarks illustrate our method can more effectively seek critical clues from massive information.

Related papers

• Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language [60.91064560080974]
  We propose an efficient approach, SpotVMR, to trim the query-relevant clip.<n>Our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods.
  arXiv  Detail & Related papers  (2026-05-28T11:42:28Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos [59.391265901911005]
  We propose CoT-RVS, a novel framework employing the zero-shot Chain-of-Thought (CoT) capability of MLLM to address complex challenges by temporal-semantic reasoning.<n>CoT-RVS analyzes the visible objects within a given frame that possibly match the language query (semantic), and chooses a corresponding for each object that can be observed effortlessly among all frames (temporal)<n>Our framework's training-free feature further allows its extension to process online video streams, where the CoT is used at test time to update the object of interest when a better target starts to emerge
  arXiv  Detail & Related papers  (2025-05-24T07:01:31Z)
• Long-VMNet: Accelerating Long-Form Video Understanding via Fixed Memory [5.311777874655448]
  Long-Video Memory Network, Long-VMNet, is a novel video understanding method.<n>Long-VMNet achieves improved efficiency by leveraging a neural sampler that identifies discriminative tokens.<n>Our results on the Rest-ADL dataset demonstrate an 18x -- 75x improvement in inference times for long-form video retrieval and answering questions.
  arXiv  Detail & Related papers  (2025-03-17T20:25:41Z)
• AdaCM$^2$: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction [10.579335027350263]
  AdaCM$2$ is an adaptive cross-modality memory reduction approach to video-text alignment on video streams.<n>It achieves a 4.5% improvement across multiple tasks in the LVU dataset with a GPU memory consumption reduction of up to 65%.
  arXiv  Detail & Related papers  (2024-11-19T18:04:13Z)
• LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding [65.46303012350207]
  LongVU is an adaptive compression mechanism that reduces the number of video tokens while preserving visual details of long videos. We leverage DINOv2 features to remove redundant frames that exhibit high similarity. We perform spatial token reduction across frames based on their temporal dependencies.
  arXiv  Detail & Related papers  (2024-10-22T21:21:37Z)
• Visual Context Window Extension: A New Perspective for Long Video Understanding [45.134271969594614]
  We tackle the challenge of long video understanding from the perspective of context windows. We propose to adapt LMMs for long video understanding tasks by extending the visual context window. Our method consistently improves the performance as the number of video frames increases.
  arXiv  Detail & Related papers  (2024-09-30T07:25:16Z)
• Hallucination Mitigation Prompts Long-term Video Understanding [36.26790392889717]
  This paper constructs a comprehensive hallucination mitigation pipeline based on existing MLLMs. We use the CLIP Score to guide the frame sampling process with questions, selecting key frames relevant to the question. During the answer generation stage, we utilize chain-of-thought and in-context learning techniques to explicitly control the generation of answers.
  arXiv  Detail & Related papers  (2024-06-17T08:44:03Z)
• Streaming Long Video Understanding with Large Language Models [83.11094441893435]
  VideoStreaming is an advanced vision-language large model (VLLM) for video understanding. It capably understands arbitrary-length video with a constant number of video streaming tokens encoded and propagatedly selected. Our model achieves superior performance and higher efficiency on long video benchmarks.
  arXiv  Detail & Related papers  (2024-05-25T02:22:09Z)
• LongVLM: Efficient Long Video Understanding via Large Language Models [55.813206751150716]
  LongVLM is a simple yet powerful VideoLLM for long video understanding. We encode video representations that incorporate both local and global information. Our model produces more precise responses for long video understanding.
  arXiv  Detail & Related papers  (2024-04-04T11:33:29Z)
• Long Video Understanding with Learnable Retrieval in Video-Language Models [48.3525267216256]
  We introduce a learnable retrieval-based video-language model (R-VLM) for efficient long video understanding.<n>Specifically, given a question (Query) and a long video, our model identifies and selects the most relevant K video chunks.<n>This effectively reduces the number of video tokens, eliminates noise interference, and enhances system performance.
  arXiv  Detail & Related papers  (2023-12-08T09:48:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.