Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding

• URL: http://arxiv.org/abs/2607.24794v1
• Date: Tue, 30 Jun 2026 10:55:56 GMT
• Title: Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
• Abstract summary: ReMem is a temporal granularity-adaptive selection framework for training-free LongVideoQA.<n>By explicitly preserving temporal information with memory mechanisms, ReMem suppresses redundancy and empowers MLLMs to perform robust multi-granular video reasoning.
• Score: 47.48582887762574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Multimodal Large Language Models (MLLMs) demonstrate superior generalization in fundamental video tasks, restricted context windows limit their long video understanding. To accommodate this constraint, models typically resort to keyframe selection. However, uniform sampling or static query-guided selection often overlooks critical temporal context, failing to adapt to the varying query temporal granularities. In this paper, we propose ReMem, a temporal granularity-adaptive keyframe selection framework for training-free LongVideoQA. ReMem introduces a dual-level memory-augmented adaptation. At the query level, Memory-Driven Question Parsing leverages LLM long-term memory to decode question temporal granularity and extract semantic entities. At the video level, Synergistic Dual-Semantic Frame Alignment exploits intrinsic structural memory to align frames with query semantics, guiding Structure-Aware Dynamic Frame Routing to cluster events and optimally distribute sampling budgets. By explicitly preserving temporal information with memory mechanisms, ReMem suppresses redundancy and empowers MLLMs to perform robust multi-granular video reasoning. Evaluations across four popular LongVideoQA benchmarks using three MLLMs demonstrate highly efficient, state-of-the-art zero-shot performance; notably, LLaVA-Video with ReMem reaches 54.5% (+12.3%) on LVBench and 67.1% (+8.2%) on LongVideoBench.

Related papers

• WAT: Online Video Understanding Needs Watching Before Thinking [25.136741695647213]
  WAT (Watching Before Thinking) is a two-stage framework for online video reasoning.<n>It separates processing into a query-independent watching stage and a query-triggered thinking stage.<n>It achieves state-of-the-art performance on online video benchmarks, including 77.7% accuracy on StreamingBench and 55.2% on OVO-Bench.
  arXiv  Detail & Related papers  (2026-03-12T13:42:32Z)
• See More, Store Less: Memory-Efficient Resolution for Video Moment Retrieval [5.835635134105812]
  We propose SMORE (See MORE, store less), a framework that enhances memory efficiency while maintaining high information resolution.<n>SMORE (1) uses query-guided captions to encode semantics aligned with user intent, (2) applies query-aware importance modulation to highlight relevant segments, and (3) adaptively compresses frames to preserve key content.<n> Experimental validation reveals that SMORE achieves state-of-the-art performance on QVHighlights, Charades-STA, and ActivityNet-Captions benchmarks.
  arXiv  Detail & Related papers  (2026-01-14T10:28:11Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• Episodic Memory Representation for Long-form Video Understanding [52.33907540905242]
  Large Video Language Models excel at general video understanding but struggle with long-form context window limits.<n>We introduce Video-EM, a training free framework inspired by the principles of human memory.<n>Video-EM achieves performance gains of 4-9 percent over respective baselines while utilizing fewer frames.
  arXiv  Detail & Related papers  (2025-08-13T04:33:07Z)
• Moment Sampling in Video LLMs for Long-Form Video QA [22.638644170177013]
  "moment sampling" is a model-agnostic approach that enables the model to select the most relevant frames according to the context of the question.<n>By focusing on the frames most pertinent to the given question, our method enhances long-form VideoQA performance in Video LLMs.
  arXiv  Detail & Related papers  (2025-06-18T03:23:56Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• ReWind: Understanding Long Videos with Instructed Learnable Memory [8.002949551539297]
  Vision-Language Models (VLMs) are crucial for applications requiring integrated understanding textual and visual information.<n>We introduce ReWind, a novel memory-based VLM designed for efficient long video understanding while preserving temporal fidelity.<n>We empirically demonstrate ReWind's superior performance in visual question answering (VQA) and temporal grounding tasks, surpassing previous methods on long video benchmarks.
  arXiv  Detail & Related papers  (2024-11-23T13:23:22Z)
• LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding [65.46303012350207]
  LongVU is an adaptive compression mechanism that reduces the number of video tokens while preserving visual details of long videos. We leverage DINOv2 features to remove redundant frames that exhibit high similarity. We perform spatial token reduction across frames based on their temporal dependencies.
  arXiv  Detail & Related papers  (2024-10-22T21:21:37Z)
• VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges [39.666361965650836]
  VideoLLaMB is a framework for long video understanding.<n> SceneTiling algorithm segments videos into coherent semantic units.<n>VideoLLaMB processes up to 320 frames using a single Nvidia A100 GPU.
  arXiv  Detail & Related papers  (2024-09-02T08:52:58Z)
• Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA [40.21221568678641]
  Long-form videos that span across wide temporal intervals are highly information redundant.<n>All information necessary to generate a correct response can often be contained within a small subset of frames.<n>Recent literature explore use of large language models in LVQA benchmarks, achieving exceptional performance.
  arXiv  Detail & Related papers  (2024-06-13T17:59:16Z)
• Streaming Long Video Understanding with Large Language Models [83.11094441893435]
  VideoStreaming is an advanced vision-language large model (VLLM) for video understanding. It capably understands arbitrary-length video with a constant number of video streaming tokens encoded and propagatedly selected. Our model achieves superior performance and higher efficiency on long video benchmarks.
  arXiv  Detail & Related papers  (2024-05-25T02:22:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.