VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding

• URL: http://arxiv.org/abs/2512.12360v1
• Date: Sat, 13 Dec 2025 15:11:03 GMT
• Title: VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding
• Abstract summary: VideoARM is an Agentic Reasoning-over-hierarchical-Memory paradigm for long-form video understanding.<n>Instead of static, exhaustive preprocessing, VideoARM performs adaptive, on-the-fly agentic reasoning and memory construction.<n> Experiments on prevalent benchmarks demonstrate that VideoARM outperforms the state-of-the-art method, DVD, while significantly reducing token consumption for long-form videos.
• Score: 21.301669124238693
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-form video understanding remains challenging due to the extended temporal structure and dense multimodal cues. Despite recent progress, many existing approaches still rely on hand-crafted reasoning pipelines or employ token-consuming video preprocessing to guide MLLMs in autonomous reasoning. To overcome these limitations, we introduce VideoARM, an Agentic Reasoning-over-hierarchical-Memory paradigm for long-form video understanding. Instead of static, exhaustive preprocessing, VideoARM performs adaptive, on-the-fly agentic reasoning and memory construction. Specifically, VideoARM performs an adaptive and continuous loop of observing, thinking, acting, and memorizing, where a controller autonomously invokes tools to interpret the video in a coarse-to-fine manner, thereby substantially reducing token consumption. In parallel, a hierarchical multimodal memory continuously captures and updates multi-level clues throughout the operation of the agent, providing precise contextual information to support the controller in decision-making. Experiments on prevalent benchmarks demonstrate that VideoARM outperforms the state-of-the-art method, DVD, while significantly reducing token consumption for long-form videos.

Related papers

• HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning [0.764671395172401]
  Hierarchical Program Probing (HPP) is a framework that reformulates long video understanding as iterative, programmatic exploration of a hierarchically segmented video.<n>To make probing tractable over long videos, we introduce three components: information--aware hierarchical segmentation, late-interaction semantic retrieval, structured and probing functions for coarse-to-fine temporal localization.
  arXiv  Detail & Related papers  (2026-06-19T20:43:49Z)
• InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoning [61.87294123934038]
  We present InternVideo3, a framework enhancing multimodal understanding in video tasks.<n>MCR treats understanding as a closed-loop process over a shared, evolving context.<n>We show InternVideo3 achieves strong performance on benchmarks like Video-MME, MLVU, and Ego.
  arXiv  Detail & Related papers  (2026-06-10T15:17:08Z)
• Watch, Remember, Reason: Human-View Video Understanding with MLLMs [115.44608894992399]
  Video understanding is being rapidly transformed by large language models (MLLMs)<n>This work presents a human-view on LLM-based video understanding.<n>Rather than treating tasks as isolated benchmarks, this view provides a unified structure for analyzing how MLLMs acquire evidence.
  arXiv  Detail & Related papers  (2026-06-05T16:29:13Z)
• VideoMem: Enhancing Ultra-Long Video Understanding via Adaptive Memory Management [17.645183933549458]
  VideoMem is a novel framework that pioneers models long video understanding as a sequential generation task via adaptive memory management.<n>We show that VideoMem significantly outperforms existing open-source models across diverse benchmarks for ultra-long video understanding tasks.
  arXiv  Detail & Related papers  (2025-12-04T07:42:13Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory [59.869552603264076]
  We introduce GCAgent, a novel Global-Context-Aware Agent framework that achieves comprehensive long-video understanding.<n>Our core innovation is the Schematic and Narrative Episodic Memory, which structurally models events and their causal and temporal relations into a concise, organized context.<n>Experiments confirm that GCAgent significantly enhances long-video understanding, achieving up to 23.5% accuracy improvement on the Video-MME Long split over a strong MLLM baseline.
  arXiv  Detail & Related papers  (2025-11-15T04:29:00Z)
• Infinite Video Understanding [50.78256932424239]
  We argue that framing Infinite Video Understanding as a blue-sky research objective provides a vital north star for the multimedia.<n>We outline the core challenges and key research directions towards achieving this transformative capability.
  arXiv  Detail & Related papers  (2025-07-11T23:07:04Z)
• Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding [60.88843818016968]
  Long-form video understanding presents significant challenges due to temporal-spatial complexity and difficulty of question answering.<n>We propose the Deep Video Discovery (DVD) agent to leverage an agentic search strategy over segmented video clips.<n>Our DVD agent achieves state-of-the-art performance on the challenging LVBench dataset, reaching an accuracy of 74.2%.
  arXiv  Detail & Related papers  (2025-05-23T16:37:36Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• $\infty$-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidation [19.616624959353697]
  $infty$-Video can process arbitrarily long videos through a continuous-time long-term memory (LTM) consolidation mechanism.<n>Our framework augments video Q-formers by allowing them to process video contexts efficiently and without requiring additional training.
  arXiv  Detail & Related papers  (2025-01-31T12:45:46Z)
• IQViC: In-context, Question Adaptive Vision Compressor for Long-term Video Understanding LMMs [0.0]
  We propose a framework for long-term video understanding that incorporates a novel visual compressor, the In-context, Question Adaptive Visual (IQViC)<n>IQViC, a transformer-based visual compressor, enables question-conditioned in-context compression, unlike existing methods that rely on full video visual features.<n>We demonstrate the effectiveness of our proposed IQViC framework and its superiority over state-of-the-art methods in terms of video understanding accuracy and memory efficiency.
  arXiv  Detail & Related papers  (2024-12-13T06:52:02Z)
• InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions [104.90258030688256]
  This project introduces disentangled streaming perception, reasoning, and memory mechanisms, enabling real-time interaction with streaming video and audio input.<n>This project simulates human-like cognition, enabling multimodal large language models to provide continuous and adaptive service over time.
  arXiv  Detail & Related papers  (2024-12-12T18:58:30Z)
• Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing [52.050036778325094]
  Video-Ma$2$mba is a novel architecture that incorporates State Space Models (SSMs) within the Mamba-2 framework.<n>Our approach significantly reduces the memory footprint compared to standard gradient checkpointing.<n>By maintaining a detailed capture of temporal dynamics, our model improves the accuracy and relevance of responses in long video understanding tasks.
  arXiv  Detail & Related papers  (2024-11-29T04:12:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.