ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding

• URL: http://arxiv.org/abs/2605.13228v1
• Date: Wed, 13 May 2026 09:19:22 GMT
• Title: ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding
• Abstract summary: Video understanding requires active evidence seeking, motivating tool-augmented video agents for temporal reasoning, cross-modal understanding, and complex question answering.<n>Existing video agents have improved video reasoning with retrieval, memory, frame inspection, and verifier tools, but they still face two limitations.<n>In this paper, we address these challenges with two complementary designs.
• Score: 30.50286834537385
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video understanding requires active evidence seeking, motivating tool-augmented video agents for temporal reasoning, cross-modal understanding, and complex question answering. Existing video agents have improved video reasoning with retrieval, memory, frame inspection, and verifier tools, but they still face two limitations: (1) a coarse tool space that lacks fine-grained operations for compositional reasoning; and (2) a flat action space that forces high-level video intents into primitive executable tool calls. In this paper, we address these challenges with two complementary designs. First, we construct a MetaAug-Video Tool Library (MVTL), an extensible tool library with 134 registered tools, including 26 base tools for general multimodal signal processing and 108 meta tools for filtering, aggregation, reranking, formatting, and other intermediate-result operations. MVTL supports dual-level access to both structured video information and raw modal evidence, enabling diverse video reasoning scenarios. Second, we propose ReTool-Video, a recursive tool-using method that grounds high-level video intents into executable tool chains. In ReTool-Video, matched actions are executed directly, while unmatched intents are delegated to a resolver for parameter repair, tool substitution, or decomposition. This allows abstract actions such as temporal merging, cross-modal verification, or repeated-event aggregation to be progressively translated into concrete multimodal operations at runtime. Experiments on MVBench, MLVU, and Video-MME w/o sub. show that ReTool-Video consistently outperforms strong baselines. Further analysis demonstrates that recursive grounding and fine-grained meta tools improve the stability and effectiveness of complex video understanding.

Related papers

• LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agents [31.505673981223385]
  Long-video understanding requires models to efficiently acquire and reuse sparse visual evidence from long and redundant video streams.<n>Recent video tool-use agents address this challenge by iteratively invoking visual Tools at different temporal scales.<n>We propose Latent Visual Evidence-Enhanced Planning (LAVE), a training-free framework for reusing latent visual evidence from completed Tool calls.
  arXiv  Detail & Related papers  (2026-08-05T10:25:47Z)
• VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG [33.938909878793815]
  VideoStir is a structured and multimodal intent-aware long-video RAG framework.<n>It structures a video as atemporal graph at clip level, and then performs multi-hop retrieval to aggregate evidence.<n>It also introduces an MLLM-backed intent-relevance scorer that retrieves frames based on their alignment with the query's reasoning intent.
  arXiv  Detail & Related papers  (2026-04-07T04:26:59Z)
• VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking [49.09710846673772]
  VideoSeek is a video agent that actively seeks answer-critical evidence instead of exhaustively parsing the full video.<n>VideoSeek operates in a think-act-observe loop with a well-designed toolkit for collecting multi-granular video observations.
  arXiv  Detail & Related papers  (2026-03-20T17:58:47Z)
• Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task [8.444156994500881]
  VideoQA task serves as a critical playground for evaluating whether foundation models can effectively perceive, understand, and reason about dynamic real-world scenarios.<n>We equip Multimodal Large Language Models (MLLMs) with a comprehensive Video Toolkit to enhance spatial reasoning capabilities.<n>We propose a Stemporal Reasoning Framework (STAR) that strategically schedules temporal and spatial tools, thereby progressively localizing the key area in the video.
  arXiv  Detail & Related papers  (2025-12-11T07:17:57Z)
• Towards Effective and Efficient Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrieval [57.88666884515147]
  We propose One-shot video-Clip based Retrieval AuGmentation (OneClip-RAG)<n>OneClip-RAG makes full use of the merits of video clips for augmented video understanding.<n>It is also equipped with a novel query-guided video chunking algorithm.
  arXiv  Detail & Related papers  (2025-12-09T09:40:20Z)
• LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling [87.98096428508181]
  LongVT is an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought.<n>We exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip and resample finer-grained video frames.<n>Our training dataset consists of 247.9K samples for tool-integrated cold-start supervised fine-tuning, 1.6K samples for agentic reinforcement learning, and 15.4K samples for agentic reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2025-11-25T19:22:48Z)
• COLT: Enhancing Video Large Language Models with Continual Tool Usage [9.709506072510512]
  We propose COntinuaL Tool usage (termed COLT), which automatically acquires tool-use ability in a successive tool stream.<n>Our COLT incorporates a learnable tool codebook as a tool-specific memory system.<n>To unleash the tool usage potential of video LLMs, we collect a video-centric tool-use instruction tuning dataset VideoToolBench.
  arXiv  Detail & Related papers  (2025-09-23T07:49:30Z)
• CAViAR: Critic-Augmented Video Agentic Reasoning [90.48729440775223]
  We ask: can perception capabilities be leveraged to perform more complex video reasoning?<n>We develop a large language model agent given access to video modules as subagents or tools.<n>We show that the combination of our agent and critic achieve strong performance on datasets.
  arXiv  Detail & Related papers  (2025-09-09T17:59:39Z)
• Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding [60.88843818016968]
  Long-form video understanding presents significant challenges due to temporal-spatial complexity and difficulty of question answering.<n>We propose the Deep Video Discovery (DVD) agent to leverage an agentic search strategy over segmented video clips.<n>Our DVD agent achieves state-of-the-art performance on the challenging LVBench dataset, reaching an accuracy of 74.2%.
  arXiv  Detail & Related papers  (2025-05-23T16:37:36Z)
• ControlLLM: Augment Language Models with Tools by Searching on Graphs [97.62758830255002]
  We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving real-world tasks. Our framework comprises three key components: (1) a textittask decomposer that breaks down a complex task into clear subtasks with well-defined inputs and outputs; (2) a textitThoughts-on-Graph (ToG) paradigm that searches the optimal solution path on a pre-built tool graph; and (3) an textitexecution engine with a rich toolbox that interprets the solution path and runs the
  arXiv  Detail & Related papers  (2023-10-26T21:57:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.