VTO: Visual Tool Orchestration for Video Anomaly Detection

• URL: http://arxiv.org/abs/2608.08219v1
• Date: Sat, 08 Aug 2026 16:28:36 GMT
• Title: VTO: Visual Tool Orchestration for Video Anomaly Detection
• Abstract summary: Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios.<n>We propose VTO, a process-supervised reinforcement learning framework.<n>VTO enables the agent to dynamically explore and interact with the environment.
• Score: 16.135543208452308
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Traditional deep learning approaches are fundamentally limited by poor generalization across diverse scenarios. While multimodal agents offer a promising tool-learning paradigm for VAD, current systems relying on supervised fine-tuning struggle with complex orchestration, and standard reinforcement learning often causes premature termination due to coarse-grained outcome rewards. To address these challenges, we propose VTO, a process-supervised reinforcement learning framework. Moving beyond static tool usage, VTO enables the agent to dynamically explore and interact with the environment. Specifically, we introduce a foundation model-driven cognitive evaluator to provide context-aware semantic feedback, which is seamlessly integrated into a Process-Supervised Cognitive Alignment that delivers fine-grained, step-wise supervision. By explicitly penalizing logical truncation and rewarding complete causal chains, the agent optimizes its multi-step reasoning policy for interrelated tool orchestration. To support our proposed framework, we meticulously crafted VAD-Tool, a hierarchical visual tool set comprising 12 specialized vision tools spanning from entity tracking to high-stakes hazard detection, and established the corresponding benchmark for rigorous multi-step reasoning evaluation. Extensive experiments on VAD-Tool demonstrate that VTO significantly outperforms baselines, achieving up to a 10.2\% absolute accuracy improvement in tool scheduling. Code and data are available at https://github.com/MICLAB-BUPT/VTO.

Related papers

• Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization [56.150184676745205]
  Video anomaly understanding (VAU) relies on sparse, context-dependent cues.<n>We propose $Anomtext-$, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process.<n>Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy.
  arXiv  Detail & Related papers  (2026-07-01T08:41:49Z)
• OmniVL-Guard Pro: A Tool-Augmented Agent for Omnibus Vision-Language Forensics [63.13200245209719]
  We propose textbf OmniVL-Guard Pro, a tool-augmented agent that extends unified forensics from closed-world prediction to open-world clues-driven reasoning.<n>To generate high-quality tool-reasoning trajectories, we introduce textbfTree-Structured Self-Evolving Tool Trajectory Generation.<n>We also propose textbfChecker-Guided Agentic Reinforcement Learning, which provides process-level supervision to penalize cases where the answer is correct but the reasoning is distorted.
  arXiv  Detail & Related papers  (2026-05-16T12:26:04Z)
• Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models [65.4947731385794]
  We present a unified multi-agent visual reasoning framework that evolves from our foundational image-centric model, Insight-V.<n>We introduce two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness.<n>Experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:28:07Z)
• Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models [7.802379200026965]
  We propose an adaptive framework that dynamically routes VLA execution based on the complexity of the perceived state.<n>Our approach transforms the VLA's vision-language backbone into an active detection tool by projecting latent embeddings into an ensemble of parametric and non-parametric estimators.
  arXiv  Detail & Related papers  (2026-03-05T13:14:41Z)
• Steering LLMs via Scalable Interactive Oversight [74.12746881843044]
  Large Language Models increasingly automate complex, long-horizon tasks such as emphvibe coding, a supervision gap has emerged.<n>It presents a critical challenge in scalable oversight: enabling humans to responsibly steer AI systems on tasks that surpass their own ability to specify or verify.
  arXiv  Detail & Related papers  (2026-02-04T04:52:00Z)
• Agentic Reward Modeling: Verifying GUI Agent via Online Proactive Interaction [7.731207237810125]
  VAGEN is a framework that employs a verifier agent equipped with interaction tools to autonomously plan verification strategies.<n>We show that VAGEN significantly improves evaluation accuracy compared to LLM-as-a-Judge baselines.
  arXiv  Detail & Related papers  (2026-01-31T07:36:54Z)
• Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection [59.04089915447622]
  ForenAgent is an interactive IFD framework that enables MLLMs to autonomously generate, execute, and refine Python-based low-level tools around the detection objective.<n>Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication.<n>Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks.
  arXiv  Detail & Related papers  (2025-12-18T08:38:44Z)
• VICoT-Agent: A Vision-Interleaved Chain-of-Thought Framework for Interpretable Multimodal Reasoning and Scalable Remote Sensing Analysis [10.584087870930354]
  We propose a new multimodal agent framework, Vision-Interleaved Chain-of-Thought Framework (VICoT)<n>VICoT implements explicit multi-round reasoning by dynamically incorporating visual tools into the chain of thought.<n>We also propose the Reasoning Stack distillation method to migrate complex Agent behaviors to small, lightweight models.
  arXiv  Detail & Related papers  (2025-11-25T09:00:28Z)
• SOI is the Root of All Evil: Quantifying and Breaking Similar Object Interference in Single Object Tracking [25.076012214989433]
  We present the first systematic investigation and quantification of Similar Object Interference (SOI)<n> eliminating interference sources leads to substantial performance improvements (AUC gains up to 4.35) across all SOTA trackers.<n>We construct SOIBench-the first semantic cognitive guidance benchmark specifically targeting SOI challenges.
  arXiv  Detail & Related papers  (2025-08-13T06:12:43Z)
• OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning [57.89304342666846]
  We introduce OpenThinkIMG, the first open-source, comprehensive end-to-end framework for tool-augmented LVLMs.<n>We propose a novel reinforcement learning framework V-ToolRL to train LVLMs to learn adaptive policies for invoking external vision tools.<n>V-ToolRL enables LVLMs to autonomously discover optimal tool-usage strategies.
  arXiv  Detail & Related papers  (2025-05-13T14:35:51Z)
• VIBR: Learning View-Invariant Value Functions for Robust Visual Control [3.2307366446033945]
  VIBR (View-Invariant Bellman Residuals) is a method that combines multi-view training and invariant prediction to reduce out-of-distribution gap for RL based visuomotor control. We show that VIBR outperforms existing methods on complex visuo-motor control environment with high visual perturbation.
  arXiv  Detail & Related papers  (2023-06-14T14:37:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.