Video-ToC: Video Tree-of-Cue Reasoning

• URL: http://arxiv.org/abs/2604.20473v1
• Date: Wed, 22 Apr 2026 12:02:24 GMT
• Title: Video-ToC: Video Tree-of-Cue Reasoning
• Abstract summary: Video Large Language Models (Video LLMs) struggle with complex video understanding.<n>Video-ToC is a novel video reasoning framework that enhances video understanding through tree-of-cue reasoning.
• Score: 65.66891474457466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing Video Large Language Models (Video LLMs) struggle with complex video understanding, exhibiting limited reasoning capabilities and potential hallucinations. In particular, these methods tend to perform reasoning solely relying on the pretrained inherent reasoning rationales whilst lacking perception-aware adaptation to the input video content. To address this, we propose \textbf{Video-ToC}, a novel video reasoning framework that enhances video understanding through tree-of-cue reasoning. Specifically, our approach introduces three key innovations: (1) A tree-guided visual cue localization mechanism, which endows the model with enhanced fine-grained perceptual capabilities through structured reasoning patterns; (2) A reasoning-demand reward mechanism, which dynamically adjusts the reward value for reinforcement learning (RL) based on the estimation of reasoning demands, enabling on-demand incentives for more effective reasoning strategies; and (3) An automated annotation pipeline that constructs the Video-ToC-SFT-1k and Video-ToC-RL-2k datasets for supervised fine-tuning (SFT) and RL training, respectively. Extensive evaluations on six video understanding benchmarks and a video hallucination benchmark demonstrate the superiority of Video-ToC over baselines and recent methods. Code is available at https://github.com/qizhongtan/Video-ToC.

Related papers

• VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding [52.925648827667935]
  We introduce VideoKR, the first large-scale training corpus specifically designed to strengthen video understanding.<n>VideoKR comprises 315K video reasoning examples over 145K newly collected, CC-licensed, expert-domain videos.<n>We develop a human-in-the-loop, skill-oriented example generation pipeline that targets progressively deeper video reasoning capabilities.
  arXiv  Detail & Related papers  (2026-06-03T16:14:20Z)
• Clue Matters: Leveraging Latent Visual Clues to Empower Video Reasoning [14.945921705882725]
  This work bridges the perception-to-generation gap in MLLM video understanding, providing an interpretable, faithful reasoning paradigm for VideoQA applications.<n>Inspired by hierarchical human visual cognition, we propose ClueNet, a clue-aware video reasoning framework with a two-stage supervised fine-tuning paradigm.
  arXiv  Detail & Related papers  (2026-03-16T09:15:12Z)
• LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding [106.23494088118571]
  LongVideo-R1 is a multimodal large language model (MLLM) agent for efficient video context navigation.<n>It infers the most informative video clip for subsequent processing.<n>The LongVideo-R1 agent is fine-tuned upon the Qwen-3-8B model through a two-stage paradigm.
  arXiv  Detail & Related papers  (2026-02-24T13:49:47Z)
• Video-CoM: Interactive Video Reasoning via Chain of Manipulations [78.64256470920166]
  We introduce Interactive Video Reasoning, enabling models to "think with videos"<n>Our model, Video CoM, reasons through a Chain of Manipulations (CoM), performing iterative visual actions to gather and refine evidence.<n>Video CoM achieves strong results across nine video reasoning benchmarks, improving average performance by 3.6 percent over recent state of art models.
  arXiv  Detail & Related papers  (2025-11-28T18:59:57Z)
• VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning [35.64831081829936]
  Reinforcement fine-tuning (RFT) has shown great promise in achieving humanlevel reasoning capabilities.<n>VideoRFT follows the standard two-stage scheme in RFT: supervised fine-tuning (SFT) with chain-of-thought (CoT) annotations, followed by reinforcement learning (RL) to improve generalization.<n>We show that VideoRFT achieves state-of-the-art performance on six video reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-05-18T14:14:35Z)
• Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1 [53.894789613838654]
  We introduce SEED-Bench-R1, a benchmark designed to evaluate post-training methods for MLLMs in video understanding.<n>It includes intricate real-world videos and complex everyday planning tasks in the format of multiple-choice questions.<n>Using Qwen2-VL-Instruct-7B as a base model, we compare RL with supervised fine-tuning (SFT)<n>Our detailed analysis reveals that RL enhances visual perception but often produces less coherent reasoning chains.
  arXiv  Detail & Related papers  (2025-03-31T17:55:23Z)
• STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training [87.58996020705258]
  Video Large Language Models (Video-LLMs) have recently shown strong derivation in basic video understanding tasks.<n>Video-LLMs struggle with compositional reasoning that requires multi-step explicit-temporal inference across object relations, interactions and events.<n>We propose STEP, a novel graph-guided self-training method that enables VideoLLMs to generate reasoning-rich finetuning data from any raw videos to improve itself.
  arXiv  Detail & Related papers  (2024-11-29T11:54:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.