Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions

• URL: http://arxiv.org/abs/2604.18459v1
• Date: Mon, 20 Apr 2026 16:15:33 GMT
• Title: Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
• Abstract summary: textbfmodel is a framework that decouples reasoning control from memory integration.<n>emphActive Thinking Decision Maker (ATDM) is a transparent reasoning controller that externalizes its decision process.<n>emphHierarchical Progressive Semantic Integration (HPSI) module acts as an efficient memory system.
• Score: 75.23170605943457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual agents operating in the wild must respond to queries precisely when sufficient evidence first appears in a video stream, a critical capability that is overlooked by conventional video LLMs evaluated in offline settings. The shift to an online, streaming paradigm introduces significant challenges: a lack of decision transparency, the difficulty of aligning response timing with visual evidence, and the need to maintain a global, causally consistent understanding under tight computational budgets. To address these issues, we propose a novel framework that decouples reasoning control from memory integration. We introduce \textbf{\model{}}, an instantiation of this framework with two core components. First, the \emph{Active Thinking Decision Maker (ATDM)} is a transparent reasoning controller that externalizes its decision process using observable progress ($\boldsymbolρ$) and confidence ($\boldsymbol{c}$) metrics. This allows it to precisely time its response $t_r$ to match the first-sufficient-evidence timestamp $t^\star$ while streaming its reasoning to the user. Second, the \emph{Hierarchical Progressive Semantic Integration (HPSI)} module acts as an efficient memory system. It employs a set of learnable, multi-level aggregation tokens that are propagated across clips to build a rich, global cognitive state without exceeding token budgets. %Our approach sets a new standard on key online video understanding benchmarks, achieving strong performance of \textbf{71.6\%} on StreamingBench and \textbf{46.9\%} on OVOBench, demonstrating a robust solution for evidence-aligned and transparent online video analysis. Extensive experiments demonstrate the effectiveness of ATDM and HPSI, e.g., Thinking-QwenVL improves the accuracy of the previous state-of-the-art from 67.63\% to 71.60\% on the StreamingBench benchmark.

Related papers

• VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs [38.33158077819229]
  VideoMM is a paradigm shift from model-centric downsizing to adaptive perceptual granularity.<n>It achieves a 6.13$times$ speedup and a 7.4% accuracy gain over full-context baselines on LongVideoBench.
  arXiv  Detail & Related papers  (2026-09-15T06:47:20Z)
• Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understanding [56.17043150838547]
  CausalMem is dedicated to constructing a dynamic visual memory update mechanism.<n>It estimates the redundancy of visual tokens and updates the memory bank via an online semantic basis.<n>It achieves more than textbf20$times$ visual token compression ratio and only occupies about textbf82 MB storage.
  arXiv  Detail & Related papers  (2026-06-24T10:11:08Z)
• MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention [53.80990122465395]
  We show how a cross-attention backbone is better suited to real-time vision-language fusion than the prevailing decoder-only design.<n>Our model trails the strong Qwen2.5-VL-7B baseline overall.<n>On a single H200 with 256 frames per video, achieves it about a 5x speedup in time to first token.
  arXiv  Detail & Related papers  (2026-06-01T09:07:15Z)
• Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory [79.01059178883817]
  IAMFlow is a training-free identity-aware memory framework that explicitly models and tracks persistent entity identities.<n>A VLM asynchronously verifies and refines attributes from rendered frames, enabling explicit entity tracking in place of implicit similarity-based matching.<n>NarraStream-Bench is a benchmark for narrative streaming video generation that features 324 multi-prompt scripts spanning six dimensions.
  arXiv  Detail & Related papers  (2026-05-18T17:54:34Z)
• AdaFocus: Adaptive Relevance-Diversity Sampling with Zero-Cache Look-back for Efficient Long Video Understanding [4.238932162084854]
  Long video understanding is heavily bottlenecked by a rigid one-shot paradigm.<n>We propose AdaFocus, an efficient framework that rethinks long-video understanding as progressive evidence acquisition rather than one-pass encoding.
  arXiv  Detail & Related papers  (2026-05-13T03:40:21Z)
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
  Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:59:56Z)
• Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding [16.537148896706036]
  Video Temporal Grounding (VTG) localizes the temporal boundaries of a query-relevant moment in long, untrimmed videos.<n>We propose SemVID, a training-free pruning framework that constructs a compact yet coherent token subset with semantic roles.
  arXiv  Detail & Related papers  (2026-03-05T20:25:32Z)
• Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models [14.21980212001207]
  Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs.<n>To better match streaming inputs, we propose textbfThink-as-You-See (TaYS), a unified framework enabling true concurrent reasoning.
  arXiv  Detail & Related papers  (2026-03-03T11:24:55Z)
• Going Down Memory Lane: Scaling Tokens for Video Stream Understanding with Dynamic KV-Cache Memory [50.30283773196725]
  Existing approaches rely on key-value caching to accumulate frame-level details over time, but use a limited number of tokens per frame.<n>We propose scaling the token budget to enable more granular-temporal understanding and reasoning.
  arXiv  Detail & Related papers  (2026-02-20T18:59:50Z)
• Fast SAM2 with Text-Driven Token Pruning [52.8350457627401]
  Segment Anything Model 2 (SAM2), a vision computation model has significantly advanced in prompt-driven video object segmentation.<n>SAM2 pipelines propagate all visual tokens produced by the image encoder through downstream temporal reasoning modules, regardless of their relevance to the target object.<n>We introduce a text-guided token pruning framework that improves inference efficiency by selectively reducing token density prior to temporal propagation.
  arXiv  Detail & Related papers  (2025-12-24T18:59:05Z)
• Seeing the Forest and the Trees: Query-Aware Tokenizer for Long-Video Multimodal Language Models [24.875526594002434]
  We present QTSplus, a visual token selection module for long video understanding scenarios.<n>It is integrated into Qwen2.5-VL and compresses the vision stream by up to textbf89% and reduces end-to-end latency by textbf28% on long videos.<n>Results show that QTSplus is an effective, general mechanism for scaling MLLMs to realworld long-video scenarios.
  arXiv  Detail & Related papers  (2025-11-14T22:41:27Z)
• Dense Video Understanding with Gated Residual Tokenization [49.17263029080152]
  High temporal resolution is essential for capturing fine-grained details in video understanding.<n>Current benchmarks rely mostly on low-frame-rate sampling.<n>Dense Video Understanding (DVU) enables high-FPS video comprehension by reducing both tokenization time and token overhead.
  arXiv  Detail & Related papers  (2025-09-17T17:34:40Z)
• VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carriers [23.541896057977745]
  VideoScan is an efficient vision-language model (VLM) inference framework for real-time video interaction.<n>VideoScan employs a single semantic carrier token to represent each frame.
  arXiv  Detail & Related papers  (2025-03-12T13:30:40Z)
• QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video Comprehension [86.0749609778104]
  We propose QuoTA, an ante-hoc training-free modular that extends existing large video-language models.<n>QuoTA strategically allocates frame-level importance scores based on query relevance.<n>We decouple the query through Chain-of-Thoughts reasoning to facilitate more precise LVLM-based frame importance scoring.
  arXiv  Detail & Related papers  (2025-03-11T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.