STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding

• URL: http://arxiv.org/abs/2603.27593v1
• Date: Sun, 29 Mar 2026 09:23:45 GMT
• Title: STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understanding
• Abstract summary: Real-world deployments increasingly require streaming perception and proactive interaction.<n>In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem.<n>We propose STRIDE, which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals.
• Score: 77.20037111885226
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recent progress in video large language models (Video-LLMs) has enabled strong offline reasoning over long and complex videos. However, real-world deployments increasingly require streaming perception and proactive interaction, where video frames arrive online and the system must decide not only what to respond, but also when to respond. In this work, we revisit proactive activation in streaming video as a structured sequence modeling problem, motivated by the observation that temporal transitions in streaming video naturally form span-structured activation patterns. To capture this span-level structure, we model activation signals jointly over a sliding temporal window and update them iteratively as new frames arrive. We propose STRIDE (Structured Temporal Refinement with Iterative DEnoising), which employs a lightweight masked diffusion module at the activation interface to jointly predict and progressively refine activation signals across the window. Extensive experiments on diverse streaming benchmarks and downstream models demonstrate that STRIDE shows more reliable and temporally coherent proactive responses, significantly improving when-to-speak decision quality in online streaming scenarios.

Related papers

• Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation [75.95737622994548]
  Event cameras provide high-resolution motion cues that are well suited for bridging temporal gaps.<n>We propose an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-video diffusion model.
  arXiv  Detail & Related papers  (2026-08-11T04:47:44Z)
• Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing [7.704586969939547]
  Video-text temporal localization requires precise alignment between natural language queries and corresponding video segments.<n>We present a novel framework that addresses two critical limitations of existing methods.<n>Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained video-language understanding.
  arXiv  Detail & Related papers  (2026-07-06T13:52:46Z)
• Act2See: Emergent Active Visual Perception for Video Reasoning [70.52373240573856]
  Vision-Language Models (VLMs) typically rely on static initial frames for video reasoning.<n>We introduce Act-to-See (Act2See), a novel framework that enables active visual perception.
  arXiv  Detail & Related papers  (2026-05-03T00:52:51Z)
• AURA: Always-On Understanding and Real-Time Assistance via Video Streams [37.80002196035961]
  AURA (Always-On Understanding and Real-Time Assistance) is an end-to-end streaming visual interaction framework.<n>It enables a unified VideoLLM to continuously process video streams and support both real-time question answering and proactive responses.<n>AURA achieves state-of-the-art performance on streaming benchmarks and supports a real-time demo system with ASR and TTS running at 2 FPS on two 80G accelerators.
  arXiv  Detail & Related papers  (2026-04-05T16:53:46Z)
• ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning [32.581163427518185]
  ActionPlan bridges real-time streaming with high-quality offline generation within a single model.<n>Our real-time streaming is 5.25x faster while also achieving 18% motion quality improvement over the best previous method.
  arXiv  Detail & Related papers  (2026-03-13T18:24:32Z)
• SneakPeek: Future-Guided Instructional Streaming Video Generation [27.65687424500011]
  We introduce a pipeline for future-driven streaming instructional video generation, dubbed SneakPeek.<n>Our approach introduces three key innovations to enhance consistency and controllability.<n> Experimental results demonstrate that our method produces temporally coherent and semantically faithful instructional videos.
  arXiv  Detail & Related papers  (2025-12-15T06:32:57Z)
• StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding [52.55809460075286]
  We propose a StreamAgent that anticipates the temporal intervals and spatial regions expected to contain future task-relevant information.<n>We integrate question semantics and historical observations through prompting the anticipatory agent to anticipate the temporal progression of key events.<n>Our method outperforms existing methods in response accuracy and real-time efficiency, highlighting its practical value for real-world streaming scenarios.
  arXiv  Detail & Related papers  (2025-08-03T18:15:42Z)
• RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism [73.38167494118746]
  We propose a framework to improve the realism of motion in generated videos.<n>We advocate for the incorporation of a retrieval mechanism during the generation phase.<n>Our pipeline is designed to apply to any text-to-video diffusion model.
  arXiv  Detail & Related papers  (2025-04-09T08:14:05Z)
• StreamChat: Chatting with Streaming Video [85.02875830683637]
  StreamChat is a novel approach that enhances the interaction capabilities of Large Multimodal Models with streaming video content.<n>We introduce a flexible and efficient crossattention-based architecture to process dynamic streaming inputs.<n>We construct a new dense instruction dataset to facilitate the training of streaming interaction models.
  arXiv  Detail & Related papers  (2024-12-11T18:59:54Z)
• Live2Diff: Live Stream Translation via Uni-directional Attention in Video Diffusion Models [64.2445487645478]
  Large Language Models have shown remarkable efficacy in generating streaming data such as text and audio. We present Live2Diff, the first attempt at designing a video diffusion model with uni-directional temporal attention, specifically targeting live streaming video translation.
  arXiv  Detail & Related papers  (2024-07-11T17:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.