EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant

• URL: http://arxiv.org/abs/2605.10343v1
• Date: Mon, 11 May 2026 10:43:59 GMT
• Title: EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
• Abstract summary: EvoStreaming is a self-evolved streaming adaptation framework.<n>We show that EvoStreaming consistently improves the overall RealStreamEval score by up to $10.8$ points across five open VideoLLM backbones.<n>These results suggest that data-efficient interaction tuning is a practical path for adapting existing VideoLLMs to streaming assistants.
• Score: 14.309187098312115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Streaming video understanding demands more than watching longer videos: assistants must decide when to speak in real time, balancing responsiveness against verbosity. Yet most video-language models (VideoLLMs) are trained for offline inference, and existing streaming benchmarks externalize this timing decision to the evaluator. We address this gap with RealStreamEval, a frame-level multi-turn evaluation protocol that exposes models to sequential observations and penalizes unnecessary responses. Under this protocol, we observed that strong offline VideoLLMs retain useful visual understanding but lack an interaction policy for deciding when to respond. Motivated by this observation, we propose EvoStreaming, a self-evolved streaming adaptation framework in which the base model itself acts as data generator, relevance annotator, and roll-out policy to synthesize streaming trajectories without external supervision. With only $1{,}000$ self-generated samples ($139\times$ less than the leading streaming instruction-tuning approach) and no architectural changes, EvoStreaming consistently improves the overall RealStreamEval score by up to $10.8$ points across five open VideoLLM backbones (Qwen2/2.5/3-VL, InternVL-3.5, MiniCPM-V4.5) while largely preserving offline video performance. These results suggest that data-efficient interaction tuning is a practical path for adapting existing VideoLLMs to streaming assistants.

Related papers

• Harnessing Streaming Video in the Wild [53.23721420272668]
  Vision-Language Models (VLMs) are increasingly required to process video streams in applications such as video-call assistants, live commentary, and embodied robots.<n>An ideal streaming system should support proactive interaction, long-horizon memory, and real-time processing.<n>ExistingVLMs excel at offline video understanding but fall short in streaming capabilities and lack dedicated infrastructure for streaming deployment.
  arXiv  Detail & Related papers  (2026-06-07T13:00:19Z)
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding [69.296913137409]
  We introduce a novel paradigm for online video understanding: Streaming Video-Language Synchrony (SVLS)<n>We present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations.<n>First, the Frame-Driven Transition Controller (FDTC), makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent.<n>Second, the Streaming Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the language generation rate to match the pace of the visual content.
  arXiv  Detail & Related papers  (2026-06-05T07:29:20Z)
• FlowNar: Scalable Streaming Narration for Long-Form Videos [24.674831574110645]
  FlowNar is a novel framework for scalable streaming video narration.<n>The core of FlowNar is a dynamic context management strategy for historical visual context removal.<n> Experiments on the Ego4D, EgoExo4D, and EpicKitchens100 datasets demonstrate that FlowNar substantially improves narration quality over strong baselines.
  arXiv  Detail & Related papers  (2026-05-30T08:51:28Z)
• VSAS-BENCH: Real-Time Evaluation of Visual Streaming Assistant Models [39.78979236902648]
  Streaming vision-language models (VLMs) continuously generate responses given an instruction prompt and an online stream of input frames.<n>We propose VSAS-Bench, a new framework and benchmark for Visual Streaming Assistants.
  arXiv  Detail & Related papers  (2026-04-08T22:31:20Z)
• Thinking in Streaming Video [30.61790766076081]
  ThinkStream is a framework for streaming video reasoning based on a Watch--Think--Speak paradigm.<n> Reasoning-Compressed Streaming Memory (RCSM) treats intermediate reasoning traces as compact semantic memory.<n>Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models.
  arXiv  Detail & Related papers  (2026-03-13T12:33:36Z)
• Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously [69.0264594684213]
  Video Streaming Thinking (VST) is a novel paradigm for streaming video understanding.<n>It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming.<n>VST improves timely comprehension and coherent cognition while preserving real-time responsiveness.
  arXiv  Detail & Related papers  (2026-03-12T17:59:51Z)
• LiveStar: Live Streaming Assistant for Real-World Online Video Understanding [67.71551356747948]
  LiveStar is a pioneering live streaming assistant that achieves always-on proactive responses through adaptive streaming decoding.<n>LiveStar incorporates: (1) a training strategy enabling incremental video-language alignment for variable-length video streams, preserving temporal consistency across dynamically evolving frame sequences; (2) a response-silence decoding framework that determines optimal proactive response timing via a single forward pass verification; and (3) memory-aware acceleration via peak-end memory compression for online inference on 10+ minute videos, combined with streaming key-value cache to achieve 1.53x faster inference.
  arXiv  Detail & Related papers  (2025-11-07T15:00:37Z)
• StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant [27.21599940736722]
  We present StreamBridge, a framework that seamlessly transforms offline Video-LLMs into streaming-capable models.<n>It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms.
  arXiv  Detail & Related papers  (2025-05-08T17:57:40Z)
• StreamChat: Chatting with Streaming Video [85.02875830683637]
  StreamChat is a novel approach that enhances the interaction capabilities of Large Multimodal Models with streaming video content.<n>We introduce a flexible and efficient crossattention-based architecture to process dynamic streaming inputs.<n>We construct a new dense instruction dataset to facilitate the training of streaming interaction models.
  arXiv  Detail & Related papers  (2024-12-11T18:59:54Z)
• VideoLLM-online: Online Video Large Language Model for Streaming Video [27.073238234038826]
  We propose a novel Learning-In-Video-Stream framework, which enables temporally aligned, long-context, and real-time conversation within a continuous video stream. Our framework supports streaming dialogue in a 5-minute video clip at over 10 FPS on an A100 GPU. It also showcases state-of-the-art performance on public offline video benchmarks, such as recognition, captioning, and forecasting.
  arXiv  Detail & Related papers  (2024-06-17T17:55:32Z)
• Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams [78.72965584414368]
  We present Flash-VStream, a video-language model that simulates the memory mechanism of human. Compared to existing models, Flash-VStream achieves significant reductions in latency inference and VRAM consumption. We propose VStream-QA, a novel question answering benchmark specifically designed for online video streaming understanding.
  arXiv  Detail & Related papers  (2024-06-12T11:07:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.