StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

• URL: http://arxiv.org/abs/2608.05703v1
• Date: Thu, 06 Aug 2026 07:46:37 GMT
• Title: StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding
• Abstract summary: We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding.<n> evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension.<n>We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled workers.
• Score: 42.88155526332792
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms existing streaming baselines across all four capabilities and reduces query-to-answer latency by reusing persistent state.

Related papers

• LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams [59.485485426790966]
  We introduce LiveStarPro, a live streaming assistant that is designed for proactive video understanding over long-horizon streams.<n>LiveStarPro consistently surpasses existing methods, attaining a 28.9% improvement in semantic correctness.<n>Its streaming key-value cache further yields a 1.58x inference speedup over the same model without caching.
  arXiv  Detail & Related papers  (2026-06-16T11:18:05Z)
• Harnessing Streaming Video in the Wild [53.23721420272668]
  Vision-Language Models (VLMs) are increasingly required to process video streams in applications such as video-call assistants, live commentary, and embodied robots.<n>An ideal streaming system should support proactive interaction, long-horizon memory, and real-time processing.<n>ExistingVLMs excel at offline video understanding but fall short in streaming capabilities and lack dedicated infrastructure for streaming deployment.
  arXiv  Detail & Related papers  (2026-06-07T13:00:19Z)
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding [69.296913137409]
  We introduce a novel paradigm for online video understanding: Streaming Video-Language Synchrony (SVLS)<n>We present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations.<n>First, the Frame-Driven Transition Controller (FDTC), makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent.<n>Second, the Streaming Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the language generation rate to match the pace of the visual content.
  arXiv  Detail & Related papers  (2026-06-05T07:29:20Z)
• StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering [39.92453666681465]
  StreamOV is a novel Streaming Omni-Video understanding framework for efficient online audio-visual reasoning with bounded memory and proactive response triggering.<n>It employs a hidden-state-driven trigger to decide when to respond, avoiding explicit silence-token generation and external routers.<n>It achieves state-of-the-art performance across diverse streaming and omni-video benchmarks, demonstrating its effectiveness for both online and offline video understanding.
  arXiv  Detail & Related papers  (2026-05-25T09:23:19Z)
• Advancing Narrative Long Video Generation via Training-Free Identity-Aware Memory [79.01059178883817]
  IAMFlow is a training-free identity-aware memory framework that explicitly models and tracks persistent entity identities.<n>A VLM asynchronously verifies and refines attributes from rendered frames, enabling explicit entity tracking in place of implicit similarity-based matching.<n>NarraStream-Bench is a benchmark for narrative streaming video generation that features 324 multi-prompt scripts spanning six dimensions.
  arXiv  Detail & Related papers  (2026-05-18T17:54:34Z)
• AURA: Always-On Understanding and Real-Time Assistance via Video Streams [37.80002196035961]
  AURA (Always-On Understanding and Real-Time Assistance) is an end-to-end streaming visual interaction framework.<n>It enables a unified VideoLLM to continuously process video streams and support both real-time question answering and proactive responses.<n>AURA achieves state-of-the-art performance on streaming benchmarks and supports a real-time demo system with ASR and TTS running at 2 FPS on two 80G accelerators.
  arXiv  Detail & Related papers  (2026-04-05T16:53:46Z)
• Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models [25.48403877647779]
  Think While Watching is a memory-anchored streaming video reasoning framework.<n>It preserves continuous segment-level memory during multi-turn interaction.<n>It improves single-round accuracy by 2.6% on StreamingBench and by 3.79% on OVO-Bench.
  arXiv  Detail & Related papers  (2026-03-12T13:13:50Z)
• InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions [104.90258030688256]
  This project introduces disentangled streaming perception, reasoning, and memory mechanisms, enabling real-time interaction with streaming video and audio input.<n>This project simulates human-like cognition, enabling multimodal large language models to provide continuous and adaptive service over time.
  arXiv  Detail & Related papers  (2024-12-12T18:58:30Z)
• Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams [78.72965584414368]
  We present Flash-VStream, a video-language model that simulates the memory mechanism of human. Compared to existing models, Flash-VStream achieves significant reductions in latency inference and VRAM consumption. We propose VStream-QA, a novel question answering benchmark specifically designed for online video streaming understanding.
  arXiv  Detail & Related papers  (2024-06-12T11:07:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.