Video = World + Event Stream

• URL: http://arxiv.org/abs/2607.15038v2
• Date: Fri, 17 Jul 2026 01:17:33 GMT
• Title: Video = World + Event Stream
• Abstract summary: Wan-Streamer v0.3 reframes our native interaction model under a single organizing: a is a world plus an event stream.<n>This yields a general-purpose pretraining task over large amounts of real video.<n> Functionally, the model's multimodal understanding process is vision--action-like.
• Score: 70.56197496608006
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a video is a world plus an event stream. The world is the persistent context in which a video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and other relatively stable conditions. The event stream is everything that changes over time within that world, including scene or environmental changes, subject behavior, speech, and other sounds. This yields a general-purpose pretraining task over large amounts of real video: given a world and incoming input, predict how the world moves, changes, and responds in real time. The resulting competence can be specialized to a broad family of real-time downstream tasks. We instantiate it on real-time full-duplex audio-visual interaction, where the event stream is the agent's speech together with free-form behavior. Functionally, the model's multimodal understanding process is vision-language-action-like: it maps multimodal user input to language-form speech and behavior actions. Wan-Streamer v0.3 preserves the v0.2 operating point: 640x368 video at 25 FPS, a 160 ms streaming unit, approximately 200 ms model-side response latency, and approximately 550 ms total interaction latency under a 350 ms bidirectional network budget.

Related papers

• Wan-Streamer v0.2: Higher Resolution, Same Latency [68.78755383327113]
  Wan-Streamer v0.2 is a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model.<n>It supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation.
  arXiv  Detail & Related papers  (2026-07-05T18:05:20Z)
• Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models [68.51256516936338]
  Wan-Streamer is an end-to-end native interactive foundation model for low-streaming interaction.<n>It seamlessly models audio and video as both input and output within single Transformer sequence.<n>It achieves approximately 200 ms model-side response latency and approximately 550 ms total interaction latency.
  arXiv  Detail & Related papers  (2026-06-23T18:01:03Z)
• EgoCS-400K: An Egocentric Gameplay Dataset for World Models [53.9389499966106]
  We introduce EgoCS-400K, a large-scale replay-grounded egocentric Counter-Strike dataset for world models.<n>It is built from public professional CS and CS2 match demos that preserve human gameplay trajectories.<n>EgoCS-400K contains over 400,000 first-person videos and 10,000 hours of gameplay from more than 1,000 matches and 40,000 rounds.
  arXiv  Detail & Related papers  (2026-06-16T17:13:58Z)
• Harnessing Streaming Video in the Wild [53.23721420272668]
  Vision-Language Models (VLMs) are increasingly required to process video streams in applications such as video-call assistants, live commentary, and embodied robots.<n>An ideal streaming system should support proactive interaction, long-horizon memory, and real-time processing.<n>ExistingVLMs excel at offline video understanding but fall short in streaming capabilities and lack dedicated infrastructure for streaming deployment.
  arXiv  Detail & Related papers  (2026-06-07T13:00:19Z)
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding [69.296913137409]
  We introduce a novel paradigm for online video understanding: Streaming Video-Language Synchrony (SVLS)<n>We present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations.<n>First, the Frame-Driven Transition Controller (FDTC), makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent.<n>Second, the Streaming Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the language generation rate to match the pace of the visual content.
  arXiv  Detail & Related papers  (2026-06-05T07:29:20Z)
• OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants [43.2907783462394]
  Streaming benchmark for realtime omnimodal language models evaluated through native inference over audio-visual streams.<n>We evaluate response correctness, timing, invalid outputs interruption, and context continuity using Interaction-Aware QualityTimeliness F1, Interruption Diagnostic Suite, and 368 1QnA slots for continuous task monitoring and step guidance.
  arXiv  Detail & Related papers  (2026-05-26T02:49:47Z)
• VideoLLM-online: Online Video Large Language Model for Streaming Video [27.073238234038826]
  We propose a novel Learning-In-Video-Stream framework, which enables temporally aligned, long-context, and real-time conversation within a continuous video stream. Our framework supports streaming dialogue in a 5-minute video clip at over 10 FPS on an A100 GPU. It also showcases state-of-the-art performance on public offline video benchmarks, such as recognition, captioning, and forecasting.
  arXiv  Detail & Related papers  (2024-06-17T17:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.