Wan-Streamer v0.2: Higher Resolution, Same Latency

• URL: http://arxiv.org/abs/2607.04443v3
• Date: Wed, 08 Jul 2026 11:37:37 GMT
• Title: Wan-Streamer v0.2: Higher Resolution, Same Latency
• Abstract summary: Wan-Streamer v0.2 is a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model.<n>It supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation.
• Score: 68.78755383327113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

Related papers

• Aero Realtime: Fully Aligned Input-Output Streams for Low-Latency Streaming Multimodal Generation [33.4073718746898]
  We introduce Aero Realtime, which aligns video, audio and textual output on a shared temporal grid for realtime generation.<n>On four NVIDIA A6000 workstations, Aero Realtime maintains 84-ms median and 173-ms P95 processing lag over 20 minutes of a continuously streamed video.<n>Results demonstrate the feasibility of fully input-output modeling for duplex proactive and hardware-aligned multimodal interaction.
  arXiv  Detail & Related papers  (2026-08-09T04:22:57Z)
• StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding [42.88155526332792]
  We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding.<n> evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension.<n>We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled workers.
  arXiv  Detail & Related papers  (2026-08-06T07:46:37Z)
• Video = World + Event Stream [70.56197496608006]
  Wan-Streamer v0.3 reframes our native interaction model under a single organizing: a is a world plus an event stream.<n>This yields a general-purpose pretraining task over large amounts of real video.<n> Functionally, the model's multimodal understanding process is vision--action-like.
  arXiv  Detail & Related papers  (2026-07-16T14:19:42Z)
• Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models [68.51256516936338]
  Wan-Streamer is an end-to-end native interactive foundation model for low-streaming interaction.<n>It seamlessly models audio and video as both input and output within single Transformer sequence.<n>It achieves approximately 200 ms model-side response latency and approximately 550 ms total interaction latency.
  arXiv  Detail & Related papers  (2026-06-23T18:01:03Z)
• ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference [28.470202983949886]
  ViCoStream is a stage-wise streaming framework that combines chunk-wise execution, visual token control, bounded visual attention, query-side retrieval, and memory costs.<n>We show that ViCoStream achieves 134 FPS video throughput and less than 50 ms TTFT on a single A100 GPU while maintaining accuracy close to full-ordinated baselines.
  arXiv  Detail & Related papers  (2026-06-18T06:57:31Z)
• Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation [59.178597408747095]
  We present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video.<n>Conventional pipelines capture frames, generate text, and synthesize speech sequentially for each utterance, and do not request the next generation until speech playback has completed.<n>Our system runs text generation in parallel with speech playback and buffers multiple candidate utterances ahead of time, enabling immediate synthesis at playback boundaries.
  arXiv  Detail & Related papers  (2026-06-11T13:15:13Z)
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding [69.296913137409]
  We introduce a novel paradigm for online video understanding: Streaming Video-Language Synchrony (SVLS)<n>We present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations.<n>First, the Frame-Driven Transition Controller (FDTC), makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent.<n>Second, the Streaming Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the language generation rate to match the pace of the visual content.
  arXiv  Detail & Related papers  (2026-06-05T07:29:20Z)
• Small Vision-Language Models are Smart Compressors for Long Video Understanding [73.65465038390771]
  Long-form video understanding relies on intent-driven efficiency rather than greedily padded context.<n>We propose Tempo, an efficient query-aware framework compressing long videos for downstream understanding.<n>Tests show Tempo compresses hour-long videos substantially below theoretical limits, proving true long-form video understanding relies on intent-driven efficiency.
  arXiv  Detail & Related papers  (2026-04-09T11:40:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.