Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Visual Avatar Generation

• URL: http://arxiv.org/abs/2608.13602v1
• Date: Fri, 07 Aug 2026 15:34:15 GMT
• Title: Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Visual Avatar Generation
• Abstract summary: We present Omni-LiveAvatar, the first framework for minute-level, real-time streaming joint audio-video avatar generation.<n>Experiments show that Omni-LiveAvatar generates high-quality, synchronized minute-level avatars in real time.
• Score: 26.625772912688387
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint audio-video generative models serve as foundation for immersive and interactive digital-human generation. Nevertheless, most existing models rely on bidirectional attention and multi-step denoising and can generate only short clips, making them unsuitable for real-time interaction over extended durations. We present Omni-LiveAvatar, the first framework for minute-level, real-time streaming joint audio-video avatar generation. Specifically, we propose (1) a progressive autoregressive distillation pipeline that transfers a large bidirectional joint audio-video diffusion model into a few-step autoregressive generator without auxiliary stabilization mechanisms; (2) a synchronized audio-video long-short-term memory that preserves global consistency under a bounded memory budget; and (3) a hierarchical rolling prompt planning strategy that enables coherent semantic evolution and seamless prompt transitions. Extensive experiments show that Omni-LiveAvatar generates high-quality, synchronized minute-level avatars in real time. In terms of speed, it achieves a 33$\times$ generation speedup over its teacher, LTX-2, on a single NVIDIA H200 GPU; in terms of generation quality, it outperforms accelerated baselines across visual quality, audio quality, cross-modal synchronization, and human fidelity. Our code is available at https://github.com/Aoko955/Omni-LiveAvatar.

Related papers

• Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds [60.18437892555415]
  We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants.<n>The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio.<n>The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway.
  arXiv  Detail & Related papers  (2026-08-24T15:31:12Z)
• LiveAnimate: Stable Long-Form Streaming Human Animation in Real-Time [26.186924581106087]
  We present the first animation system to combine real-time streaming with stable long-form generation at billion scale.<n>LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute.<n>These results establish a new operating point in quality, latency, and duration for interactive full-body animation.
  arXiv  Detail & Related papers  (2026-08-12T07:35:52Z)
• Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation [29.637920236800394]
  We present Hallo-Live, a streaming framework for joint audio-visual avatar generation.<n>Future-Expanding Attention allows each video block to access synchronous audio together with a short horizon of future phonetic cues.<n>HP-DMD reweights training samples using rewards from visual fidelity, speech naturalness, and audio-visual synchronization.
  arXiv  Detail & Related papers  (2026-04-26T09:46:58Z)
• Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model [52.79436545460808]
  We present daVinci-MagiHuman, an open-source audio-video generative foundation model for human-centric generation.<n>DaVinci-MagiHuman jointly generates synchronized video and audio using a single-stream Transformer.<n>It supports multilingual spoken generation across Chinese (Mandarin and Cantonese), English, Japanese, Korean, German, and French.
  arXiv  Detail & Related papers  (2026-03-23T13:49:06Z)
• ALIVE: Animate Your World with Lifelike Audio-Video Generation [50.693986608051716]
  ALIVE is a generation model that adapts a pretrained Text-to-Video (T2V) model to Sora-style audio-video generation and animation.<n>To support the audio-visual synchronization and reference animation, we augment the popular MMDiT architecture with a joint audio-video branch.<n>ALIVE demonstrates outstanding performance, consistently outperforming open-source models and matching or surpassing state-of-the-art commercial solutions.
  arXiv  Detail & Related papers  (2026-02-09T14:06:03Z)
• Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length [57.458450695137664]
  We present Live Avatar, an algorithm-system co-designed framework for efficient, high-fidelity, and infinite-length avatar generation.<n>Live Avatar is first to achieve practical, real-time, high-fidelity avatar generation at this scale.
  arXiv  Detail & Related papers  (2025-12-04T11:11:24Z)
• SoundReactor: Frame-level Online Video-to-Audio Generation [39.113214321291586]
  Video-to-Audio generation models operate offline, assuming an entire video sequence or chunks of frames are available beforehand.<n>We introduce the novel task of frame-level online V2A generation, where a model autoregressively generates audio from video without access to future video frames.<n>SoundReactor is the first simple yet effective framework explicitly tailored for this task.
  arXiv  Detail & Related papers  (2025-10-02T15:18:00Z)
• StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation [91.45910771331741]
  Current diffusion models for audio-driven avatar video generation struggle to synthesize long videos with natural audio synchronization and identity consistency.<n>This paper presents StableAvatar, the first end-to-end video diffusion transformer that synthesizes infinite-length high-quality videos without post-processing.
  arXiv  Detail & Related papers  (2025-08-11T17:58:24Z)
• DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation [50.66658181705527]
  We present DAWN, a framework that enables all-at-once generation of dynamic-length video sequences.<n>DAWN consists of two main components: (1) audio-driven holistic facial dynamics generation in the latent motion space, and (2) audio-driven head pose and blink generation.<n>Our method generates authentic and vivid videos with precise lip motions, and natural pose/blink movements.
  arXiv  Detail & Related papers  (2024-10-17T16:32:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.