Related papers: EchoWM: Open and Enterable Omnimodal World Models

EchoWM: Open and Enterable Omnimodal World Models

URL: http://arxiv.org/abs/2608.23189v1
Date: Mon, 24 Aug 2026 12:39:59 GMT
Title: EchoWM: Open and Enterable Omnimodal World Models
Abstract summary: EchoWM is an omnimodal world model for enterable generative media.<n>It responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech.
Score: 71.49148067556887
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech. We organize interaction around camera intent: in first-person scenes, it specifies observer motion, while in third-person scenes, camera--character dynamics are learned from data without view-specific controllers. Discrete commands and continuous poses are mapped to a shared metric-scale relative 6-DoF trajectory, with dataset-level calibration preserving motion magnitude across heterogeneous data. To jointly learn audio-visual generation and trajectory control, we construct a complementary data engine and adopt progressive training followed by autoregressive post-training for long-horizon generation. Extensive evaluations show that \model achieves strong trajectory following and high visual quality on public world-model benchmarks, supporting both first- and third-person interaction across varied subjects, and maintaining synchronized environmental sound and speech over long-horizon generation.

Related papers

Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds [60.18437892555415]
We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants.<n>The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio.<n>The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway.
arXiv Detail & Related papers (2026-08-24T15:31:12Z)
WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory [106.28234880163723]
WorldDirector is a highly controllable video world model framework.<n>Our framework explicitly decouples semantic motion orchestration from visual generation.<n>We successfully preserve the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
arXiv Detail & Related papers (2026-07-02T17:59:59Z)
MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
arXiv Detail & Related papers (2026-06-01T18:20:20Z)
Causal World Modeling for Robot Control [56.31803788587547]
Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
arXiv Detail & Related papers (2026-01-29T17:07:43Z)
Astra: General Interactive World Model with Autoregressive Denoising [73.6594791733982]
Astra is an interactive general world model that generates real-world futures for diverse scenarios.<n>We propose an autoregressive denoising architecture and use temporal causal attention to aggregate past observations.<n>Astra achieves interactive, consistent, and general long-term video prediction and supports various forms of interactions.
arXiv Detail & Related papers (2025-12-09T18:59:57Z)
Temporally Consistent Dynamic Scene Graphs: An End-to-End Approach for Action Tracklet Generation [1.6584112749108326]
TCDSG, Temporally Consistent Dynamic Scene Graphs, is an end-to-end framework that detects, tracks, and links subject-object relationships across time.<n>Our work sets a new standard in multi-frame video analysis, opening new avenues for high-impact applications in surveillance, autonomous navigation, and beyond.
arXiv Detail & Related papers (2024-12-03T20:19:20Z)
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation [16.033455552126348]
We propose a multi-scale audio-visual synchrony loss and a multi-scale autoregressive GAN.<n>We train a stack of syncer models on multimodal input pyramids and use these models as guidance in a multi-scale generator network.<n>Experiments show significant improvements over the state-of-the-art in head motion dynamics quality.
arXiv Detail & Related papers (2023-07-04T08:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.