HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

• URL: http://arxiv.org/abs/2604.14268v1
• Date: Wed, 15 Apr 2026 17:59:17 GMT
• Title: HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds
• Abstract summary: HY-World 2.0 is a multi-modal world model framework that advances our prior project HYWorld 1.0.<n>We introduce key innovations to enhance panorama fidelity, enable 3D scene understanding and planning, and upgrade WorldStereo.<n>Also, we introduce WorldLens, a high-performance 3DGS rendering platform featuring a flexible engine-agnostic architecture.
• Score: 50.80574129942699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce HY-World 2.0, a multi-modal world model framework that advances our prior project HY-World 1.0. HY-World 2.0 accommodates diverse input modalities, including text prompts, single-view images, multi-view images, and videos, and produces 3D world representations. With text or single-view image inputs, the model performs world generation, synthesizing high-fidelity, navigable 3D Gaussian Splatting (3DGS) scenes. This is achieved through a four-stage method: a) Panorama Generation with HY-Pano 2.0, b) Trajectory Planning with WorldNav, c) World Expansion with WorldStereo 2.0, and d) World Composition with WorldMirror 2.0. Specifically, we introduce key innovations to enhance panorama fidelity, enable 3D scene understanding and planning, and upgrade WorldStereo, our keyframe-based view generation model with consistent memory. We also upgrade WorldMirror, a feed-forward model for universal 3D prediction, by refining model architecture and learning strategy, enabling world reconstruction from multi-view images or videos. Also, we introduce WorldLens, a high-performance 3DGS rendering platform featuring a flexible engine-agnostic architecture, automatic IBL lighting, efficient collision detection, and training-rendering co-design, enabling interactive exploration of 3D worlds with character support. Extensive experiments demonstrate that HY-World 2.0 achieves state-of-the-art performance on several benchmarks among open-source approaches, delivering results comparable to the closed-source model Marble. We release all model weights, code, and technical details to facilitate reproducibility and support further research on 3D world models.

Related papers

• Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States [106.40454498751295]
  Puffin-World integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules.<n>Our framework jointly models three native world states: physics (gravity field and latitude), geometry (depth), and appearance (image)<n>By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation.
  arXiv  Detail & Related papers  (2026-09-03T17:59:13Z)
• ABot-3DWorld 0: A Universal World Model to Explore Any 3D Space [49.372892542771886]
  ABot-3DWorld 0 is a universal multimodal 3D world model that turns text, image, and video inputs into high-fidelity, explorable 3D worlds.
  arXiv  Detail & Related papers  (2026-07-13T15:14:48Z)
• Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving [51.90209659403234]
  This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving.<n>For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries.<n>For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning.
  arXiv  Detail & Related papers  (2026-05-18T09:46:16Z)
• WorldAgents: Can Foundation Image Models be Agents for 3D World Models? [82.83725150353915]
  We demonstrate that 2D models do indeed encapsulate a grasp of 3D worlds.<n>By exploiting this understanding, our method successfully synthesizes expansive, realistic, and 3D-consistent worlds.
  arXiv  Detail & Related papers  (2026-03-20T07:22:41Z)
• WorldGen: From Text to Traversable and Interactive 3D Worlds [87.95088818329403]
  We introduce WorldGen, a system that enables the automatic creation of large-scale, interactive 3D worlds directly from text prompts.<n>Our approach transforms natural language descriptions into fully textured environments that can be immediately explored or edited within standard game engines.<n>This work represents a step towards accessible, generative world-building at scale, advancing the frontier of 3D generative AI for applications in gaming, simulation, and immersive social environments.
  arXiv  Detail & Related papers  (2025-11-20T22:13:18Z)
• Terra: Explorable Native 3D World Model with Point Latents [74.90179419859415]
  We present Terra, a native 3D world model that represents and generates explorable environments in an intrinsic 3D latent space.<n>Specifically, we propose a novel point-to-Gaussian variational autoencoder (P2G-VAE) that encodes 3D inputs into a latent point representation.<n>We then introduce a sparse point flow matching network (SPFlow) for generating the latent point representation, which simultaneously denoises the positions and features of the point latents.
  arXiv  Detail & Related papers  (2025-10-16T17:59:56Z)
• LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation [35.4193352348583]
  We propose a simple yet effective 3D world generation framework that streamlines the industrial production pipeline of 3D environments.<n>LatticeWorld creates large-scale 3D interactive worlds with dynamic agents, featuring competitive multi-agent interaction.<n>LatticeWorld achieves over a $90times$ increase in industrial production efficiency.
  arXiv  Detail & Related papers  (2025-09-05T17:22:33Z)
• HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels [30.986527559921335]
  HunyuanWorld 1.0 is a novel framework that combines the best of both worlds for generating immersive, explorable, and interactive 3D scenes from text and image conditions.<n>Our approach features three key advantages: 1) 360deg immersive experiences via panoramic world proxies; 2) mesh export capabilities for seamless compatibility with existing computer graphics pipelines; 3) disentangled object representations for augmented interactivity.
  arXiv  Detail & Related papers  (2025-07-29T13:43:35Z)
• FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesis [61.78754615822363]
  We introduce FlexWorld, a framework for generating flexible-view 3D scenes from single images.<n>Our V2V model can generate novel views under large camera pose variations.<n>Building upon it, FlexWorld progressively generates new 3D content and integrates it into the global scene.
  arXiv  Detail & Related papers  (2025-03-17T15:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.