OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder

• URL: http://arxiv.org/abs/2603.16099v1
• Date: Tue, 17 Mar 2026 03:43:37 GMT
• Title: OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder
• Abstract summary: We present OneWorld, a framework that performs diffusion directly within a coherent 3D representation space.<n>OneWorld generates high-quality 3D scenes with superior cross-view consistency compared to state-of-the-art 2D-based methods.
• Score: 90.8453349494245
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing diffusion-based 3D scene generation methods primarily operate in 2D image/video latent spaces, which makes maintaining cross-view appearance and geometric consistency inherently challenging. To bridge this gap, we present OneWorld, a framework that performs diffusion directly within a coherent 3D representation space. Central to our approach is the 3D Unified Representation Autoencoder (3D-URAE); it leverages pretrained 3D foundation models and augments their geometry-centric nature by injecting appearance and distilling semantics into a unified 3D latent space. Furthermore, we introduce token-level Cross-View-Correspondence (CVC) consistency loss to explicitly enforce structural alignment across views, and propose Manifold-Drift Forcing (MDF) to mitigate train-inference exposure bias and shape a robust 3D manifold by mixing drifted and original representations. Comprehensive experiments demonstrate that OneWorld generates high-quality 3D scenes with superior cross-view consistency compared to state-of-the-art 2D-based methods. Our code will be available at https://github.com/SensenGao/OneWorld.

Related papers

• PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space [25.621179726094056]
  3D reconstruction and generation are commonly tackled by separate paradigms.<n>Recent works attempt to unify them in latent space, but with notable drawbacks.<n>We introduce PixWorld, a single model that jointly addresses 3D reconstruction and generation.
  arXiv  Detail & Related papers  (2026-07-06T17:51:00Z)
• PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation [51.385664546670284]
  We present PAIWorld, a framework that augments diffusion-transformer world models via three core components.<n>It achieves state-of-the-art multi-view 3D consistency on robotic benchmarks, ranking 1st on the WorldArena leaderboard and 2nd on the AgiBot-Challenge2026 leaderboard.
  arXiv  Detail & Related papers  (2026-06-16T18:23:23Z)
• Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence [28.712695548782037]
  We present Tango3D, a foundation model that unifies dense correspondence and global retrieval.<n>We use a geometry-aware 2D visual backbone and a pretrained 3D VAE to encode images into 2D patches and point clouds into 3D tokens.<n>Experiments show our model successfully achieves object-level pixel-to-point alignment while maintaining competitive global retrieval.
  arXiv  Detail & Related papers  (2026-05-19T12:01:23Z)
• Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale [47.551405833477986]
  3D scene generation has long been dominated by 2D multi-view or video diffusion models.<n>In this paper, we propose to perform 3D scene generation directly within an implicit 3D latent space.
  arXiv  Detail & Related papers  (2026-04-13T11:32:36Z)
• 3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image [26.04490259188974]
  We introduce 3D-Fixer, a novel in-place completion paradigm for compositional 3D scene generation.<n>Unlike prior works that require explicit pose alignment, 3D-Fixer uses fragmented geometry as a spatial anchor to preserve layout fidelity.<n>We present ARSG-110K, the largest scene-level dataset to date, comprising over 110K diverse scenes and 3M annotated images with high-fidelity 3D ground truth.
  arXiv  Detail & Related papers  (2026-04-06T04:11:09Z)
• 3D-IDE: 3D Implicit Depth Emergent [45.72771473431863]
  3D-Implicit Depth Emergence is a method that reframes 3D perception as an emergent property derived from geometric self-vision.<n>Our method enables 3D perception to emerge implicitly, disentangling features in dense regions.<n>Our approach achieves a 55% reduction in inference latency while maintaining strong performance across diverse downstream tasks.
  arXiv  Detail & Related papers  (2026-03-28T00:54:19Z)
• WorldGrow: Generating Infinite 3D World [75.81531067447203]
  We tackle the challenge of generating the infinitely extendable 3D world -- large, continuous environments with coherent geometry and realistic appearance.<n>We propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis.<n>Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity.
  arXiv  Detail & Related papers  (2025-10-24T17:39:52Z)
• Constructing a 3D Scene from a Single Image [31.11317559252235]
  SceneFuse-3D is a training-free framework designed to synthesize coherent 3D scenes from a single top-down view.<n>We decompose the input image into overlapping regions and generate each using a pretrained 3D object generator.<n>This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning.
  arXiv  Detail & Related papers  (2025-05-21T17:10:47Z)
• GSV3D: Gaussian Splatting-based Geometric Distillation with Stable Video Diffusion for Single-Image 3D Object Generation [24.255633621887988]
  We propose a method that leverages 2D diffusion models' implicit 3D reasoning ability while ensuring 3D consistency.<n>Specifically, the proposed Gaussian Splatting Decoder enforces 3D consistency by transforming SV3D latent outputs into an explicit 3D representation.<n>As a result, our approach simultaneously generates high-quality, multi-view-consistent images and accurate 3D models.
  arXiv  Detail & Related papers  (2025-03-08T09:10:31Z)
• Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior [57.986512832738704]
  We present a new framework Sculpt3D that equips the current pipeline with explicit injection of 3D priors from retrieved reference objects without re-training the 2D diffusion model. Specifically, we demonstrate that high-quality and diverse 3D geometry can be guaranteed by keypoints supervision through a sparse ray sampling approach. These two decoupled designs effectively harness 3D information from reference objects to generate 3D objects while preserving the generation quality of the 2D diffusion model.
  arXiv  Detail & Related papers  (2024-03-14T07:39:59Z)
• Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Prior [52.44678180286886]
  2D diffusion models find a distillation approach that achieves excellent generalization and rich details without any 3D data. We propose Sherpa3D, a new text-to-3D framework that achieves high-fidelity, generalizability, and geometric consistency simultaneously.
  arXiv  Detail & Related papers  (2023-12-11T18:59:18Z)
• RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation [68.06991943974195]
  We present RenderDiffusion, the first diffusion model for 3D generation and inference, trained using only monocular 2D supervision. We evaluate RenderDiffusion on FFHQ, AFHQ, ShapeNet and CLEVR datasets, showing competitive performance for generation of 3D scenes and inference of 3D scenes from 2D images.
  arXiv  Detail & Related papers  (2022-11-17T20:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.