GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion

• URL: http://arxiv.org/abs/2605.12957v1
• Date: Wed, 13 May 2026 03:43:02 GMT
• Title: GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion
• Abstract summary: We propose GTA, a novel image-to-3D world generation method following a Geometry-Then-Appearance paradigm.<n>Specifically, given a single input image, GTA adopts a two-stage framework with two dedicated video diffusion models.<n>Extensive experiments have demonstrated that our proposed method consistently outperforms existing approaches in terms of fidelity, visual quality, and geometric accuracy.
• Score: 29.999238067855245
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent developments in generative models and large-scale datasets have substantially advanced 3D world generation, facilitating a broad range of domains including spatial intelligence, embodied intelligence, and autonomous driving. While achieving remarkable progress, existing approaches to 3D world generation typically prioritize appearance prediction with limited modeling of the underlying geometry, leading to issues such as unreliable scene structure estimation and degraded cross-view consistency. To address these limitations, motivated by the coarse-to-fine nature of human visual perception, we propose GTA, a novel image-to-3D world generation method following a Geometry-Then-Appearance paradigm. Specifically, given a single input image, to improve the structural fidelity of synthesized 3D scenes, GTA adopts a two-stage framework with two dedicated video diffusion models, which first generate coarse geometric structure from novel viewpoints and then synthesize fine-grained appearance conditioned on the predicted geometry. To further enhance cross-view appearance consistency, we introduce a random latent shuffle strategy during the training process, along with a test-time scaling scheme that improves perceptual quality without compromising quantitative performance. Extensive experiments have demonstrated that our proposed method consistently outperforms existing approaches in terms of fidelity, visual quality, and geometric accuracy. Moreover, GTA is shown to be effective as a general enhancement module that further improves the generation quality of existing image-to-3D world pipelines, as well as supporting multiple downstream applications and exhibiting favorable data efficiency during model training, highlighting its versatility and broad applicability. Project page: https://hanxinzhu-lab.github.io/GTA/.

Related papers

• World-R1: Reinforcing 3D Constraints for Text-to-Video Generation [36.18760315762835]
  We propose World-R1, a framework that aligns video generation with 3D constraints through reinforcement learning.<n>We optimize the model using feedback from pre-trained 3D foundation models and vision-language models.<n>Our approach significantly enhances 3D consistency while preserving the original visual quality of the foundation model.
  arXiv  Detail & Related papers  (2026-04-27T17:59:56Z)
• Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective [91.23306722968509]
  Generalizable feed-forward 3D reconstruction has witnessed rapid development in recent years.<n>Existing feed-forward approaches share similar high-level architectural patterns.<n>We propose a novel taxonomy centered on model design strategies that are agnostic to the output format.
  arXiv  Detail & Related papers  (2026-04-15T16:07:18Z)
• Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors [61.34273238077091]
  We present a novel method for generating orbital videos from a single image of an object.<n>Our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-14T05:35:46Z)
• TAPESTRY: From Geometry to Appearance via Consistent Turntable Videos [65.99602532894241]
  We introduce TAPESTRY, a framework for generating high-fidelity TTVs conditioned on explicit 3D geometry.<n>We also design a method for downstream reconstruction tasks from the TTV input, featuring a multi-stage pipeline with 3D-Aware Inpainting.<n>Results demonstrate that our method outperforms existing approaches in both video consistency and final reconstruction quality.
  arXiv  Detail & Related papers  (2026-03-18T14:02:09Z)
• Geometry and Perception Guided Gaussians for Multiview-consistent 3D Generation from a Single Image [10.648593818811976]
  Existing approaches often rely on fine-tuning pretrained 2D diffusion models or directly generating 3D information through fast network inference.<n>We present a novel method that seamlessly integrates geometry and perception information without requiring additional model training.<n> Experimental results show that we outperform existing methods on novel view synthesis and 3D reconstruction, demonstrating robust and consistent 3D object generation.
  arXiv  Detail & Related papers  (2025-06-26T11:22:06Z)
• Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridging [15.36983068580743]
  Hi3DGen is a novel framework for generating high-fidelity 3D geometry from images via normal bridging.<n>Our work provides a new direction for high-fidelity 3D geometry generation from images by leveraging normal maps as an intermediate representation.
  arXiv  Detail & Related papers  (2025-03-28T08:39:20Z)
• TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models [69.0220314849478]
  TripoSG is a new streamlined shape diffusion paradigm capable of generating high-fidelity 3D meshes with precise correspondence to input images.<n>The resulting 3D shapes exhibit enhanced detail due to high-resolution capabilities and demonstrate exceptional fidelity to input images.<n>To foster progress and innovation in the field of 3D generation, we will make our model publicly available.
  arXiv  Detail & Related papers  (2025-02-10T16:07:54Z)
• DreamPolish: Domain Score Distillation With Progressive Geometry Generation [66.94803919328815]
  We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain.
  arXiv  Detail & Related papers  (2024-11-03T15:15:01Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.