World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible

• URL: http://arxiv.org/abs/2606.13652v1
• Date: Thu, 11 Jun 2026 17:52:48 GMT
• Title: World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible
• Abstract summary: We introduce World Tracing, a generative pixel-aligned geometry representation that predicts 3D points aligned with observed pixels.<n>For each input pixel, World Tracing predicts an ordered stack of camera-space 3D points, where the first layer represents the visible surface.<n>We instantiate this representation with a world-tracing diffusion transformer, WT-DiT, which treats multiple geometry layers as separate denoising tokens.
• Score: 34.47550981952829
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Image-to-3D methods often trade off faithfulness and completeness: depth estimators are anchored to input pixels but stop at the visible surface, while image-to-3D models generate complete shapes that are often misaligned with the input. We introduce World Tracing, a generative pixel-aligned geometry representation that predicts 3D points aligned with observed pixels while completing geometry beyond the visible surface. For each input pixel, World Tracing predicts an ordered stack of camera-space 3D points, where the first layer represents the visible surface and subsequent layers represent front-to-back intersections with occluded surfaces. We instantiate this representation with a world-tracing diffusion transformer, WT-DiT, which treats multiple geometry layers as separate denoising tokens coupled through factorized and global attention. WT-DiT is trained with pixel-space flow matching and a mixed noise schedule that balances visible-surface reconstruction with occluded-geometry generation. World Tracing achieves strong performance on visible-surface reconstruction and complete geometry generation across object, scene, and dynamic benchmarks, outperforming both depth predictors and image-to-3D generators. It also preserves 2D-to-3D correspondence, enabling text-driven 3D scene editing, geometry-conditioned novel-view video synthesis, and training-free integration with textured-mesh generators.

Related papers

• Surfsvr: 2D Surface Priors as 3D Geometric Regularizers for Sparse Voxel Reconstruction [56.51413081953813]
  SurfSVR is a novel sparse voxel reconstruction paradigm that treats 2D surface priors as explicit 3D geometric regularizers.<n>It consistently improves sparse voxel reconstruction across scenes with substantially different visibility and geometry characteristics.
  arXiv  Detail & Related papers  (2026-08-12T11:25:31Z)
• PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space [25.621179726094056]
  3D reconstruction and generation are commonly tackled by separate paradigms.<n>Recent works attempt to unify them in latent space, but with notable drawbacks.<n>We introduce PixWorld, a single model that jointly addresses 3D reconstruction and generation.
  arXiv  Detail & Related papers  (2026-07-06T17:51:00Z)
• GeoFace: Consistent Multi-View Face Generation with Geometry-Constrained Diffusion [36.691978700341515]
  GeoFace is a geometry-constrained multi-view diffusion framework for consistent face generation from a single input.<n>Experiments on RenderMe-360 and NeRSemble demonstrate that GeoFace consistently outperforms existing methods in both visual quality and cross-view geometric consistency.
  arXiv  Detail & Related papers  (2026-06-26T02:31:06Z)
• IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation [76.36174247570716]
  We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images.<n> IVGT learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions.<n>It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2026-05-15T17:59:57Z)
• Interact3D: Compositional 3D Generation of Interactive Objects [31.12099147294145]
  We present a novel framework designed to generate physically plausible interacting 3D compositional objects.<n>Our approach first leverages advanced generative priors to curate high-quality individual assets.<n>To physically compose these assets, we then introduce a robust two-stage composition pipeline.
  arXiv  Detail & Related papers  (2026-03-17T03:21:06Z)
• Constructing a 3D Scene from a Single Image [31.11317559252235]
  SceneFuse-3D is a training-free framework designed to synthesize coherent 3D scenes from a single top-down view.<n>We decompose the input image into overlapping regions and generate each using a pretrained 3D object generator.<n>This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning.
  arXiv  Detail & Related papers  (2025-05-21T17:10:47Z)
• Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation [27.43973967994717]
  MT3D is a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias.<n>By incorporating geometric details from a 3D asset, MT3D enables the creation of diverse and geometrically consistent objects.
  arXiv  Detail & Related papers  (2024-08-12T06:25:44Z)
• NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation [52.772319840580074]
  3D shape generation aims to produce innovative 3D content adhering to specific conditions and constraints. Existing methods often decompose 3D shapes into a sequence of localized components, treating each element in isolation. We introduce a novel spatial-aware 3D shape generation framework that leverages 2D plane representations for enhanced 3D shape modeling.
  arXiv  Detail & Related papers  (2024-03-27T04:09:34Z)
• Flattening-Net: Deep Regular 2D Representation for 3D Point Cloud Analysis [66.49788145564004]
  We present an unsupervised deep neural architecture called Flattening-Net to represent irregular 3D point clouds of arbitrary geometry and topology. Our methods perform favorably against the current state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-12-17T15:05:25Z)
• Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape [77.95154911528365]
  3D Morphable Model (3DMM) fitting has widely benefited face analysis due to its strong 3D priori. Previous reconstructed 3D faces suffer from degraded visual verisimilitude due to the loss of fine-grained geometry. This paper proposes a complete solution to capture the personalized shape so that the reconstructed shape looks identical to the corresponding person.
  arXiv  Detail & Related papers  (2022-04-09T03:46:18Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.