PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation

• URL: http://arxiv.org/abs/2607.02515v1
• Date: Thu, 02 Jul 2026 17:59:56 GMT
• Title: PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation
• Abstract summary: State-of-the-art single-image 3D reconstruction methods often rely on complex hybrid architectures and loss functions.<n>We show that such architectural overhead and intricate loss formulations are unnecessary.<n>We introduce a minimalist pixel-space Diffusion Transformer, built on a plain ViT, that operates directly on raw 3D point map patches.
• Score: 105.86385617182879
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: State-of-the-art single-image 3D reconstruction methods often rely on complex hybrid architectures and loss functions, or compress geometry into latent spaces in order to leverage pre-trained latent diffusion models. In this work, we show that such architectural overhead and intricate loss formulations are unnecessary. We introduce a minimalist pixel-space Diffusion Transformer, built on a plain ViT, that operates directly on raw 3D point map patches and is conditioned on image tokens from a pre-trained DINOv3. Unlike existing latent diffusion approaches, we train our diffusion backbone entirely from scratch, eliminating the need for point map tokenizers. Despite its simplicity, our approach surpasses complex latent-based diffusion models while remaining significantly simpler than hybrid alternatives. Notably, it produces sharper geometric structure and is more robust in highly ambiguous regions, such as transparent objects.

Related papers

• VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching [39.72379398607968]
  We introduce VolFill, a generative framework that predicts the 3D structure of the complete scene.<n>Our method utilizes a hybrid 3D VAE to compress sparse truncated unsigned distance function grids into a compact latent space.
  arXiv  Detail & Related papers  (2026-05-29T15:59:20Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes [67.39455433337316]
  We propose a novel approach that directly generates texture on 3D meshes.<n>By integrating this technique into a generative diffusion pipeline, we significantly improve the efficiency of texture generation.
  arXiv  Detail & Related papers  (2025-01-06T21:34:52Z)
• FlexiDreamer: Single Image-to-3D Generation with FlexiCubes [20.871847154995688]
  FlexiDreamer is a novel framework that directly reconstructs high-quality meshes from multi-view generated images. Our approach can generate high-fidelity 3D meshes in the single image-to-3D downstream task with approximately 1 minute.
  arXiv  Detail & Related papers  (2024-04-01T08:20:18Z)
• Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis [70.40950409274312]
  We modify density fields to encourage them to converge towards surfaces, without compromising their ability to reconstruct thin structures. We also develop a fusion-based meshing strategy followed by mesh simplification and appearance model fitting. The compact meshes produced by our model can be rendered in real-time on mobile devices.
  arXiv  Detail & Related papers  (2024-02-19T18:59:41Z)
• $PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction [97.06927852165464]
  Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision. We propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process.
  arXiv  Detail & Related papers  (2023-02-21T13:37:07Z)
• Representing 3D Shapes with Probabilistic Directed Distance Fields [7.528141488548544]
  We develop a novel shape representation that allows fast differentiable rendering within an implicit architecture. We show how to model inherent discontinuities in the underlying field. We also apply our method to fitting single shapes, unpaired 3D-aware generative image modelling, and single-image 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2021-12-10T02:15:47Z)
• Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis [90.26556260531707]
  DMTet is a conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such as coarse voxels. Unlike deep 3D generative models that directly generate explicit representations such as meshes, our model can synthesize shapes with arbitrary topology.
  arXiv  Detail & Related papers  (2021-11-08T05:29:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.