Complete Gaussian Splats from a Single Image with Denoising Diffusion Models

• URL: http://arxiv.org/abs/2508.21542v1
• Date: Fri, 29 Aug 2025 11:55:47 GMT
• Title: Complete Gaussian Splats from a Single Image with Denoising Diffusion Models
• Authors: Ziwei Liao, Mohamed Sayed, Steven L. Waslander, Sara Vicente, Daniyar Turmukhambetov, Michael Firman,
• Abstract summary: We propose a latent diffusion model to reconstruct a complete 3D scene with Gaussian splats from only a single image during inference.<n>Our method generates faithful reconstructions and diverse samples with the ability to complete the occluded surfaces for high-quality 360-degree renderings.
• Score: 24.95877981068105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gaussian splatting typically requires dense observations of the scene and can fail to reconstruct occluded and unobserved areas. We propose a latent diffusion model to reconstruct a complete 3D scene with Gaussian splats, including the occluded parts, from only a single image during inference. Completing the unobserved surfaces of a scene is challenging due to the ambiguity of the plausible surfaces. Conventional methods use a regression-based formulation to predict a single "mode" for occluded and out-of-frustum surfaces, leading to blurriness, implausibility, and failure to capture multiple possible explanations. Thus, they often address this problem partially, focusing either on objects isolated from the background, reconstructing only visible surfaces, or failing to extrapolate far from the input views. In contrast, we propose a generative formulation to learn a distribution of 3D representations of Gaussian splats conditioned on a single input image. To address the lack of ground-truth training data, we propose a Variational AutoReconstructor to learn a latent space only from 2D images in a self-supervised manner, over which a diffusion model is trained. Our method generates faithful reconstructions and diverse samples with the ability to complete the occluded surfaces for high-quality 360-degree renderings.

Related papers

• OracleGS: Grounding Generative Priors for Sparse-View Gaussian Splatting [78.70702961852119]
  OracleGS reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting.<n>Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions.
  arXiv  Detail & Related papers  (2025-09-27T11:19:32Z)
• DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models [67.50989119438508]
  We introduce DSplats, a novel method that directly denoises multiview images using Gaussian-based Reconstructors to produce realistic 3D assets.<n>Our experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction.
  arXiv  Detail & Related papers  (2024-12-11T07:32:17Z)
• No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images [100.80376573969045]
  NoPoSplat is a feed-forward model capable of reconstructing 3D scenes parameterized by 3D Gaussians from multi-view images. Our model achieves real-time 3D Gaussian reconstruction during inference. This work makes significant advances in pose-free generalizable 3D reconstruction and demonstrates its applicability to real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-31T17:58:22Z)
• OFER: Occluded Face Expression Reconstruction [16.06622406877353]
  OFER is a novel approach for single-image 3D face reconstruction that can generate plausible, diverse, and expressive 3D faces.<n>To maintain consistency across diverse expressions, we propose a novel ranking mechanism that sorts the outputs of the shape diffusion network based on predicted shape accuracy scores.
  arXiv  Detail & Related papers  (2024-10-29T00:21:26Z)
• Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild [22.82439286651921]
  We propose a unified regression model that integrates segmentation and reconstruction, specifically designed for 3D shape reconstruction.<n>We also introduce a scalable data synthesis pipeline that simulates a wide range of variations in objects, occluders, and backgrounds.<n>Our training on our synthetic data enables the proposed model to achieve state-of-the-art zero-shot results on real-world images.
  arXiv  Detail & Related papers  (2024-03-21T16:40:10Z)
• GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting [81.03553265684184]
  We introduce GeoGS3D, a framework for reconstructing detailed 3D objects from single-view images. We propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization. Experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects.
  arXiv  Detail & Related papers  (2024-03-15T12:24:36Z)
• NeuSD: Surface Completion with Multi-View Text-to-Image Diffusion [56.98287481620215]
  We present a novel method for 3D surface reconstruction from multiple images where only a part of the object of interest is captured. Our approach builds on two recent developments: surface reconstruction using neural radiance fields for the reconstruction of the visible parts of the surface, and guidance of pre-trained 2D diffusion models in the form of Score Distillation Sampling (SDS) to complete the shape in unobserved regions in a plausible manner.
  arXiv  Detail & Related papers  (2023-12-07T19:30:55Z)
• Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision [76.32860119056964]
  We propose a novel class of denoising diffusion probabilistic models that learn to sample from distributions of signals that are never directly observed. We demonstrate the effectiveness of our method on three challenging computer vision tasks.
  arXiv  Detail & Related papers  (2023-06-20T17:53:00Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)
• DiffRF: Rendering-Guided 3D Radiance Field Diffusion [18.20324411024166]
  We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models. In contrast to 2D-diffusion models, our model learns multi-view consistent priors, enabling free-view synthesis and accurate shape generation.
  arXiv  Detail & Related papers  (2022-12-02T14:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.