3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2404.00409v1
• Date: Sat, 30 Mar 2024 16:35:38 GMT
• Title: 3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting
• Authors: Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang, Xiuzhe Wu, Ziyi Yang, Yilun Chen, Jiangmiao Pang, Xiaojuan Qi,
• Abstract summary: In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR. The key insight is incorporating an implicit signed distance field (SDF) within 3D Gaussians to enable them to be aligned and jointly optimized. Our experimental results demonstrate that our 3DGSR method enables high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS.
• Score: 58.95801720309658
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR, that allows for accurate 3D reconstruction with intricate details while inheriting the high efficiency and rendering quality of 3DGS. The key insight is incorporating an implicit signed distance field (SDF) within 3D Gaussians to enable them to be aligned and jointly optimized. First, we introduce a differentiable SDF-to-opacity transformation function that converts SDF values into corresponding Gaussians' opacities. This function connects the SDF and 3D Gaussians, allowing for unified optimization and enforcing surface constraints on the 3D Gaussians. During learning, optimizing the 3D Gaussians provides supervisory signals for SDF learning, enabling the reconstruction of intricate details. However, this only provides sparse supervisory signals to the SDF at locations occupied by Gaussians, which is insufficient for learning a continuous SDF. Then, to address this limitation, we incorporate volumetric rendering and align the rendered geometric attributes (depth, normal) with those derived from 3D Gaussians. This consistency regularization introduces supervisory signals to locations not covered by discrete 3D Gaussians, effectively eliminating redundant surfaces outside the Gaussian sampling range. Our extensive experimental results demonstrate that our 3DGSR method enables high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS. Besides, our method competes favorably with leading surface reconstruction techniques while offering a more efficient learning process and much better rendering qualities. The code will be available at https://github.com/CVMI-Lab/3DGSR.

Related papers

• Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set [49.780302894956776]
  It is vital to infer a signed distance function (SDF) in multi-view based surface reconstruction. We propose a method that seamlessly merge 3DGS with the learning of neural SDFs. Our numerical and visual comparisons show our superiority over the state-of-the-art results on the widely used benchmarks.
  arXiv  Detail & Related papers  (2024-10-18T05:48:06Z)
• L3DG: Latent 3D Gaussian Diffusion [74.36431175937285]
  L3DG is the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. We employ a sparse convolutional architecture to efficiently operate on room-scale scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time.
  arXiv  Detail & Related papers  (2024-10-17T13:19:32Z)
• GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting [4.255847344539736]
  We introduce a novel approach that combines octree-based implicit surface representations with Gaussian splatting. Our method, which leverages the distribution of 3D Gaussians with SDFs, reconstructs more accurate geometry, particularly in images with specular highlights caused by strong lighting.
  arXiv  Detail & Related papers  (2024-06-26T09:29:56Z)
• GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction [3.043712258792239]
  We present a unified framework integrating neural SDF with 3DGS. This framework incorporates a learnable neural SDF field to guide the densification and pruning of Gaussians. Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-05-30T03:46:59Z)
• GS-ROR: 3D Gaussian Splatting for Reflective Object Relighting via SDF Priors [20.523085632567717]
  We propose GS-ROR for reflective objects relighting with 3DGS aided by SDF priors. Our method outperforms the existing Gaussian-based inverse rendering methods in terms of relighting quality.
  arXiv  Detail & Related papers  (2024-05-22T09:40:25Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)
• SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition [66.80822249039235]
  3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis. We propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS. Our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.
  arXiv  Detail & Related papers  (2024-01-31T14:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.