DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization

• URL: http://arxiv.org/abs/2403.06912v3
• Date: Sun, 24 Mar 2024 18:10:11 GMT
• Title: DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization
• Authors: Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, Lin Gu,
• Abstract summary: Radiance fields have demonstrated impressive performance in synthesizing novel views from sparse input views, yet prevailing methods suffer from high training costs and slow inference speed. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian radiance fields, offering real-time and high-quality few-shot novel view synthesis at low costs.
• Score: 21.474938045227702
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Radiance fields have demonstrated impressive performance in synthesizing novel views from sparse input views, yet prevailing methods suffer from high training costs and slow inference speed. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian radiance fields, offering real-time and high-quality few-shot novel view synthesis at low costs. Our motivation stems from the highly efficient representation and surprising quality of the recent 3D Gaussian Splatting, despite it will encounter a geometry degradation when input views decrease. In the Gaussian radiance fields, we find this degradation in scene geometry primarily lined to the positioning of Gaussian primitives and can be mitigated by depth constraint. Consequently, we propose a Hard and Soft Depth Regularization to restore accurate scene geometry under coarse monocular depth supervision while maintaining a fine-grained color appearance. To further refine detailed geometry reshaping, we introduce Global-Local Depth Normalization, enhancing the focus on small local depth changes. Extensive experiments on LLFF, DTU, and Blender datasets demonstrate that DNGaussian outperforms state-of-the-art methods, achieving comparable or better results with significantly reduced memory cost, a $25 \times$ reduction in training time, and over $3000 \times$ faster rendering speed.

Related papers

• CityGaussianV2: Efficient and Geometrically Accurate Reconstruction for Large-Scale Scenes [53.107474952492396]
  CityGaussianV2 is a novel approach for large-scale scene reconstruction. We implement a decomposed-gradient-based densification and depth regression technique to eliminate blurry artifacts and accelerate convergence. Our method strikes a promising balance between visual quality, geometric accuracy, as well as storage and training costs.
  arXiv  Detail & Related papers  (2024-11-01T17:59:31Z)
• GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction [3.043712258792239]
  We present a unified framework integrating neural SDF with 3DGS. This framework incorporates a learnable neural SDF field to guide the densification and pruning of Gaussians. Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-05-30T03:46:59Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields [50.056790168812114]
  3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. We demonstrate that our differentiable terms allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering.
  arXiv  Detail & Related papers  (2024-03-26T17:21:24Z)
• DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing [19.437747560051566]
  We propose an adaptive depth loss based on the gradient of color images, improving depth estimation and novel view synthesis results over various baselines. Our simple yet effective regularization technique enables direct mesh extraction from the Gaussian representation, yielding more physically accurate reconstructions of indoor scenes.
  arXiv  Detail & Related papers  (2024-03-26T16:00:31Z)
• Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting [55.71424195454963]
  Spec-Gaussian is an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces.
  arXiv  Detail & Related papers  (2024-02-24T17:22:15Z)
• FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting [58.41056963451056]
  We propose a few-shot view synthesis framework based on 3D Gaussian Splatting. This framework enables real-time and photo-realistic view synthesis with as few as three training views. FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets.
  arXiv  Detail & Related papers  (2023-12-01T09:30:02Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces [45.15827491185572]
  We present a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces. Experiments show that GaussianShader strikes a commendable balance between efficiency and visual quality.
  arXiv  Detail & Related papers  (2023-11-29T17:22:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.