Volumetrically Consistent 3D Gaussian Rasterization

• URL: http://arxiv.org/abs/2412.03378v2
• Date: Wed, 22 Jan 2025 00:06:08 GMT
• Title: Volumetrically Consistent 3D Gaussian Rasterization
• Authors: Chinmay Talegaonkar, Yash Belhe, Ravi Ramamoorthi, Nicholas Antipa,
• Abstract summary: We show that splatting and its approximations are unnecessary, even within a viewizer.<n>We use this analytic transmittance framework to derive more physically-accurate alpha values than 3DGS.<n>Our method represents opaque surfaces with higher accuracy and fewer points than 3DGS.
• Score: 18.84882580327324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, 3D Gaussian Splatting (3DGS) has enabled photorealistic view synthesis at high inference speeds. However, its splatting-based rendering model makes several approximations to the rendering equation, reducing physical accuracy. We show that splatting and its approximations are unnecessary, even within a rasterizer; we instead volumetrically integrate 3D Gaussians directly to compute the transmittance across them analytically. We use this analytic transmittance to derive more physically-accurate alpha values than 3DGS, which can directly be used within their framework. The result is a method that more closely follows the volume rendering equation (similar to ray-tracing) while enjoying the speed benefits of rasterization. Our method represents opaque surfaces with higher accuracy and fewer points than 3DGS. This enables it to outperform 3DGS for view synthesis (measured in SSIM and LPIPS). Being volumetrically consistent also enables our method to work out of the box for tomography. We match the state-of-the-art 3DGS-based tomography method with fewer points.

Related papers

• View-Dependent Uncertainty Estimation of 3D Gaussian Splatting [0.6138671548064356]
  3D Gaussian Splatting (3DGS) has become increasingly popular in 3D scene reconstruction for its high visual accuracy. We propose to model uncertainty in 3DGS as an additional view-dependent per-gaussian feature that can be modeled with spherical harmonics. This simple yet effective modeling is easily interpretable and can be integrated into the traditional 3DGS pipeline.
  arXiv  Detail & Related papers  (2025-04-10T01:22:53Z)
• EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesis [61.1662426227688]
  Existing NeRF and 3DGS-based methods show promising results in achieving photorealistic renderings but require slow, per-scene optimization. We introduce EVolSplat, an efficient 3D Gaussian Splatting model for urban scenes that works in a feed-forward manner.
  arXiv  Detail & Related papers  (2025-03-26T02:47:27Z)
• Lightweight Gradient-Aware Upscaling of 3D Gaussian Splatting Images [0.0]
  Compared to 3DGS, it achieves significantly higher rendering speeds and reduces artifacts commonly observed in 3DGS reconstructions. Our technique upscales low-resolution 3DGS renderings with a marginal increase in cost.
  arXiv  Detail & Related papers  (2025-03-18T11:42:52Z)
• Does 3D Gaussian Splatting Need Accurate Volumetric Rendering? [8.421214057144569]
  3D Gaussian Splatting (3DGS) is an important reference method for learning 3D representations of a captured scene. NeRFs, which preceded 3DGS, are based on a principled ray-marching approach for rendering. We present an in-depth analysis of the various approximations and assumptions used by the original 3DGS solution.
  arXiv  Detail & Related papers  (2025-02-26T17:11:26Z)
• GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization [1.4466437171584356]
  3D Gaussian Splatting (3DGS) allows for the compact encoding of both 3D geometry and scene appearance with its spatial features. We propose distilling dense keypoint descriptors into 3DGS to improve the model's spatial understanding. Our approach surpasses state-of-the-art Neural Render Pose (NRP) methods, including NeRFMatch and PNeRFLoc.
  arXiv  Detail & Related papers  (2024-09-24T23:18:32Z)
• Mipmap-GS: Let Gaussians Deform with Scale-specific Mipmap for Anti-aliasing Rendering [81.88246351984908]
  We propose a unified optimization method to make Gaussians adaptive for arbitrary scales. Inspired by the mipmap technique, we design pseudo ground-truth for the target scale and propose a scale-consistency guidance loss to inject scale information into 3D Gaussians. Our method outperforms 3DGS in PSNR by an average of 9.25 dB for zoom-in and 10.40 dB for zoom-out.
  arXiv  Detail & Related papers  (2024-08-12T16:49:22Z)
• RaDe-GS: Rasterizing Depth in Gaussian Splatting [32.38730602146176]
  Gaussian Splatting (GS) has proven to be highly effective in novel view synthesis, achieving high-quality and real-time rendering. Our work introduces a Chamfer distance error comparable to NeuraLangelo on the DTU dataset and maintains similar computational efficiency as the original 3D GS methods.
  arXiv  Detail & Related papers  (2024-06-03T15:56:58Z)
• R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction [53.19869886963333]
  3D Gaussian splatting (3DGS) has shown promising results in rendering image and surface reconstruction. This paper introduces R2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction.
  arXiv  Detail & Related papers  (2024-05-31T08:39:02Z)
• LP-3DGS: Learning to Prune 3D Gaussian Splatting [71.97762528812187]
  We propose learning-to-prune 3DGS, where a trainable binary mask is applied to the importance score that can find optimal pruning ratio automatically. Experiments have shown that LP-3DGS consistently produces a good balance that is both efficient and high quality.
  arXiv  Detail & Related papers  (2024-05-29T05:58:34Z)
• Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh [44.57625460339714]
  We propose a triangular mesh to manipulate 3DGS directly with self-adaptation. Our approach is capable of handling large deformations, local manipulations, and soft body simulations while keeping high-fidelity rendering.
  arXiv  Detail & Related papers  (2024-05-28T04:13:21Z)
• SAGS: Structure-Aware 3D Gaussian Splatting [53.6730827668389]
  We propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene. SAGS reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets.
  arXiv  Detail & Related papers  (2024-04-29T23:26:30Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• GaussianPro: 3D Gaussian Splatting with Progressive Propagation [49.918797726059545]
  3DGS relies heavily on the point cloud produced by Structure-from-Motion (SfM) techniques. We propose a novel method that applies a progressive propagation strategy to guide the densification of the 3D Gaussians. Our method significantly surpasses 3DGS on the dataset, exhibiting an improvement of 1.15dB in terms of PSNR.
  arXiv  Detail & Related papers  (2024-02-22T16:00:20Z)
• GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting [51.96353586773191]
  We introduce textbfGS-SLAM that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping system. Our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets.
  arXiv  Detail & Related papers  (2023-11-20T12:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.