3iGS: Factorised Tensorial Illumination for 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2408.03753v1
• Date: Wed, 7 Aug 2024 13:06:29 GMT
• Title: 3iGS: Factorised Tensorial Illumination for 3D Gaussian Splatting
• Authors: Zhe Jun Tang, Tat-Jen Cham,
• Abstract summary: 3D Gaussian Splatting, or 3iGS, improves upon 3D Gaussian Splatting (3DGS) rendering quality. Use of 3D Gaussians as representation of radiance fields has enabled high quality novel view synthesis at real-time rendering speed.
• Score: 15.059156311856087
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of 3D Gaussians as representation of radiance fields has enabled high quality novel view synthesis at real-time rendering speed. However, the choice of optimising the outgoing radiance of each Gaussian independently as spherical harmonics results in unsatisfactory view dependent effects. In response to these limitations, our work, Factorised Tensorial Illumination for 3D Gaussian Splatting, or 3iGS, improves upon 3D Gaussian Splatting (3DGS) rendering quality. Instead of optimising a single outgoing radiance parameter, 3iGS enhances 3DGS view-dependent effects by expressing the outgoing radiance as a function of a local illumination field and Bidirectional Reflectance Distribution Function (BRDF) features. We optimise a continuous incident illumination field through a Tensorial Factorisation representation, while separately fine-tuning the BRDF features of each 3D Gaussian relative to this illumination field. Our methodology significantly enhances the rendering quality of specular view-dependent effects of 3DGS, while maintaining rapid training and rendering speeds.

Related papers

• GlossGau: Efficient Inverse Rendering for Glossy Surface with Anisotropic Spherical Gaussian [4.5442067197725]
  GlossGau is an efficient inverse rendering framework that reconstructs scenes with glossy surfaces while maintaining training and rendering speeds comparable to vanilla 3D-GS. Experiments demonstrate that GlossGau achieves competitive or superior reconstruction on datasets with glossy surfaces.
  arXiv  Detail & Related papers  (2025-02-19T22:20:57Z)
• IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing [3.147103287687791]
  We introduce inter-reflective Gaussian splatting (IRGS) for inverse rendering. We apply the full rendering equation without simplification and compute incident radiance on the fly. Furthermore, we introduce a novel strategy for querying the indirect radiance of incident light when relighting the optimized scenes.
  arXiv  Detail & Related papers  (2024-12-20T13:10:43Z)
• 3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting [15.124165321341646]
  3D Gaussian Splatting (3DGS) has shown great potential for efficient reconstruction and high-fidelity real-time rendering on consumer hardware. 3DGS is constrained to ideal pinhole cameras and lacks support for secondary lighting effects. Recent methods address these limitations by tracing particles instead. We propose 3D Gaussian Unscented Transform (3DGUT), replacing the EWA splatting in 3DGS with the Unscented Transform.
  arXiv  Detail & Related papers  (2024-12-17T03:21:25Z)
• Beyond Gaussians: Fast and High-Fidelity 3D Splatting with Linear Kernels [51.08794269211701]
  We introduce 3D Linear Splatting (3DLS), which replaces Gaussian kernels with linear kernels to achieve sharper and more precise results. 3DLS demonstrates state-of-the-art fidelity and accuracy, along with a 30% FPS improvement over baseline 3DGS.
  arXiv  Detail & Related papers  (2024-11-19T11:59:54Z)
• L3DG: Latent 3D Gaussian Diffusion [74.36431175937285]
  L3DG is the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. We employ a sparse convolutional architecture to efficiently operate on room-scale scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time.
  arXiv  Detail & Related papers  (2024-10-17T13:19:32Z)
• PRTGS: Precomputed Radiance Transfer of Gaussian Splats for Real-Time High-Quality Relighting [24.48524333222271]
  We propose a real-time high-quality relighting method for Gaussian splats in low-frequency lighting environments. This method captures soft shadows and interreflections by precomputing 3D Gaussian splats' radiance transfer. We introduce distinct precomputing methods tailored for training and rendering stages, along with unique ray tracing and indirect lighting precomputation techniques.
  arXiv  Detail & Related papers  (2024-08-07T04:35:06Z)
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields [50.056790168812114]
  3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. We demonstrate that our differentiable terms allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering.
  arXiv  Detail & Related papers  (2024-03-26T17:21:24Z)
• Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting [55.71424195454963]
  Spec-Gaussian is an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces.
  arXiv  Detail & Related papers  (2024-02-24T17:22:15Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.