3D Gaussian Splatting with Deferred Reflection

• URL: http://arxiv.org/abs/2404.18454v2
• Date: Tue, 4 Jun 2024 11:59:54 GMT
• Title: 3D Gaussian Splatting with Deferred Reflection
• Authors: Keyang Ye, Qiming Hou, Kun Zhou,
• Abstract summary: We present a deferred shading method to render specular reflection with Gaussian splatting. Our method significantly outperforms state-of-the-art techniques and concurrent work in synthesizing high-quality specular reflection effects.
• Score: 25.254842246219585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of neural and Gaussian-based radiance field methods have achieved great success in the field of novel view synthesis. However, specular reflection remains non-trivial, as the high frequency radiance field is notoriously difficult to fit stably and accurately. We present a deferred shading method to effectively render specular reflection with Gaussian splatting. The key challenge comes from the environment map reflection model, which requires accurate surface normal while simultaneously bottlenecks normal estimation with discontinuous gradients. We leverage the per-pixel reflection gradients generated by deferred shading to bridge the optimization process of neighboring Gaussians, allowing nearly correct normal estimations to gradually propagate and eventually spread over all reflective objects. Our method significantly outperforms state-of-the-art techniques and concurrent work in synthesizing high-quality specular reflection effects, demonstrating a consistent improvement of peak signal-to-noise ratio (PSNR) for both synthetic and real-world scenes, while running at a frame rate almost identical to vanilla Gaussian splatting.

Related papers

• Reflective Gaussian Splatting [36.111845416439095]
  We introduce a Reflective Gaussian splatting (Ref-Gaussian) framework characterized with two components. Ref-Gaussian surpasses existing approaches in terms of quantitative metrics, visual quality, and compute efficiency. Also, we show that Ref-Gaussian supports more applications such as relighting and editing.
  arXiv  Detail & Related papers  (2024-12-26T16:58:35Z)
• IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing [3.147103287687791]
  We introduce inter-reflective Gaussian splatting (IRGS) for inverse rendering. We apply the full rendering equation without simplification and compute incident radiance on the fly. Furthermore, we introduce a novel strategy for querying the indirect radiance of incident light when relighting the optimized scenes.
  arXiv  Detail & Related papers  (2024-12-20T13:10:43Z)
• EnvGS: Modeling View-Dependent Appearance with Environment Gaussian [78.74634059559891]
  EnvGS is a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments. To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based reflection that leverages the GPU's RT core for fast rendering. Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections.
  arXiv  Detail & Related papers  (2024-12-19T18:59:57Z)
• GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering [83.69136534797686]
  We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
  arXiv  Detail & Related papers  (2024-11-12T01:51:05Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading [50.331929164207324]
  We introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view and editing tasks.
  arXiv  Detail & Related papers  (2024-04-15T01:58:54Z)
• GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces [45.15827491185572]
  We present a novel method that applies a simplified shading function on 3D Gaussians to enhance the neural rendering in scenes with reflective surfaces. Experiments show that GaussianShader strikes a commendable balance between efficiency and visual quality.
  arXiv  Detail & Related papers  (2023-11-29T17:22:26Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)
• BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields [0.0]
  We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
  arXiv  Detail & Related papers  (2023-10-05T14:27:06Z)
• IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis [90.03590032170169]
  We present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method. Our experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results.
  arXiv  Detail & Related papers  (2022-10-02T22:45:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.