GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering

• URL: http://arxiv.org/abs/2411.07478v1
• Date: Tue, 12 Nov 2024 01:51:05 GMT
• Title: GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering
• Authors: Zhihao Liang, Hongdong Li, Kui Jia, Kailing Guo, Qi Zhang,
• Abstract summary: We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
• Score: 83.69136534797686
• License:
• Abstract: Recovering the intrinsic physical attributes of a scene from images, generally termed as the inverse rendering problem, has been a central and challenging task in computer vision and computer graphics. In this paper, we present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading, effectiveness in handling complex materials. More importantly, we propose a unified shading solution that combines the advantages of both techniques for better decomposition. In addition, we analyze the normal modeling in 3D Gaussian Splatting (3DGS) and utilize the shortest axis as normal for each particle in GUS-IR, along with a depth-related regularization, resulting in improved geometric representation and better shape reconstruction. Furthermore, we enhance the probe-based baking scheme proposed by GS-IR to achieve more accurate ambient occlusion modeling to better handle indirect illumination. Extensive experiments have demonstrated the superior performance of GUS-IR in achieving precise intrinsic decomposition and geometric representation, supporting many downstream tasks (such as relighting, retouching) in computer vision, graphics, and extended reality.

Related papers

• GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering [6.820642721852439]
  We present GI-GS, a novel inverse rendering framework that leverages 3D Gaussian Splatting (3DGS) and deferred shading. In our framework, we first render a G-buffer to capture the detailed geometry and material properties of the scene. With the G-buffer and previous rendering results, the indirect lighting can be calculated through a lightweight path tracing.
  arXiv  Detail & Related papers  (2024-10-03T15:58:18Z)
• GS-ID: Illumination Decomposition on Gaussian Splatting via Diffusion Prior and Parametric Light Source Optimization [4.928698209254161]
  We present GS-ID, a novel framework for illumination decomposition on Gaussian Splatting. GS-ID produces state-of-the-art illumination decomposition results while achieving better geometry reconstruction and rendering performance.
  arXiv  Detail & Related papers  (2024-08-16T04:38:31Z)
• PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction [37.14913599050765]
  We propose a fast planar-based Gaussian splatting reconstruction representation (PGSR) to achieve high-fidelity surface reconstruction. We then introduce single-view geometric, multi-view photometric, and geometric regularization to preserve global geometric accuracy. Our method achieves fast training and rendering while maintaining high-fidelity rendering and geometric reconstruction, outperforming 3DGS-based and NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-10T17:59:01Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading [50.331929164207324]
  We introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view and editing tasks.
  arXiv  Detail & Related papers  (2024-04-15T01:58:54Z)
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields [50.056790168812114]
  3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. We demonstrate that our differentiable terms allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering.
  arXiv  Detail & Related papers  (2024-03-26T17:21:24Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.