VoD-3DGS: View-opacity-Dependent 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2501.17978v2
• Date: Fri, 31 Jan 2025 12:35:35 GMT
• Title: VoD-3DGS: View-opacity-Dependent 3D Gaussian Splatting
• Authors: Mateusz Nowak, Wojciech Jarosz, Peter Chin,
• Abstract summary: In computer graphics, materials can be classified as diffuse or specular, interacting with light differently. The standard 3D Gaussian Splatting model struggles to represent view-dependent content. We introduce an additional symmetric matrix to enhance the opacity representation of each 3D Gaussian.
• Score: 8.612696795829555
• License:
• Abstract: Reconstructing a 3D scene from images is challenging due to the different ways light interacts with surfaces depending on the viewer's position and the surface's material. In classical computer graphics, materials can be classified as diffuse or specular, interacting with light differently. The standard 3D Gaussian Splatting model struggles to represent view-dependent content, since it cannot differentiate an object within the scene from the light interacting with its specular surfaces, which produce highlights or reflections. In this paper, we propose to extend the 3D Gaussian Splatting model by introducing an additional symmetric matrix to enhance the opacity representation of each 3D Gaussian. This improvement allows certain Gaussians to be suppressed based on the viewer's perspective, resulting in a more accurate representation of view-dependent reflections and specular highlights without compromising the scene's integrity. By allowing the opacity to be view dependent, our enhanced model achieves state-of-the-art performance on Mip-Nerf, Tanks&Temples, Deep Blending, and Nerf-Synthetic datasets without a significant loss in rendering speed, achieving >60FPS, and only incurring a minimal increase in memory used.

Related papers

• Decoupling Appearance Variations with 3D Consistent Features in Gaussian Splatting [50.98884579463359]
  We propose DAVIGS, a method that decouples appearance variations in a plug-and-play manner. By transforming the rendering results at the image level instead of the Gaussian level, our approach can model appearance variations with minimal optimization time and memory overhead. We validate our method on several appearance-variant scenes, and demonstrate that it achieves state-of-the-art rendering quality with minimal training time and memory usage.
  arXiv  Detail & Related papers  (2025-01-18T14:55:58Z)
• DehazeGS: Seeing Through Fog with 3D Gaussian Splatting [17.119969983512533]
  We introduce DehazeGS, a method capable of decomposing and rendering a fog-free background from participating media. Experiments on both synthetic and real-world foggy datasets demonstrate that DehazeGS achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-01-07T09:47:46Z)
• EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis [72.53316783628803]
  We present Exact Volumetric Ellipsoid Rendering (EVER), a method for real-time differentiable emission-only volume rendering. Unlike recentization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation allows for exact volume rendering. We show that our method is more accurate with blending issues than 3DGS and follow-up work on view rendering.
  arXiv  Detail & Related papers  (2024-10-02T17:59:09Z)
• BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting [10.918133974256913]
  We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects.
  arXiv  Detail & Related papers  (2024-08-23T21:04:40Z)
• SpecGaussian with Latent Features: A High-quality Modeling of the View-dependent Appearance for 3D Gaussian Splatting [11.978842116007563]
  Lantent-SpecGS is an approach that utilizes a universal latent neural descriptor within each 3D Gaussian. Two parallel CNNs are designed to decoder the splatting feature maps into diffuse color and specular color separately. A mask that depends on the viewpoint is learned to merge these two colors, resulting in the final rendered image.
  arXiv  Detail & Related papers  (2024-08-23T15:25:08Z)
• Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting [23.94465817405213]
  3D Gaussian Splatting can synthesize remarkable novel views using consistent multi-view images as input. However, images captured in dark environments can exhibit considerable brightness variations and multi-view inconsistency. We propose Gaussian-DK, which produces high-quality renderings without ghosting and floater artifacts.
  arXiv  Detail & Related papers  (2024-08-17T08:05:09Z)
• Hybrid Explicit Representation for Ultra-Realistic Head Avatars [55.829497543262214]
  We introduce a novel approach to creating ultra-realistic head avatars and rendering them in real-time. UV-mapped 3D mesh is utilized to capture sharp and rich textures on smooth surfaces, while 3D Gaussian Splatting is employed to represent complex geometric structures. Experiments that our modeled results exceed those of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-18T04:01:26Z)
• Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting [55.71424195454963]
  Spec-Gaussian is an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces.
  arXiv  Detail & Related papers  (2024-02-24T17:22:15Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting [149.1673041605155]
  We address the problem of jointly estimating albedo, normals, depth and 3D spatially-varying lighting from a single image. Most existing methods formulate the task as image-to-image translation, ignoring the 3D properties of the scene. We propose a unified, learning-based inverse framework that formulates 3D spatially-varying lighting.
  arXiv  Detail & Related papers  (2021-09-13T15:29:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.