Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting

• URL: http://arxiv.org/abs/2408.09130v2
• Date: Tue, 20 Aug 2024 13:03:51 GMT
• Title: Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting
• Authors: Sheng Ye, Zhen-Hui Dong, Yubin Hu, Yu-Hui Wen, Yong-Jin Liu,
• Abstract summary: 3D Gaussian Splatting can synthesize remarkable novel views using consistent multi-view images as input. However, images captured in dark environments can exhibit considerable brightness variations and multi-view inconsistency. We propose Gaussian-DK, which produces high-quality renderings without ghosting and floater artifacts.
• Score: 23.94465817405213
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting has recently emerged as a powerful representation that can synthesize remarkable novel views using consistent multi-view images as input. However, we notice that images captured in dark environments where the scenes are not fully illuminated can exhibit considerable brightness variations and multi-view inconsistency, which poses great challenges to 3D Gaussian Splatting and severely degrades its performance. To tackle this problem, we propose Gaussian-DK. Observing that inconsistencies are mainly caused by camera imaging, we represent a consistent radiance field of the physical world using a set of anisotropic 3D Gaussians, and design a camera response module to compensate for multi-view inconsistencies. We also introduce a step-based gradient scaling strategy to constrain Gaussians near the camera, which turn out to be floaters, from splitting and cloning. Experiments on our proposed benchmark dataset demonstrate that Gaussian-DK produces high-quality renderings without ghosting and floater artifacts and significantly outperforms existing methods. Furthermore, we can also synthesize light-up images by controlling exposure levels that clearly show details in shadow areas.

Related papers

• DehazeGS: Seeing Through Fog with 3D Gaussian Splatting [17.119969983512533]
  We introduce DehazeGS, a method capable of decomposing and rendering a fog-free background from participating media. Experiments on both synthetic and real-world foggy datasets demonstrate that DehazeGS achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-01-07T09:47:46Z)
• Pushing Rendering Boundaries: Hard Gaussian Splatting [72.28941128988292]
  3D Gaussian Splatting (3DGS) has demonstrated impressive Novel View Synthesis (NVS) results in a real-time rendering manner. We propose Hard Gaussian Splatting, dubbed HGS, which considers multi-view significant positional gradients and rendering errors to grow hard Gaussians. Our method achieves state-of-the-art rendering quality while maintaining real-time efficiency.
  arXiv  Detail & Related papers  (2024-12-06T07:42:47Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos [58.22272760132996]
  We show that existing 4D Gaussian methods dramatically fail in this setup because the monocular setting is underconstrained. We propose Dynamic Gaussian Marbles, which consist of three core modifications that target the difficulties of the monocular setting. We evaluate on the Nvidia Dynamic Scenes dataset and the DyCheck iPhone dataset, and show that Gaussian Marbles significantly outperforms other Gaussian baselines in quality.
  arXiv  Detail & Related papers  (2024-06-26T19:37:07Z)
• RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting [51.51310922527121]
  We present a real-time 3D reconstruction system with an RGBD camera for large-scale environments using Gaussian splatting. We force each Gaussian to be either opaque or nearly transparent, with the opaque ones fitting the surface and dominant colors, and transparent ones fitting residual colors. We show real-time reconstructions of a variety of large scenes and show superior performance in the realism of novel view synthesis and camera tracking accuracy.
  arXiv  Detail & Related papers  (2024-04-30T16:54:59Z)
• Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusion [20.464224937528222]
  Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS suffers from a well-known'missing cone' problem, which results in poor reconstruction along the depth axis. We propose fusion algorithms that simultaneously utilize RGB camera data and sonar data.
  arXiv  Detail & Related papers  (2024-04-06T17:23:43Z)
• BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting [8.380954205255104]
  BAD-Gaussians is a novel approach to handle severe motion-blurred images with inaccurate camera poses. Our method achieves superior rendering quality compared to previous state-of-the-art deblur neural rendering methods.
  arXiv  Detail & Related papers  (2024-03-18T14:43:04Z)
• Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering [71.44349029439944]
  Recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering.
  arXiv  Detail & Related papers  (2023-11-30T17:58:57Z)
• LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS [55.85673901231235]
  We introduce LightGaussian, a method for transforming 3D Gaussians into a more compact format. Inspired by Network Pruning, LightGaussian identifies Gaussians with minimal global significance on scene reconstruction. LightGaussian achieves an average 15x compression rate while boosting FPS from 144 to 237 within the 3D-GS framework.
  arXiv  Detail & Related papers  (2023-11-28T21:39:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.