GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction

• URL: http://arxiv.org/abs/2504.21067v1
• Date: Tue, 29 Apr 2025 13:47:14 GMT
• Title: GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction
• Authors: Yuhan Xie, Yixi Cai, Yinqiang Zhang, Lei Yang, Jia Pan,
• Abstract summary: This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction.<n>Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models.
• Score: 18.528424809910884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction. Visual quality is a critical aspect of 3D reconstruction. Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models. Nonetheless, the efficient and effective acquisition of input images for reconstruction-specifically, the selection of the most informative viewpoint-remains an open challenge, which is crucial for active reconstruction. Existing studies have primarily focused on evaluating geometric completeness and exploring unobserved or unknown regions, without direct evaluation of the visual uncertainty within the reconstruction model. To address this gap, this paper introduces a probabilistic model that quantifies visual uncertainty for each Gaussian. Leveraging Shannon Mutual Information, we formulate a criterion, Gaussian Splatting Shannon Mutual Information (GauSS-MI), for real-time assessment of visual mutual information from novel viewpoints, facilitating the selection of next best view. GauSS-MI is implemented within an active reconstruction system integrated with a view and motion planner. Extensive experiments across various simulated and real-world scenes showcase the superior visual quality and reconstruction efficiency performance of the proposed system.

Related papers

• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Frequency-based View Selection in Gaussian Splatting Reconstruction [9.603843571051744]
  We investigate the problem of active view selection to perform 3D Gaussian Splatting reconstructions with as few input images as possible. By ranking the potential views in the frequency domain, we are able to effectively estimate the potential information gain of new viewpoints. Our method achieves state-of-the-art results in view selection, demonstrating its potential for efficient image-based 3D reconstruction.
  arXiv  Detail & Related papers  (2024-09-24T21:44:26Z)
• GaussianStego: A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting [38.33958617286536]
  GaussianStego is a method for embedding steganographic information in the rendering of generated 3D assets. Our approach employs an optimization framework that enables the accurate extraction of hidden information.
  arXiv  Detail & Related papers  (2024-07-01T13:57:44Z)
• MVSBoost: An Efficient Point Cloud-based 3D Reconstruction [4.282795945742752]
  Efficient and accurate 3D reconstruction is crucial for various applications, including augmented and virtual reality, medical imaging, and cinematic special effects. Traditional Multi-View Stereo (MVS) systems have been fundamental in these applications, but implicit 3D scene modeling has introduced new possibilities for handling complex topologies and continuous surfaces.
  arXiv  Detail & Related papers  (2024-06-19T13:02:17Z)
• Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [66.3814684757376]
  This work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level. The outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.
  arXiv  Detail & Related papers  (2024-03-21T10:38:18Z)
• Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields [82.21033337949757]
  This paper introduces a ray-based volumetric uncertainty estimator, which computes the entropy of the weight distribution of the color samples along each ray of the object's implicit neural representation. We show that it is possible to infer the uncertainty of the underlying 3D geometry given a novel view with the proposed estimator. We present a next-best-view selection policy guided by the ray-based volumetric uncertainty in neural radiance fields-based representations.
  arXiv  Detail & Related papers  (2022-09-17T21:28:57Z)
• NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction [64.36535692191343]
  Implicit neural representations have shown compelling results in offline 3D reconstruction and also recently demonstrated the potential for online SLAM systems. This paper addresses two key challenges: 1) seeking a criterion to measure the quality of the candidate viewpoints for the view planning based on the new representations, and 2) learning the criterion from data that can generalize to different scenes instead of hand-crafting one. Our method demonstrates significant improvements on various metrics for the rendered image quality and the geometry quality of the reconstructed 3D models when compared with variants using TSDF or reconstruction without view planning.
  arXiv  Detail & Related papers  (2022-07-22T10:05:36Z)
• MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction [72.05649682685197]
  State-of-the-art neural implicit methods allow for high-quality reconstructions of simple scenes from many input views. This is caused primarily by the inherent ambiguity in the RGB reconstruction loss that does not provide enough constraints. Motivated by recent advances in the area of monocular geometry prediction, we explore the utility these cues provide for improving neural implicit surface reconstruction.
  arXiv  Detail & Related papers  (2022-06-01T17:58:15Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.