3D Reconstruction with Fast Dipole Sums

• URL: http://arxiv.org/abs/2405.16788v4
• Date: Wed, 18 Sep 2024 06:09:04 GMT
• Title: 3D Reconstruction with Fast Dipole Sums
• Authors: Hanyu Chen, Bailey Miller, Ioannis Gkioulekas,
• Abstract summary: We introduce a method for high-quality 3D reconstruction from multiview images. We represent implicit geometry and radiance fields as per-point attributes of a dense point cloud. These queries facilitate the use of ray tracing to efficiently and differentiably render images.
• Score: 12.865206085308728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a method for high-quality 3D reconstruction from multi-view images. Our method uses a new point-based representation, the regularized dipole sum, which generalizes the winding number to allow for interpolation of per-point attributes in point clouds with noisy or outlier points. Using regularized dipole sums, we represent implicit geometry and radiance fields as per-point attributes of a dense point cloud, which we initialize from structure from motion. We additionally derive Barnes-Hut fast summation schemes for accelerated forward and adjoint dipole sum queries. These queries facilitate the use of ray tracing to efficiently and differentiably render images with our point-based representations, and thus update their point attributes to optimize scene geometry and appearance. We evaluate our method in inverse rendering applications against state-of-the-art alternatives, based on ray tracing of neural representations or rasterization of Gaussian point-based representations. Our method significantly improves 3D reconstruction quality and robustness at equal runtimes, while also supporting more general rendering methods such as shadow rays for direct illumination.

Related papers

• Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering [62.92985004295714]
  We present a method that avoids approximations that introduce bias into the renderings and, more importantly, the gradients used for optimization. We show that by removing these biases our approach improves the generality of radiance cache based inverse rendering, as well as increasing quality in the presence of challenging light transport effects such as specular reflections.
  arXiv  Detail & Related papers  (2024-09-09T17:59:57Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• Differentiable Point-based Inverse Rendering [9.88708409803907]
  DPIR is an analysis-by-synthesis method that processes images captured under diverse illuminations to estimate shape and spatially-varying BRDF. We devise a hybrid point-volumetric representation for geometry and a regularized basis-BRDF representation for reflectance. Our evaluations demonstrate that DPIR outperforms prior works in terms of reconstruction accuracy, computational efficiency, and memory footprint.
  arXiv  Detail & Related papers  (2023-12-05T04:13:31Z)
• RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction [3.1820300989695833]
  This paper introduces a versatile paradigm for integrating multi-view reflectance and normal maps acquired through photometric stereo. Our approach employs a pixel-wise joint re- parameterization of reflectance and normal, considering them as a vector of radiances rendered under simulated, varying illumination. It significantly improves the detailed 3D reconstruction of areas with high curvature or low visibility.
  arXiv  Detail & Related papers  (2023-12-02T19:49:27Z)
• GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting [51.96353586773191]
  We introduce textbfGS-SLAM that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping system. Our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets.
  arXiv  Detail & Related papers  (2023-11-20T12:08:23Z)
• Quadric Representations for LiDAR Odometry, Mapping and Localization [93.24140840537912]
  Current LiDAR odometry, mapping and localization methods leverage point-wise representations of 3D scenes. We propose a novel method of describing scenes using quadric surfaces, which are far more compact representations of 3D objects. Our method maintains low latency and memory utility while achieving competitive, and even superior, accuracy.
  arXiv  Detail & Related papers  (2023-04-27T13:52:01Z)
• Multi-View Mesh Reconstruction with Neural Deferred Shading [0.8514420632209809]
  State-of-the-art methods use both neural surface representations and neural shading. We represent surfaces as triangle meshes and build a differentiable rendering pipeline around triangle rendering and neural shading. We evaluate our runtime on a public 3D reconstruction dataset and show that it can match the reconstruction accuracy of traditional baselines while surpassing them in optimization.
  arXiv  Detail & Related papers  (2022-12-08T16:29:46Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• Shape As Points: A Differentiable Poisson Solver [118.12466580918172]
  In this paper, we introduce a differentiable point-to-mesh layer using a differentiable formulation of Poisson Surface Reconstruction (PSR) The differentiable PSR layer allows us to efficiently and differentiably bridge the explicit 3D point representation with the 3D mesh via the implicit indicator field. Compared to neural implicit representations, our Shape-As-Points (SAP) model is more interpretable, lightweight, and accelerates inference time by one order of magnitude.
  arXiv  Detail & Related papers  (2021-06-07T09:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.