RaNeuS: Ray-adaptive Neural Surface Reconstruction

• URL: http://arxiv.org/abs/2406.09801v1
• Date: Fri, 14 Jun 2024 07:54:25 GMT
• Title: RaNeuS: Ray-adaptive Neural Surface Reconstruction
• Authors: Yida Wang, David Joseph Tan, Nassir Navab, Federico Tombari,
• Abstract summary: We leverage a differentiable radiance field eg NeRF to reconstruct detailed 3D surfaces in addition to producing novel view renderings. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor. Our proposed textitRaNeuS are extensively evaluated on both synthetic and real datasets.
• Score: 87.20343320266215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Our objective is to leverage a differentiable radiance field \eg NeRF to reconstruct detailed 3D surfaces in addition to producing the standard novel view renderings. There have been related methods that perform such tasks, usually by utilizing a signed distance field (SDF). However, the state-of-the-art approaches still fail to correctly reconstruct the small-scale details, such as the leaves, ropes, and textile surfaces. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor to prioritize the rendering and zero-crossing surface fitting on top of establishing a perfect SDF. We propose to adaptively adjust the regularization on the signed distance field so that unsatisfying rendering rays won't enforce strong Eikonal regularization which is ineffective, and allow the gradients from regions with well-learned radiance to effectively back-propagated to the SDF. Consequently, balancing the two objectives in order to generate accurate and detailed surfaces. Additionally, concerning whether there is a geometric bias between the zero-crossing surface in SDF and rendering points in the radiance field, the projection becomes adjustable as well depending on different 3D locations during optimization. Our proposed \textit{RaNeuS} are extensively evaluated on both synthetic and real datasets, achieving state-of-the-art results on both novel view synthesis and geometric reconstruction.

Related papers

• AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction [55.69271635843385]
  We present AniSDF, a novel approach that learns fused-granularity neural surfaces with physics-based encoding for high-fidelity 3D reconstruction. Our method boosts the quality of SDF-based methods by a great scale in both geometry reconstruction and novel-view synthesis.
  arXiv  Detail & Related papers  (2024-10-02T03:10:38Z)
• ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction [50.07671826433922]
  It is non-trivial to simultaneously recover meticulous geometry and preserve smoothness across regions with differing characteristics. We propose ND-SDF, which learns a Normal Deflection field to represent the angular deviation between the scene normal and the prior normal. Our method not only obtains smooth weakly textured regions such as walls and floors but also preserves the geometric details of complex structures.
  arXiv  Detail & Related papers  (2024-08-22T17:59:01Z)
• GS-Octree: Octree-based 3D Gaussian Splatting for Robust Object-level 3D Reconstruction Under Strong Lighting [4.255847344539736]
  We introduce a novel approach that combines octree-based implicit surface representations with Gaussian splatting. Our method, which leverages the distribution of 3D Gaussians with SDFs, reconstructs more accurate geometry, particularly in images with specular highlights caused by strong lighting.
  arXiv  Detail & Related papers  (2024-06-26T09:29:56Z)
• GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction [3.043712258792239]
  We present a unified framework integrating neural SDF with 3DGS. This framework incorporates a learnable neural SDF field to guide the densification and pruning of Gaussians. Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-05-30T03:46:59Z)
• Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation [51.346733271166926]
  Mesh2NeRF is an approach to derive ground-truth radiance fields from textured meshes for 3D generation tasks. We validate the effectiveness of Mesh2NeRF across various tasks.
  arXiv  Detail & Related papers  (2024-03-28T11:22:53Z)
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail [54.03399077258403]
  We present LoD-NeuS, an efficient neural representation for high-frequency geometry detail recovery and anti-aliased novel view rendering. Our representation aggregates space features from a multi-convolved featurization within a conical frustum along a ray.
  arXiv  Detail & Related papers  (2023-09-19T05:44:00Z)
• Recovering Fine Details for Neural Implicit Surface Reconstruction [3.9702081347126943]
  We present D-NeuS, a volume rendering neural implicit surface reconstruction method capable to recover fine geometry details. We impose multi-view feature consistency on the surface points, derived by interpolating SDF zero-crossings from sampled points along rays. Our method reconstructs high-accuracy surfaces with details, and outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-11-21T10:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.