Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction

• URL: http://arxiv.org/abs/2409.03634v1
• Date: Thu, 5 Sep 2024 15:48:02 GMT
• Title: Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction
• Authors: Rui Peng, Shihe Shen, Kaiqiang Xiong, Huachen Gao, Jianbo Jiao, Xiaodong Gu, Ronggang Wang,
• Abstract summary: SuRF is a new framework that incorporates a new Region sparsification based on a matching Field. To our knowledge, this is the first unsupervised method achieving end-to-end sparsification. Experiments show that our reconstructions exhibit high-quality details and achieve new state-of-the-art performance.
• Score: 29.26047429432185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconstructing the high-fidelity surface from multi-view images, especially sparse images, is a critical and practical task that has attracted widespread attention in recent years. However, existing methods are impeded by the memory constraint or the requirement of ground-truth depths and cannot recover satisfactory geometric details. To this end, we propose SuRF, a new Surface-centric framework that incorporates a new Region sparsification based on a matching Field, achieving good trade-offs between performance, efficiency and scalability. To our knowledge, this is the first unsupervised method achieving end-to-end sparsification powered by the introduced matching field, which leverages the weight distribution to efficiently locate the boundary regions containing surface. Instead of predicting an SDF value for each voxel, we present a new region sparsification approach to sparse the volume by judging whether the voxel is inside the surface region. In this way, our model can exploit higher frequency features around the surface with less memory and computational consumption. Extensive experiments on multiple benchmarks containing complex large-scale scenes show that our reconstructions exhibit high-quality details and achieve new state-of-the-art performance, i.e., 46% improvements with 80% less memory consumption. Code is available at https://github.com/prstrive/SuRF.

Related papers

• NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction [63.85586195085141]
  Signed Distance Function (SDF)-based volume rendering has demonstrated significant capabilities in surface reconstruction. We introduce NeuRodin, a novel two-stage neural surface reconstruction framework. NeuRodin achieves high-fidelity surface reconstruction and retains the flexible optimization characteristics of density-based methods.
  arXiv  Detail & Related papers  (2024-08-19T17:36:35Z)
• GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering [83.19049705653072]
  During the Gaussian Splatting optimization process, the scene's geometry can gradually deteriorate if its structure is not deliberately preserved. We propose a novel approach called GeoGaussian to mitigate this issue. Our proposed pipeline achieves state-of-the-art performance in novel view synthesis and geometric reconstruction.
  arXiv  Detail & Related papers  (2024-03-17T20:06:41Z)
• NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views [41.03837477483364]
  We propose a novel sparse view reconstruction framework that leverages on-surface priors to achieve highly faithful surface reconstruction. Specifically, we design several constraints on global geometry alignment and local geometry refinement for jointly optimizing coarse shapes and fine details. The experimental results with DTU and BlendedMVS datasets in two prevalent sparse settings demonstrate significant improvements over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-21T16:04:45Z)
• VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis [73.50359502037232]
  VoxNeRF is a novel approach to enhance the quality and efficiency of neural indoor reconstruction and novel view synthesis. We propose an efficient voxel-guided sampling technique that allocates computational resources to selectively the most relevant segments of rays. Our approach is validated with extensive experiments on ScanNet and ScanNet++.
  arXiv  Detail & Related papers  (2023-11-09T11:32:49Z)
• Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement [50.56517624931987]
  The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions. Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry. This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations.
  arXiv  Detail & Related papers  (2023-09-14T12:05:29Z)
• Multi-View Stereo Representation Revisit: Region-Aware MVSNet [8.264851594332677]
  Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. We propose RA-MVSNet to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface.
  arXiv  Detail & Related papers  (2023-04-26T15:17:51Z)
• Grid-guided Neural Radiance Fields for Large Urban Scenes [146.06368329445857]
  Recent approaches propose to geographically divide the scene and adopt multiple sub-NeRFs to model each region individually. An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene. We present a new framework that realizes high-fidelity rendering on large urban scenes while being computationally efficient.
  arXiv  Detail & Related papers  (2023-03-24T13:56:45Z)
• NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images [17.637064969966847]
  NeAT is a new neural rendering framework that learns implicit surfaces with arbitrary topologies from multi-view images. NeAT supports easy field-to-mesh conversion using the classic Marching Cubes algorithm. Our approach is able to faithfully reconstruct both watertight and non-watertight surfaces.
  arXiv  Detail & Related papers  (2023-03-21T16:49:41Z)
• DARF: Depth-Aware Generalizable Neural Radiance Field [51.29437249009986]
  We propose the Depth-Aware Generalizable Neural Radiance Field (DARF) with a Depth-Aware Dynamic Sampling (DADS) strategy. Our framework infers the unseen scenes on both pixel level and geometry level with only a few input images. Compared with state-of-the-art generalizable NeRF methods, DARF reduces samples by 50%, while improving rendering quality and depth estimation.
  arXiv  Detail & Related papers  (2022-12-05T14:00:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.