DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid

• URL: http://arxiv.org/abs/2311.08159v1
• Date: Tue, 14 Nov 2023 13:39:01 GMT
• Title: DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid
• Authors: Mirgahney Mohamed and Lourdes Agapito
• Abstract summary: DynamicSurf is a model-free neural implicit surface reconstruction method for high-fidelity 3D modelling of non-rigid surfaces from monocular RGB-D video. We learn a neural deformation field that maps a canonical representation of the surface geometry to the current frame. We demonstrate it can optimize sequences of varying frames with $6$ speedup over pure-based approaches.
• Score: 7.702806654565181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose DynamicSurf, a model-free neural implicit surface reconstruction method for high-fidelity 3D modelling of non-rigid surfaces from monocular RGB-D video. To cope with the lack of multi-view cues in monocular sequences of deforming surfaces, one of the most challenging settings for 3D reconstruction, DynamicSurf exploits depth, surface normals, and RGB losses to improve reconstruction fidelity and optimisation time. DynamicSurf learns a neural deformation field that maps a canonical representation of the surface geometry to the current frame. We depart from current neural non-rigid surface reconstruction models by designing the canonical representation as a learned feature grid which leads to faster and more accurate surface reconstruction than competing approaches that use a single MLP. We demonstrate DynamicSurf on public datasets and show that it can optimize sequences of varying frames with $6\times$ speedup over pure MLP-based approaches while achieving comparable results to the state-of-the-art methods. Project is available at https://mirgahney.github.io//DynamicSurf.io/.

Related papers

• DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting [13.762831851385227]
  We propose DynaSurfGS to achieve both photorealistic rendering and high-fidelity surface reconstruction of dynamic scenarios. The framework first incorporates Gaussian features from 4D neural voxels with the planar-based Gaussian Splatting to facilitate precise surface reconstruction. It also incorporates the as-rigid-as-possible (ARAP) constraint to maintain the approximate rigidity of local neighborhoods of 3D Gaussians between timesteps.
  arXiv  Detail & Related papers  (2024-08-26T01:36:46Z)
• DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction [93.18586302123633]
  This paper explores the problem of reconstructing temporally consistent surfaces from a 3D point cloud sequence without correspondence. We propose DynoSurf, an unsupervised learning framework integrating a template surface representation with a learnable deformation field. Experimental results demonstrate the significant superiority of DynoSurf over current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-03-18T08:58:48Z)
• MonoNPHM: Dynamic Head Reconstruction from Monocular Videos [47.504979561265536]
  We present Monocular Neural Parametric Head Models (MonoNPHM) for dynamic 3D head reconstructions from monocular RGB videos. We constrain predicted color values to be correlated with the underlying geometry such that gradients from RGB effectively influence latent geometry codes during inverse rendering.
  arXiv  Detail & Related papers  (2023-12-11T17:55:05Z)
• EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos [72.59573904930419]
  Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate representations of 3D scenes. We propose a novel neural-field-based method, called EndoSurf, which effectively learns to represent a deforming surface from an RGBD sequence.
  arXiv  Detail & Related papers  (2023-07-21T02:28:20Z)
• Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction [130.84162691963536]
  We introduce a novel dynamic grid optimization method for high-fidelity 3D surface reconstruction. We optimize the process by dynamically modifying the grid and assigning more finer-scale voxels to regions with higher complexity. The proposed approach is able to generate high-quality 3D reconstructions with fine details on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2023-04-12T22:39:57Z)
• HR-NeuS: Recovering High-Frequency Surface Geometry via Neural Implicit Surfaces [6.382138631957651]
  We present High-Resolution NeuS, a novel neural implicit surface reconstruction method. HR-NeuS recovers high-frequency surface geometry while maintaining large-scale reconstruction accuracy. We demonstrate through experiments on DTU and BlendedMVS datasets that our approach produces 3D geometries that are qualitatively more detailed and quantitatively of similar accuracy compared to previous approaches.
  arXiv  Detail & Related papers  (2023-02-14T02:25:16Z)
• HSurf-Net: Normal Estimation for 3D Point Clouds by Learning Hyper Surfaces [54.77683371400133]
  We propose a novel normal estimation method called HSurf-Net, which can accurately predict normals from point clouds with noise and density variations. Experimental results show that our HSurf-Net achieves the state-of-the-art performance on the synthetic shape dataset.
  arXiv  Detail & Related papers  (2022-10-13T16:39:53Z)
• Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera [26.410460029742456]
  We propose a template-free method to recover high-fidelity geometry and motions of a dynamic scene from a monocular RGB-D camera. Experiments on public datasets and our collected dataset demonstrate that NDR outperforms existing monocular dynamic reconstruction methods.
  arXiv  Detail & Related papers  (2022-06-30T13:09:39Z)
• MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction [72.05649682685197]
  State-of-the-art neural implicit methods allow for high-quality reconstructions of simple scenes from many input views. This is caused primarily by the inherent ambiguity in the RGB reconstruction loss that does not provide enough constraints. Motivated by recent advances in the area of monocular geometry prediction, we explore the utility these cues provide for improving neural implicit surface reconstruction.
  arXiv  Detail & Related papers  (2022-06-01T17:58:15Z)
• Deep Active Surface Models [60.027353171412216]
  Active Surface Models have a long history of being useful to model complex 3D surfaces but only Active Contours have been used in conjunction with deep networks. We introduce layers that implement them that can be integrated seamlessly into Graph Convolutional Networks to enforce sophisticated smoothness priors.
  arXiv  Detail & Related papers  (2020-11-17T18:48:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.