GradientSurf: Gradient-Domain Neural Surface Reconstruction from RGB Video

• URL: http://arxiv.org/abs/2310.05406v1
• Date: Mon, 9 Oct 2023 04:54:30 GMT
• Title: GradientSurf: Gradient-Domain Neural Surface Reconstruction from RGB Video
• Authors: Crane He Chen, Joerg Liebelt
• Abstract summary: GradientSurf is a novel algorithm for real time surface reconstruction from monocular RGB video. Inspired by Poisson Surface Reconstruction, the proposed method builds on the tight coupling between surface, volume, and oriented point cloud. For the task of indoor scene reconstruction, experimental results show that the proposed method reconstructs surfaces with more details in curved regions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes GradientSurf, a novel algorithm for real time surface reconstruction from monocular RGB video. Inspired by Poisson Surface Reconstruction, the proposed method builds on the tight coupling between surface, volume, and oriented point cloud and solves the reconstruction problem in gradient-domain. Unlike Poisson Surface Reconstruction which finds an offline solution to the Poisson equation by solving a linear system after the scanning process is finished, our method finds online solutions from partial scans with a neural network incrementally where the Poisson layer is designed to supervise both local and global reconstruction. The main challenge that existing methods suffer from when reconstructing from RGB signal is a lack of details in the reconstructed surface. We hypothesize this is due to the spectral bias of neural networks towards learning low frequency geometric features. To address this issue, the reconstruction problem is cast onto gradient domain, where zeroth-order and first-order energies are minimized. The zeroth-order term penalizes location of the surface. The first-order term penalizes the difference between the gradient of reconstructed implicit function and the vector field formulated from oriented point clouds sampled at adaptive local densities. For the task of indoor scene reconstruction, visual and quantitative experimental results show that the proposed method reconstructs surfaces with more details in curved regions and higher fidelity for small objects than previous methods.

Related papers

• VoxNeuS: Enhancing Voxel-Based Neural Surface Reconstruction via Gradient Interpolation [10.458776364195796]
  We propose VoxNeuS, a lightweight surface reconstruction method for computational and memory efficient neural surface reconstruction. The entire training process takes 15 minutes and less than 3 GB of memory on a single 2080ti GPU.
  arXiv  Detail & Related papers  (2024-06-11T11:26:27Z)
• GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering [83.19049705653072]
  During the Gaussian Splatting optimization process, the scene's geometry can gradually deteriorate if its structure is not deliberately preserved. We propose a novel approach called GeoGaussian to mitigate this issue. Our proposed pipeline achieves state-of-the-art performance in novel view synthesis and geometric reconstruction.
  arXiv  Detail & Related papers  (2024-03-17T20:06:41Z)
• Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement [50.56517624931987]
  The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions. Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry. This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations.
  arXiv  Detail & Related papers  (2023-09-14T12:05:29Z)
• Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian [44.28251558359345]
  We propose a new approach for reconstructing surfaces from point clouds. Our technique aligns the gradients for a near-surface point and its on-surface projection point, producing a rough but faithful shape within just a few iterations.
  arXiv  Detail & Related papers  (2023-09-04T20:10:38Z)
• Neural Poisson Surface Reconstruction: Resolution-Agnostic Shape Reconstruction from Point Clouds [53.02191521770926]
  We introduce Neural Poisson Surface Reconstruction (nPSR), an architecture for shape reconstruction that addresses the challenge of recovering 3D shapes from points. nPSR exhibits two main advantages: First, it enables efficient training on low-resolution data while achieving comparable performance at high-resolution evaluation. Overall, the neural Poisson surface reconstruction not only improves upon the limitations of classical deep neural networks in shape reconstruction but also achieves superior results in terms of reconstruction quality, running time, and resolution agnosticism.
  arXiv  Detail & Related papers  (2023-08-03T13:56:07Z)
• Depth-NeuS: Neural Implicit Surfaces Learning for Multi-view Reconstruction Based on Depth Information Optimization [6.493546601668505]
  Methods for neural surface representation and rendering, for example NeuS, have shown that learning neural implicit surfaces through volume rendering is becoming increasingly popular. Existing methods lack a direct representation of depth information, which makes object reconstruction unrestricted by geometric features. This is because existing methods only use surface normals to represent implicit surfaces without using depth information. We propose a neural implicit surface learning method called Depth-NeuS based on depth information optimization for multi-view reconstruction.
  arXiv  Detail & Related papers  (2023-03-30T01:19:27Z)
• Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data [63.34506218832164]
  In this work, we investigate the implicit bias of gradient flow and gradient descent in two-layer fully-connected neural networks with ReLU activations. For gradient flow, we leverage recent work on the implicit bias for homogeneous neural networks to show that leakyally, gradient flow produces a neural network with rank at most two. For gradient descent, provided the random variance is small enough, we show that a single step of gradient descent suffices to drastically reduce the rank of the network, and that the rank remains small throughout training.
  arXiv  Detail & Related papers  (2022-10-13T15:09:54Z)
• Learning Modified Indicator Functions for Surface Reconstruction [10.413340575612233]
  We propose a learning-based approach for implicit surface reconstruction from raw point clouds without normals. Our method is inspired by Gauss Lemma in potential energy theory, which gives an explicit integral formula for the indicator functions. We design a novel deep neural network to perform surface integral and learn the modified indicator functions from un-oriented and noisy point clouds.
  arXiv  Detail & Related papers  (2021-11-18T05:30:35Z)
• Projective Manifold Gradient Layer for Deep Rotation Regression [49.85464297105456]
  Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem. We propose a manifold-aware gradient that directly backpropagates into deep network weights.
  arXiv  Detail & Related papers  (2021-10-22T08:34:15Z)
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction [88.02850205432763]
  We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconstruction approaches, such as DVR and IDR, require foreground mask as supervision. We observe that the conventional volume rendering method causes inherent geometric errors for surface reconstruction. We propose a new formulation that is free of bias in the first order of approximation, thus leading to more accurate surface reconstruction even without the mask supervision.
  arXiv  Detail & Related papers  (2021-06-20T12:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.