NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies

• URL: http://arxiv.org/abs/2211.14173v1
• Date: Fri, 25 Nov 2022 15:21:45 GMT
• Title: NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies
• Authors: Xiaoxiao Long, Cheng Lin, Lingjie Liu, Yuan Liu, Peng Wang, Christian Theobalt, Taku Komura, Wenping Wang
• Abstract summary: We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
• Score: 87.06532943371575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. Recent advances in neural rendering based reconstruction have achieved compelling results. However, these methods are limited to objects with closed surfaces since they adopt Signed Distance Function (SDF) as surface representation which requires the target shape to be divided into inside and outside. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation. Specifically, a new density function that correlates the property of UDF with the volume rendering scheme is introduced for robust optimization of the UDF fields. Experiments on the DTU and DeepFashion3D datasets show that our method not only enables high-quality reconstruction of non-closed shapes with complex typologies, but also achieves comparable performance to the SDF based methods on the reconstruction of closed surfaces.

Related papers

• Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization [73.3605319281966]
  We propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.
  arXiv  Detail & Related papers  (2023-09-04T10:42:56Z)
• NeUDF: Leaning Neural Unsigned Distance Fields with Volume Rendering [25.078149064632218]
  NeUDF can reconstruct surfaces with arbitrary topologies solely from multi-view supervision. We extensively evaluate our method over a number of challenging datasets, including DTU, MGN, and Deep Fashion 3D.
  arXiv  Detail & Related papers  (2023-04-20T04:14:42Z)
• NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images [17.637064969966847]
  NeAT is a new neural rendering framework that learns implicit surfaces with arbitrary topologies from multi-view images. NeAT supports easy field-to-mesh conversion using the classic Marching Cubes algorithm. Our approach is able to faithfully reconstruct both watertight and non-watertight surfaces.
  arXiv  Detail & Related papers  (2023-03-21T16:49:41Z)
• Recovering Fine Details for Neural Implicit Surface Reconstruction [3.9702081347126943]
  We present D-NeuS, a volume rendering neural implicit surface reconstruction method capable to recover fine geometry details. We impose multi-view feature consistency on the surface points, derived by interpolating SDF zero-crossings from sampled points along rays. Our method reconstructs high-accuracy surfaces with details, and outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-11-21T10:06:09Z)
• Learning Signed Distance Field for Multi-view Surface Reconstruction [24.090786783370195]
  We introduce a novel neural surface reconstruction framework that leverages the knowledge of stereo matching and feature consistency. We apply a signed distance field (SDF) and a surface light field to represent the scene geometry and appearance respectively. Our method is able to improve the robustness of geometry estimation and support reconstruction of complex scene topologies.
  arXiv  Detail & Related papers  (2021-08-23T06:23:50Z)
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction [88.02850205432763]
  We present a novel neural surface reconstruction method, called NeuS, for reconstructing objects and scenes with high fidelity from 2D image inputs. Existing neural surface reconstruction approaches, such as DVR and IDR, require foreground mask as supervision. We observe that the conventional volume rendering method causes inherent geometric errors for surface reconstruction. We propose a new formulation that is free of bias in the first order of approximation, thus leading to more accurate surface reconstruction even without the mask supervision.
  arXiv  Detail & Related papers  (2021-06-20T12:59:42Z)
• Deep Implicit Surface Point Prediction Networks [49.286550880464866]
  Deep neural representations of 3D shapes as implicit functions have been shown to produce high fidelity models. This paper presents a novel approach that models such surfaces using a new class of implicit representations called the closest surface-point (CSP) representation.
  arXiv  Detail & Related papers  (2021-06-10T14:31:54Z)
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes [77.6741486264257]
  We introduce an efficient neural representation that, for the first time, enables real-time rendering of high-fidelity neural SDFs. We show that our representation is 2-3 orders of magnitude more efficient in terms of rendering speed compared to previous works.
  arXiv  Detail & Related papers  (2021-01-26T18:50:22Z)
• Neural Unsigned Distance Fields for Implicit Function Learning [53.241423815726925]
  We propose Neural Distance Fields (NDF), a neural network based model which predicts the unsigned distance field for arbitrary 3D shapes. NDF represent surfaces at high resolutions as prior implicit models, but do not require closed surface data. NDF can be used for multi-target regression (multiple outputs for one input) with techniques that have been exclusively used for rendering in graphics.
  arXiv  Detail & Related papers  (2020-10-26T22:49:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.