Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization

• URL: http://arxiv.org/abs/2309.01512v1
• Date: Mon, 4 Sep 2023 10:42:56 GMT
• Title: Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization
• Authors: Xianghui Yang, Guosheng Lin, Zhenghao Chen, Luping Zhou
• Abstract summary: We propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.
• Score: 73.3605319281966
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent neural networks based surface reconstruction can be roughly divided into two categories, one warping templates explicitly and the other representing 3D surfaces implicitly. To enjoy the advantages of both, we propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. This is achieved by directly predicting the displacements from surface queries and modeling shapes as Vector Fields, rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods do. In this way, our approach is capable of encoding both the distance and the direction fields so that the calculation of direction fields is differentiation-free, circumventing the non-trivial surface extraction step. Furthermore, building upon NVFs, we propose to incorporate two types of shape codebooks, \ie, NVFs (Lite or Ultra), to promote cross-category reconstruction through encoding cross-object priors. Moreover, we propose a new regularization based on analyzing the zero-curl property of NVFs, and implement this through the fully differentiable framework of our NVF (ultra). We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.

Related papers

• Shrinking: Reconstruction of Parameterized Surfaces from Signed Distance Fields [2.1638817206926855]
  We propose a novel method for reconstructing explicit parameterized surfaces from Signed Distance Fields (SDFs) Our approach iteratively contracts a parameterized initial sphere to conform to the target SDF shape, preserving differentiability and surface parameterization throughout. This enables downstream applications such as texture mapping, geometry processing, animation, and finite element analysis.
  arXiv  Detail & Related papers  (2024-10-04T03:39:15Z)
• Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi Partitioning [50.684254969269546]
  We introduce a novel method for acquiring boundary representations (B-Reps) of 3D CAD models. We apply a spatial partitioning to derive a single primitive within each partition. We show that our network, coined NVD-Net for neural Voronoi diagrams, can effectively learn Voronoi partitions for CAD models from training data.
  arXiv  Detail & Related papers  (2024-06-07T21:07:49Z)
• Neural Vector Fields: Implicit Representation by Explicit Learning [63.337294707047036]
  We propose a novel 3D representation method, Neural Vector Fields (NVF) It not only adopts the explicit learning process to manipulate meshes directly, but also the implicit representation of unsigned distance functions (UDFs) Our method first predicts displacement queries towards the surface and models shapes as text reconstructions.
  arXiv  Detail & Related papers  (2023-03-08T02:36:09Z)
• Factor Fields: A Unified Framework for Neural Fields and Beyond [50.29013417187368]
  We present Factor Fields, a novel framework for modeling and representing signals. Our framework accommodates several recent signal representations including NeRF, Plenoxels, EG3D, Instant-NGP, and TensoRF. Our representation achieves better image approximation quality on 2D image regression tasks, higher geometric quality when reconstructing 3D signed distance fields, and higher compactness for radiance field reconstruction tasks.
  arXiv  Detail & Related papers  (2023-02-02T17:06:50Z)
• NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies [87.06532943371575]
  We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
  arXiv  Detail & Related papers  (2022-11-25T15:21:45Z)
• Neural Vector Fields for Implicit Surface Representation and Inference [73.25812045209001]
  Implicit fields have recently shown increasing success in representing and learning 3D shapes accurately. We develop a novel and yet a fundamental representation considering unit vectors in 3D space and call it Vector Field (VF) We show the advantages of VF representation, in learning open, closed, or multi-layered as well as piecewise planar surfaces.
  arXiv  Detail & Related papers  (2022-04-13T17:53:34Z)
• Implicit Neural Deformation for Multi-View Face Reconstruction [43.88676778013593]
  We present a new method for 3D face reconstruction from multi-view RGB images. Unlike previous methods which are built upon 3D morphable models, our method leverages an implicit representation to encode rich geometric features. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-05T07:02:53Z)
• Coupling Explicit and Implicit Surface Representations for Generative 3D Modeling [41.79675639550555]
  We propose a novel neural architecture for representing 3D surfaces, which harnesses two complementary shape representations. We make these two representations synergistic by introducing novel consistency losses. Our hybrid architecture outputs results are superior to the output of the two equivalent single-representation networks.
  arXiv  Detail & Related papers  (2020-07-20T17:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.