Neural Fields as Learnable Kernels for 3D Reconstruction

• URL: http://arxiv.org/abs/2111.13674v1
• Date: Fri, 26 Nov 2021 18:59:04 GMT
• Title: Neural Fields as Learnable Kernels for 3D Reconstruction
• Authors: Francis Williams, Zan Gojcic, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, Or Litany
• Abstract summary: We present a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression. Our technique achieves state-of-the-art results when reconstructing 3D objects and large scenes from sparse oriented points.
• Score: 101.54431372685018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Neural Kernel Fields: a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression. Our technique achieves state-of-the-art results when reconstructing 3D objects and large scenes from sparse oriented points, and can reconstruct shape categories outside the training set with almost no drop in accuracy. The core insight of our approach is that kernel methods are extremely effective for reconstructing shapes when the chosen kernel has an appropriate inductive bias. We thus factor the problem of shape reconstruction into two parts: (1) a backbone neural network which learns kernel parameters from data, and (2) a kernel ridge regression that fits the input points on-the-fly by solving a simple positive definite linear system using the learned kernel. As a result of this factorization, our reconstruction gains the benefits of data-driven methods under sparse point density while maintaining interpolatory behavior, which converges to the ground truth shape as input sampling density increases. Our experiments demonstrate a strong generalization capability to objects outside the train-set category and scanned scenes. Source code and pretrained models are available at https://nv-tlabs.github.io/nkf.

Related papers

• Neural Kernel Surface Reconstruction [80.51581494300423]
  We present a novel method for reconstructing a 3D implicit surface from a large-scale, sparse, and noisy point cloud. Our approach builds upon the recently introduced Neural Kernel Fields representation.
  arXiv  Detail & Related papers  (2023-05-31T06:25:18Z)
• Machine learning algorithms for three-dimensional mean-curvature computation in the level-set method [0.0]
  We propose a data-driven mean-curvature solver for the level-set method. Our proposed system can yield more accurate mean-curvature estimations than modern particle-based interface reconstruction.
  arXiv  Detail & Related papers  (2022-08-18T20:19:22Z)
• Learning "best" kernels from data in Gaussian process regression. With application to aerodynamics [0.4588028371034406]
  We introduce algorithms to select/design kernels in Gaussian process regression/kriging surrogate modeling techniques. A first class of algorithms is kernel flow, which was introduced in a context of classification in machine learning. A second class of algorithms is called spectral kernel ridge regression, and aims at selecting a "best" kernel such that the norm of the function to be approximated is minimal.
  arXiv  Detail & Related papers  (2022-06-03T07:50:54Z)
• NeuralEF: Deconstructing Kernels by Deep Neural Networks [47.54733625351363]
  Traditional nonparametric solutions based on the Nystr"om formula suffer from scalability issues. Recent work has resorted to a parametric approach, i.e., training neural networks to approximate the eigenfunctions. We show that these problems can be fixed by using a new series of objective functions that generalizes to space of supervised and unsupervised learning problems.
  arXiv  Detail & Related papers  (2022-04-30T05:31:07Z)
• Neural Networks as Kernel Learners: The Silent Alignment Effect [86.44610122423994]
  Neural networks in the lazy training regime converge to kernel machines. We show that this can indeed happen due to a phenomenon we term silent alignment. We also demonstrate that non-whitened data can weaken the silent alignment effect.
  arXiv  Detail & Related papers  (2021-10-29T18:22:46Z)
• DiGS : Divergence guided shape implicit neural representation for unoriented point clouds [36.60407995156801]
  Shape implicit neural representations (INRs) have recently shown to be effective in shape analysis and reconstruction tasks. We propose a divergence guided shape representation learning approach that does not require normal vectors as input.
  arXiv  Detail & Related papers  (2021-06-21T02:10:03Z)
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction [78.0514377738632]
  3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem.
  arXiv  Detail & Related papers  (2020-11-03T02:57:01Z)
• Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks [61.07202852469595]
  We present Neural Splines, a technique for 3D surface reconstruction that is based on random feature kernels arising from infinitely-wide shallow ReLU networks. Our method achieves state-of-the-art results, outperforming recent neural network-based techniques and widely used Poisson Surface Reconstruction.
  arXiv  Detail & Related papers  (2020-06-24T14:54:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.