Implicit Geometric Regularization for Learning Shapes

• URL: http://arxiv.org/abs/2002.10099v2
• Date: Thu, 9 Jul 2020 12:32:45 GMT
• Title: Implicit Geometric Regularization for Learning Shapes
• Authors: Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, Yaron Lipman
• Abstract summary: We offer a new paradigm for computing high fidelity implicit neural representations directly from raw data. We show that our method leads to state of the art implicit neural representations with higher level-of-details and fidelity compared to previous methods.
• Score: 34.052738965233445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representing shapes as level sets of neural networks has been recently proved to be useful for different shape analysis and reconstruction tasks. So far, such representations were computed using either: (i) pre-computed implicit shape representations; or (ii) loss functions explicitly defined over the neural level sets. In this paper we offer a new paradigm for computing high fidelity implicit neural representations directly from raw data (i.e., point clouds, with or without normal information). We observe that a rather simple loss function, encouraging the neural network to vanish on the input point cloud and to have a unit norm gradient, possesses an implicit geometric regularization property that favors smooth and natural zero level set surfaces, avoiding bad zero-loss solutions. We provide a theoretical analysis of this property for the linear case, and show that, in practice, our method leads to state of the art implicit neural representations with higher level-of-details and fidelity compared to previous methods.

Related papers

• Topological obstruction to the training of shallow ReLU neural networks [0.0]
  We study the interplay between the geometry of the loss landscape and the optimization trajectories of simple neural networks. This paper reveals the presence of topological obstruction in the loss landscape of shallow ReLU neural networks trained using gradient flow.
  arXiv  Detail & Related papers  (2024-10-18T19:17:48Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• From Complexity to Clarity: Analytical Expressions of Deep Neural Network Weights via Clifford's Geometric Algebra and Convexity [54.01594785269913]
  We show that optimal weights of deep ReLU neural networks are given by the wedge product of training samples when trained with standard regularized loss. The training problem reduces to convex optimization over wedge product features, which encode the geometric structure of the training dataset.
  arXiv  Detail & Related papers  (2023-09-28T15:19:30Z)
• Using Linear Regression for Iteratively Training Neural Networks [4.873362301533824]
  We present a simple linear regression based approach for learning the weights and biases of a neural network. The approach is intended to be to larger, more complex architectures.
  arXiv  Detail & Related papers  (2023-07-11T11:53:25Z)
• On the ISS Property of the Gradient Flow for Single Hidden-Layer Neural Networks with Linear Activations [0.0]
  We investigate the effects of overfitting on the robustness of gradient-descent training when subject to uncertainty on the gradient estimation. We show that the general overparametrized formulation introduces a set of spurious equilibria which lay outside the set where the loss function is minimized.
  arXiv  Detail & Related papers  (2023-05-17T02:26:34Z)
• Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets [57.06026574261203]
  We provide theoretical evidence for learning low-dimensional state spaces, which can also model long-term memory. Experiments corroborate our theory, demonstrating extrapolation via learning low-dimensional state spaces with both linear and non-linear RNNs.
  arXiv  Detail & Related papers  (2022-10-25T14:45:15Z)
• Linear Adversarial Concept Erasure [108.37226654006153]
  We formulate the problem of identifying and erasing a linear subspace that corresponds to a given concept. We show that the method is highly expressive, effectively mitigating bias in deep nonlinear classifiers while maintaining tractability and interpretability.
  arXiv  Detail & Related papers  (2022-01-28T13:00:17Z)
• The Dynamics of Gradient Descent for Overparametrized Neural Networks [19.11271777632797]
  We show that the dynamics of neural network weights under GD converge to a point which is close to the minimum norm solution. To illustrate the application of this result, we show that the GD converges to a gradient function that generalizes well.
  arXiv  Detail & Related papers  (2021-05-13T22:20:30Z)
• Over-parametrized neural networks as under-determined linear systems [31.69089186688224]
  We show that it is unsurprising simple neural networks can achieve zero training loss. We show that kernels typically associated with the ReLU activation function have fundamental flaws. We propose new activation functions that avoid the pitfalls of ReLU in that they admit zero training loss solutions for any set of distinct data points.
  arXiv  Detail & Related papers  (2020-10-29T21:43:00Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.