Neural Polytopes

• URL: http://arxiv.org/abs/2307.00721v2
• Date: Mon, 10 Jul 2023 03:00:48 GMT
• Title: Neural Polytopes
• Authors: Koji Hashimoto, Tomoya Naito, Hisashi Naito
• Abstract summary: We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and layers. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes. They are a smooth analogue of polytopes, exhibiting geometric duality. This finding initiates research of generative discrete geometry to approximate surfaces by machine learning.

Related papers

• Learning Geometric Invariant Features for Classification of Vector Polygons with Graph Message-passing Neural Network [3.804240190982697]
  We propose a novel graph message-passing neural network (PolyMP) to learn the geometric-invariant features for shape classification of polygons. We show that the proposed graph-based PolyMP network enables the learning of expressive geometric features invariant to geometric transformations of polygons.
  arXiv  Detail & Related papers  (2024-07-05T08:19:36Z)
• A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on $n$-dimensional Classes [49.32130498861987]
  We study the case of non-differentiable activation functions, such as ReLU. Two recent works introduced a geometric framework to study neural networks. We illustrate our findings with some numerical experiments on classification of images and thermodynamic problems.
  arXiv  Detail & Related papers  (2024-04-09T08:11:46Z)
• Defining Neural Network Architecture through Polytope Structures of Dataset [53.512432492636236]
  This paper defines upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. It is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces.
  arXiv  Detail & Related papers  (2024-02-04T08:57:42Z)
• Geometry of Polynomial Neural Networks [3.498371632913735]
  We study the expressivity and learning process for neural networks (PNNs) with monomial activation functions. These theoretical results are accompanied by experiments.
  arXiv  Detail & Related papers  (2024-02-01T19:06:06Z)
• Data Topology-Dependent Upper Bounds of Neural Network Widths [52.58441144171022]
  We first show that a three-layer neural network can be designed to approximate an indicator function over a compact set. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. We prove the universal approximation property of three-layer ReLU networks using our topological approach.
  arXiv  Detail & Related papers  (2023-05-25T14:17:15Z)
• Deep ReLU Networks Have Surprisingly Simple Polytopes [22.969361799006734]
  A ReLU network is a piecewise linear function over polytopes. We study the shapes of polytopes via the number of simplices obtained by triangulating the polytope. By characterizing the shape of polytopes, the number of simplices be a leverage for other problems.
  arXiv  Detail & Related papers  (2023-05-16T03:51:34Z)
• Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes [3.0079490585515343]
  We show that $lceillog_(n)rceil$ hidden layers are indeed necessary to compute the maximum of $n$ numbers. Our results are based on the known duality between neural networks and Newton polytopes via tropical geometry.
  arXiv  Detail & Related papers  (2023-02-24T10:14:53Z)
• Towards General-Purpose Representation Learning of Polygonal Geometries [62.34832826705641]
  We develop a general-purpose polygon encoding model, which can encode a polygonal geometry into an embedding space. We conduct experiments on two tasks: 1) shape classification based on MNIST; 2) spatial relation prediction based on two new datasets - DBSR-46K and DBSR-cplx46K. Our results show that NUFTspec and ResNet1D outperform multiple existing baselines with significant margins.
  arXiv  Detail & Related papers  (2022-09-29T15:59:23Z)
• A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations [77.86290991564829]
  Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. We study a particular sequence of maps between manifold, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between implementing neural networks of practical interest.
  arXiv  Detail & Related papers  (2021-12-17T11:43:30Z)
• PolyNet: Polynomial Neural Network for 3D Shape Recognition with PolyShape Representation [51.147664305955495]
  3D shape representation and its processing have substantial effects on 3D shape recognition. We propose a deep neural network-based method (PolyNet) and a specific polygon representation (PolyShape) Our experiments demonstrate the strength and the advantages of PolyNet on both 3D shape classification and retrieval tasks.
  arXiv  Detail & Related papers  (2021-10-15T06:45:59Z)
• Embed Me If You Can: A Geometric Perceptron [14.274582421372308]
  We introduce an extension of the multilayer hypersphere perceptron (MLHP) Our model is superior to the vanilla multilayer perceptron when classifying 3D Tetris shapes.
  arXiv  Detail & Related papers  (2020-06-11T15:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.