Advances in the training, pruning and enforcement of shape constraints of Morphological Neural Networks using Tropical Algebra

• URL: http://arxiv.org/abs/2011.07643v1
• Date: Sun, 15 Nov 2020 22:44:25 GMT
• Title: Advances in the training, pruning and enforcement of shape constraints of Morphological Neural Networks using Tropical Algebra
• Authors: Nikolaos Dimitriadis, Petros Maragos
• Abstract summary: We study neural networks based on the morphological operators of dilation and erosion. Our contributions include the training of morphological networks via Difference-of-Convex programming methods and extend a binary morphological to multiclass tasks.
• Score: 40.327435646554115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we study an emerging class of neural networks based on the morphological operators of dilation and erosion. We explore these networks mathematically from a tropical geometry perspective as well as mathematical morphology. Our contributions are threefold. First, we examine the training of morphological networks via Difference-of-Convex programming methods and extend a binary morphological classifier to multiclass tasks. Second, we focus on the sparsity of dense morphological networks trained via gradient descent algorithms and compare their performance to their linear counterparts under heavy pruning, showing that the morphological networks cope far better and are characterized with superior compression capabilities. Our approach incorporates the effect of the training optimizer used and offers quantitative and qualitative explanations. Finally, we study how the architectural structure of a morphological network can affect shape constraints, focusing on monotonicity. Via Maslov Dequantization, we obtain a softened version of a known architecture and show how this approach can improve training convergence and performance.

Related papers

• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Tropical Expressivity of Neural Networks [0.0]
  We use tropical geometry to characterize and study various architectural aspects of neural networks. We present a new algorithm that computes the exact number of their linear regions.
  arXiv  Detail & Related papers  (2024-05-30T15:45:03Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Training morphological neural networks with gradient descent: some theoretical insights [0.40792653193642503]
  We investigate the potential and limitations of differentiation based approaches and back-propagation applied to morphological networks. We provide insights and first theoretical guidelines, in particular regarding learning rates.
  arXiv  Detail & Related papers  (2024-02-05T12:11:15Z)
• Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics [1.0154623955833253]
  We demonstrate three types of neural network architectures for efficient learning of deformations of solid bodies. The first two are based on the recently proposed CNN U-NET and MAgNET frameworks which have shown promising performance for learning on mesh-based data. The third architecture is Perceiver IO, a very recent architecture that belongs to the family of attention-based neural networks.
  arXiv  Detail & Related papers  (2022-12-01T13:10:56Z)
• Localized Persistent Homologies for more Effective Deep Learning [60.78456721890412]
  We introduce an approach that relies on a new filtration function to account for location during network training. We demonstrate experimentally on 2D images of roads and 3D image stacks of neuronal processes that networks trained in this manner are better at recovering the topology of the curvilinear structures they extract.
  arXiv  Detail & Related papers  (2021-10-12T19:28:39Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Neural networks adapting to datasets: learning network size and topology [77.34726150561087]
  We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a gradient-based training. The resulting network has the structure of a graph tailored to the particular learning task and dataset.
  arXiv  Detail & Related papers  (2020-06-22T12:46:44Z)
• Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers [18.22501196339569]
  We present a novel network pruning algorithm called Dynamic Sparse Training that can jointly find the optimal network parameters and sparse network structure. We demonstrate that our dynamic sparse training algorithm can easily train very sparse neural network models with little performance loss.
  arXiv  Detail & Related papers  (2020-05-14T11:05:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.