Training Deep Morphological Neural Networks as Universal Approximators

• URL: http://arxiv.org/abs/2505.09710v1
• Date: Wed, 14 May 2025 18:10:49 GMT
• Title: Training Deep Morphological Neural Networks as Universal Approximators
• Authors: Konstantinos Fotopoulos, Petros Maragos,
• Abstract summary: We investigate deep morphological neural networks (DMNNs)<n>We demonstrate that activations between layers are essential for DMNNs.<n>We propose several new architectures for DMNNs, each with a different constraint on their parameters.
• Score: 23.122183338862687
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate deep morphological neural networks (DMNNs). We demonstrate that despite their inherent non-linearity, activations between layers are essential for DMNNs. We then propose several new architectures for DMNNs, each with a different constraint on their parameters. For the first (resp. second) architecture, we work under the constraint that the majority of parameters (resp. learnable parameters) should be part of morphological operations. We empirically show that our proposed networks can be successfully trained, and are more prunable than linear networks. To the best of our knowledge, we are the first to successfully train DMNNs under such constraints, although the generalization capabilities of our networks remain limited. Finally, we propose a hybrid network architecture combining linear and morphological layers, showing empirically that the inclusion of morphological layers significantly accelerates the convergence of gradient descent with large batches.

Related papers

• Sparse Hybrid Linear-Morphological Networks [22.57224128086205]
  We propose a hybrid network structure, wherein morphological layers are inserted between the linear layers of the network.<n>We conduct experiments on the Magna-Tag-A-Tune (music auto-tagging) and CIFAR-10 (image classification) datasets.<n>We demonstrate that these networks induce sparsity to their linear layers, making them more prunable under L1 unstructured pruning.
  arXiv  Detail & Related papers  (2025-04-12T17:19:46Z)
• Convexity in ReLU Neural Networks: beyond ICNNs? [17.01649106055384]
  We show that every convex function implemented by a 1-hidden-layer ReLU network can be expressed by an ICNN with the same architecture.<n>We also provide a numerical procedure that allows an exact check of convexity for ReLU neural networks with a large number of affine regions.
  arXiv  Detail & Related papers  (2025-01-06T13:53:59Z)
• Residual Kolmogorov-Arnold Network for Enhanced Deep Learning [0.5852077003870417]
  We introduce RKAN (Residual Kolmogorov-Arnold Network), which could be easily implemented into stages of traditional networks.<n>Our proposed RKAN module offers consistent improvements over the base models on various well-known benchmark datasets.
  arXiv  Detail & Related papers  (2024-10-07T21:12:32Z)
• Principled Architecture-aware Scaling of Hyperparameters [69.98414153320894]
  Training a high-quality deep neural network requires choosing suitable hyperparameters, which is a non-trivial and expensive process. In this work, we precisely characterize the dependence of initializations and maximal learning rates on the network architecture. We demonstrate that network rankings can be easily changed by better training networks in benchmarks.
  arXiv  Detail & Related papers  (2024-02-27T11:52:49Z)
• Graph Metanetworks for Processing Diverse Neural Architectures [33.686728709734105]
  Graph Metanetworks (GMNs) generalizes to neural architectures where competing methods struggle. We prove that GMNs are expressive and equivariant to parameter permutation symmetries that leave the input neural network functions.
  arXiv  Detail & Related papers  (2023-12-07T18:21:52Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• On the Application of Data-Driven Deep Neural Networks in Linear and Nonlinear Structural Dynamics [28.979990729816638]
  The use of deep neural network (DNN) models as surrogates for linear and nonlinear structural dynamical systems is explored. The focus is on the development of efficient network architectures using fully-connected, sparsely-connected, and convolutional network layers. It is shown that the proposed DNNs can be used as effective and accurate surrogates for predicting linear and nonlinear dynamical responses under harmonic loadings.
  arXiv  Detail & Related papers  (2021-11-03T13:22:19Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives [73.15276998621582]
  We propose a generic feature learning mechanism to advance CNN training with enhanced generalization ability. Partially inspired by DSN, we fork delicately designed side branches from the intermediate layers of a given neural network. Experiments on both category and instance recognition tasks demonstrate the substantial improvements of our proposed method.
  arXiv  Detail & Related papers  (2020-03-24T09:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.