Deep Neural Networks and PIDE discretizations

• URL: http://arxiv.org/abs/2108.02430v1
• Date: Thu, 5 Aug 2021 08:03:01 GMT
• Title: Deep Neural Networks and PIDE discretizations
• Authors: Bastian Bohn, Michael Griebel, Dinesh Kannan
• Abstract summary: We propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN) We propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and fractional inverse Laplacian operators. We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving.
• Score: 2.4063592468412276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN). As an alternative to increasing the network's depth or width to improve performance, we propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and inverse fractional Laplacian operators that arise in several problems in the physical sciences. The forward propagation of such networks is inspired by partial integro-differential equations (PIDEs). We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving. Moreover, we investigate the extra computational costs of these dense operators and the stability of forward propagation of the proposed neural networks.

Related papers

• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Network Inversion of Binarised Neural Nets [3.5571131514746837]
  Network inversion plays a pivotal role in unraveling the black-box nature of input to output mappings in neural networks. This paper introduces a novel approach to invert a trained BNN by encoding it into a CNF formula that captures the network's structure.
  arXiv  Detail & Related papers  (2024-02-19T09:39:54Z)
• Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss [2.07180164747172]
  We compare deeper neural networks (DeNNs) with a flexible number of layers and wider neural networks (WeNNs) with limited hidden layers. We find that a higher number of parameters tends to favor WeNNs, while an increased number of sample points and greater regularity in the loss function lean towards the adoption of DeNNs.
  arXiv  Detail & Related papers  (2024-01-31T20:10:10Z)
• Addressing caveats of neural persistence with deep graph persistence [54.424983583720675]
  We find that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. We propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues.
  arXiv  Detail & Related papers  (2023-07-20T13:34:11Z)
• Interpretable Neural Networks with Random Constructive Algorithm [3.1200894334384954]
  This paper introduces an Interpretable Neural Network (INN) incorporating spatial information to tackle the opaque parameterization process of random weighted neural networks. It devises a geometric relationship strategy using a pool of candidate nodes and established relationships to select node parameters conducive to network convergence.
  arXiv  Detail & Related papers  (2023-07-01T01:07:20Z)
• Certified Invertibility in Neural Networks via Mixed-Integer Programming [16.64960701212292]
  Neural networks are known to be vulnerable to adversarial attacks. There may exist large, meaningful perturbations that do not affect the network's decision. We discuss how our findings can be useful for invertibility certification in transformations between neural networks.
  arXiv  Detail & Related papers  (2023-01-27T15:40:38Z)
• Deep Neural Networks as Complex Networks [1.704936863091649]
  We use Complex Network Theory to represent Deep Neural Networks (DNNs) as directed weighted graphs. We introduce metrics to study DNNs as dynamical systems, with a granularity that spans from weights to layers, including neurons. We show that our metrics discriminate low vs. high performing networks.
  arXiv  Detail & Related papers  (2022-09-12T16:26:04Z)
• Learning Autonomy in Management of Wireless Random Networks [102.02142856863563]
  This paper presents a machine learning strategy that tackles a distributed optimization task in a wireless network with an arbitrary number of randomly interconnected nodes. We develop a flexible deep neural network formalism termed distributed message-passing neural network (DMPNN) with forward and backward computations independent of the network topology.
  arXiv  Detail & Related papers  (2021-06-15T09:03:28Z)
• Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity [87.32169414230822]
  Recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) In this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs.
  arXiv  Detail & Related papers  (2020-08-21T19:03:23Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.