Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations

• URL: http://arxiv.org/abs/2302.03286v2
• Date: Wed, 29 May 2024 12:22:15 GMT
• Title: Algorithmically Designed Artificial Neural Networks (ADANNs): Higher order deep operator learning for parametric partial differential equations
• Authors: Arnulf Jentzen, Adrian Riekert, Philippe von Wurstemberger,
• Abstract summary: We propose a new deep learning approach to approximate operators related to partial parametric differential equations (PDEs) In the proposed approach we combine efficient classical numerical approximation techniques with deep operator learning methodologies. We numerically test the proposed ADANN methodology in the case of several parametric PDEs.
• Score: 5.052293146674794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this article we propose a new deep learning approach to approximate operators related to parametric partial differential equations (PDEs). In particular, we introduce a new strategy to design specific artificial neural network (ANN) architectures in conjunction with specific ANN initialization schemes which are tailor-made for the particular approximation problem under consideration. In the proposed approach we combine efficient classical numerical approximation techniques with deep operator learning methodologies. Specifically, we introduce customized adaptions of existing ANN architectures together with specialized initializations for these ANN architectures so that at initialization we have that the ANNs closely mimic a chosen efficient classical numerical algorithm for the considered approximation problem. The obtained ANN architectures and their initialization schemes are thus strongly inspired by numerical algorithms as well as by popular deep learning methodologies from the literature and in that sense we refer to the introduced ANNs in conjunction with their tailor-made initialization schemes as Algorithmically Designed Artificial Neural Networks (ADANNs). We numerically test the proposed ADANN methodology in the case of several parametric PDEs. In the tested numerical examples the ADANN methodology significantly outperforms existing traditional approximation algorithms as well as existing deep operator learning methodologies from the literature.

Related papers

• An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning [5.75055574132362]
  approximation of solutions of partial differential equations with numerical algorithms is a central topic in applied mathematics. One class of methods which has received a lot of attention in recent years are machine learning-based methods. This article aims to provide an introduction to some of these methods and the mathematical theory on which they are based.
  arXiv  Detail & Related papers  (2024-08-23T16:57:34Z)
• Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory [4.066869900592636]
  This book aims to provide an introduction to the topic of deep learning algorithms. We review essential components of deep learning algorithms in full mathematical detail.
  arXiv  Detail & Related papers  (2023-10-31T11:01:23Z)
• Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles [0.8258451067861933]
  We present an automated approach to deep neural network (DNN) discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. We demonstrate the feasibility of this framework for two tasks - forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature.
  arXiv  Detail & Related papers  (2023-02-20T03:57:06Z)
• Towards Theoretically Inspired Neural Initialization Optimization [66.04735385415427]
  We propose a differentiable quantity, named GradCosine, with theoretical insights to evaluate the initial state of a neural network. We show that both the training and test performance of a network can be improved by maximizing GradCosine under norm constraint. Generalized from the sample-wise analysis into the real batch setting, NIO is able to automatically look for a better initialization with negligible cost.
  arXiv  Detail & Related papers  (2022-10-12T06:49:16Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization [101.32332941117271]
  Decision making algorithms are used in a multitude of different applications. Deep learning approaches that use highly parametric architectures tuned from data without relying on mathematical models are becoming increasingly popular. Model-based optimization and data-centric deep learning are often considered to be distinct disciplines.
  arXiv  Detail & Related papers  (2022-05-05T13:40:08Z)
• Neural Combinatorial Optimization: a New Player in the Field [69.23334811890919]
  This paper presents a critical analysis on the incorporation of algorithms based on neural networks into the classical optimization framework. A comprehensive study is carried out to analyse the fundamental aspects of such algorithms, including performance, transferability, computational cost and to larger-sized instances.
  arXiv  Detail & Related papers  (2022-05-03T07:54:56Z)
• Personalized Algorithm Generation: A Case Study in Meta-Learning ODE Integrators [6.457555233038933]
  We study the meta-learning of numerical algorithms for scientific computing. We develop a machine learning approach that automatically learns solvers for initial value problems.
  arXiv  Detail & Related papers  (2021-05-04T05:42:33Z)
• Model-Based Machine Learning for Communications [110.47840878388453]
  We review existing strategies for combining model-based algorithms and machine learning from a high level perspective. We focus on symbol detection, which is one of the fundamental tasks of communication receivers.
  arXiv  Detail & Related papers  (2021-01-12T19:55:34Z)
• Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems [59.804810122136345]
  We propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed. An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed. We show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.
  arXiv  Detail & Related papers  (2020-06-15T02:57:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.