Neural Network Representation of Time Integrators

• URL: http://arxiv.org/abs/2211.17039v1
• Date: Wed, 30 Nov 2022 14:38:59 GMT
• Title: Neural Network Representation of Time Integrators
• Authors: Rainald L\"ohner and Harbir Antil
• Abstract summary: The network weights and biases are given, i.e., no training is needed. The architecture required for the integration of a simple mass-damper-stiffness case is included as an example.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural network (DNN) architectures are constructed that are the exact equivalent of explicit Runge-Kutta schemes for numerical time integration. The network weights and biases are given, i.e., no training is needed. In this way, the only task left for physics-based integrators is the DNN approximation of the right-hand side. This allows to clearly delineate the approximation estimates for right-hand side errors and time integration errors. The architecture required for the integration of a simple mass-damper-stiffness case is included as an example.

Related papers

• Neural Control Variates with Automatic Integration [49.91408797261987]
  This paper proposes a novel approach to construct learnable parametric control variates functions from arbitrary neural network architectures. We use the network to approximate the anti-derivative of the integrand. We apply our method to solve partial differential equations using the Walk-on-sphere algorithm.
  arXiv  Detail & Related papers  (2024-09-23T06:04:28Z)
• Algebraic Representations for Faster Predictions in Convolutional Neural Networks [0.0]
  Convolutional neural networks (CNNs) are a popular choice of model for tasks in computer vision. skip connections may be added to create an easier gradient optimization problem. We show that arbitrarily complex, trained, linear CNNs with skip connections can be simplified into a single-layer model.
  arXiv  Detail & Related papers  (2024-08-14T21:10:05Z)
• Time Elastic Neural Networks [2.1756081703276]
  We introduce and detail an atypical neural network architecture, called time elastic neural network (teNN) The novelty compared to classical neural network architecture is that it explicitly incorporates time warping ability. We demonstrate that, during the training process, the teNN succeeds in reducing the number of neurons required within each cell.
  arXiv  Detail & Related papers  (2024-05-27T09:01:30Z)
• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch [72.26822499434446]
  Auto-Train-Once (ATO) is an innovative network pruning algorithm designed to automatically reduce the computational and storage costs of DNNs. We provide a comprehensive convergence analysis as well as extensive experiments, and the results show that our approach achieves state-of-the-art performance across various model architectures.
  arXiv  Detail & Related papers  (2024-03-21T02:33:37Z)
• Training Integer-Only Deep Recurrent Neural Networks [3.1829446824051195]
  We present a quantization-aware training method for obtaining a highly accurate integer-only recurrent neural network (iRNN) Our approach supports layer normalization, attention, and an adaptive piecewise linear (PWL) approximation of activation functions. The proposed method enables RNN-based language models to run on edge devices with $2times$ improvement in runtime.
  arXiv  Detail & Related papers  (2022-12-22T15:22:36Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• A quantum algorithm for training wide and deep classical neural networks [72.2614468437919]
  We show that conditions amenable to classical trainability via gradient descent coincide with those necessary for efficiently solving quantum linear systems. We numerically demonstrate that the MNIST image dataset satisfies such conditions. We provide empirical evidence for $O(log n)$ training of a convolutional neural network with pooling.
  arXiv  Detail & Related papers  (2021-07-19T23:41:03Z)
• Piecewise-constant Neural ODEs [41.116259317376475]
  We make a piecewise-constant approximation to Neural ODEs to mitigate these issues. Our model can be integrated exactly via Euler integration and can generate autoregressive samples in 3-20 times fewer steps than comparable RNN and ODE-RNN models.
  arXiv  Detail & Related papers  (2021-06-11T21:46:55Z)
• AutoInt: Automatic Integration for Fast Neural Volume Rendering [51.46232518888791]
  We propose a new framework for learning efficient, closed-form solutions to integrals using implicit neural representation networks. We demonstrate a greater than 10x improvement in photorealistic requirements, enabling fast neural volume rendering.
  arXiv  Detail & Related papers  (2020-12-03T05:46:10Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.