Quantum-Inspired Tensor Neural Networks for Partial Differential Equations

• URL: http://arxiv.org/abs/2208.02235v1
• Date: Wed, 3 Aug 2022 17:41:11 GMT
• Title: Quantum-Inspired Tensor Neural Networks for Partial Differential Equations
• Authors: Raj Patel, Chia-Wei Hsing, Serkan Sahin, Saeed S. Jahromi, Samuel Palmer, Shivam Sharma, Christophe Michel, Vincent Porte, Mustafa Abid, Stephane Aubert, Pierre Castellani, Chi-Guhn Lee, Samuel Mugel, Roman Orus
• Abstract summary: Deep learning methods are constrained by training time and memory. To tackle these shortcomings, we implement Neural Networks (TNN) We demonstrate that TNN provide significant parameter savings while attaining the same accuracy as compared to the classical Neural Network (DNN)
• Score: 5.963563752404561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Partial Differential Equations (PDEs) are used to model a variety of dynamical systems in science and engineering. Recent advances in deep learning have enabled us to solve them in a higher dimension by addressing the curse of dimensionality in new ways. However, deep learning methods are constrained by training time and memory. To tackle these shortcomings, we implement Tensor Neural Networks (TNN), a quantum-inspired neural network architecture that leverages Tensor Network ideas to improve upon deep learning approaches. We demonstrate that TNN provide significant parameter savings while attaining the same accuracy as compared to the classical Dense Neural Network (DNN). In addition, we also show how TNN can be trained faster than DNN for the same accuracy. We benchmark TNN by applying them to solve parabolic PDEs, specifically the Black-Scholes-Barenblatt equation, widely used in financial pricing theory, empirically showing the advantages of TNN over DNN. Further examples, such as the Hamilton-Jacobi-Bellman equation, are also discussed.

Related papers

• Quantum-Inspired Tensor Neural Networks for Option Pricing [4.3942901219301564]
  Recent advances in deep learning have enabled us to address the curse of dimensionality (COD) by solving problems in higher dimensions. A subset of such approaches of addressing the COD has led us to solving high-dimensional PDEs. This has resulted in opening doors to solving a variety of real-world problems ranging from mathematical finance to control for industrial applications.
  arXiv  Detail & Related papers  (2022-12-28T19:39:55Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Anisotropic, Sparse and Interpretable Physics-Informed Neural Networks for PDEs [0.0]
  We present ASPINN, an anisotropic extension of our earlier work called SPINN--Sparse, Physics-informed, and Interpretable Neural Networks--to solve PDEs. ASPINNs generalize radial basis function networks. We also streamline the training of ASPINNs into a form that is closer to that of supervised learning algorithms.
  arXiv  Detail & Related papers  (2022-07-01T12:24:43Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Learning Trajectories of Hamiltonian Systems with Neural Networks [81.38804205212425]
  We propose to enhance Hamiltonian neural networks with an estimation of a continuous-time trajectory of the modeled system. We demonstrate that the proposed integration scheme works well for HNNs, especially with low sampling rates, noisy and irregular observations.
  arXiv  Detail & Related papers  (2022-04-11T13:25:45Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• A Time Encoding approach to training Spiking Neural Networks [3.655021726150368]
  Spiking Neural Networks (SNNs) have been gaining in popularity. In this paper, we provide an extra tool to help us understand and train SNNs by using theory from the field of time encoding.
  arXiv  Detail & Related papers  (2021-10-13T14:07:11Z)
• A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs [2.28438857884398]
  Deep neural networks (DNNs) in the infinite width/channel limit have received much attention recently. Despite their theoretical appeal, this viewpoint lacks a crucial ingredient of deep learning in finite DNNs, laying at the heart of their success -- feature learning. Here we consider DNNs trained with noisy gradient descent on a large training set and derive a self consistent Gaussian Process theory accounting for strong finite-DNN and feature learning effects.
  arXiv  Detail & Related papers  (2021-06-08T05:20:00Z)
• Block-term Tensor Neural Networks [29.442026567710435]
  We show that block-term tensor layers (BT-layers) can be easily adapted to neural network models, such as CNNs and RNNs. BT-layers in CNNs and RNNs can achieve a very large compression ratio on the number of parameters while preserving or improving the representation power of the original DNNs.
  arXiv  Detail & Related papers  (2020-10-10T09:58:43Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.