Spectral Informed Neural Network: An Efficient and Low-Memory PINN

• URL: http://arxiv.org/abs/2408.16414v2
• Date: Tue, 8 Oct 2024 13:27:38 GMT
• Title: Spectral Informed Neural Network: An Efficient and Low-Memory PINN
• Authors: Tianchi Yu, Yiming Qi, Ivan Oseledets, Shiyi Chen,
• Abstract summary: We propose a spectral-based neural network that substitutes the differential operator with a multiplication. Compared to the PINNs, our approach requires lower memory and shorter training time. We provide two strategies to train networks by their spectral information.
• Score: 3.8534287291074354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With growing investigations into solving partial differential equations by physics-informed neural networks (PINNs), more accurate and efficient PINNs are required to meet the practical demands of scientific computing. One bottleneck of current PINNs is computing the high-order derivatives via automatic differentiation which often necessitates substantial computing resources. In this paper, we focus on removing the automatic differentiation of the spatial derivatives and propose a spectral-based neural network that substitutes the differential operator with a multiplication. Compared to the PINNs, our approach requires lower memory and shorter training time. Thanks to the exponential convergence of the spectral basis, our approach is more accurate. Moreover, to handle the different situations between physics domain and spectral domain, we provide two strategies to train networks by their spectral information. Through a series of comprehensive experiments, We validate the aforementioned merits of our proposed network.

Related papers

• Neuromorphic Wireless Split Computing with Multi-Level Spikes [69.73249913506042]
  In neuromorphic computing, spiking neural networks (SNNs) perform inference tasks, offering significant efficiency gains for workloads involving sequential data. Recent advances in hardware and software have demonstrated that embedding a few bits of payload in each spike exchanged between the spiking neurons can further enhance inference accuracy. This paper investigates a wireless neuromorphic split computing architecture employing multi-level SNNs.
  arXiv  Detail & Related papers  (2024-11-07T14:08:35Z)
• Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problems [40.46280139210502]
  A combination of multifidelity stacking PINNs and domain decomposition-based finite basis PINNs is employed. It can be observed that the domain decomposition approach clearly improves the PINN and stacking PINN approaches. It is demonstrated that the FBPINN approach can be extended to multifidelity physics-informed deep operator networks.
  arXiv  Detail & Related papers  (2024-01-15T18:32:53Z)
• Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks [4.604003661048267]
  Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. We propose a comprehensive theoretical framework that sheds light on this important problem. We derive an integro-differential equation that governs PINN prediction in the large data-set limit.
  arXiv  Detail & Related papers  (2023-07-12T18:00:02Z)
• PhyGNNet: Solving spatiotemporal PDEs with Physics-informed Graph Neural Network [12.385926494640932]
  We propose PhyGNNet for solving partial differential equations on the basics of a graph neural network. In particular, we divide the computing area into regular grids, define partial differential operators on the grids, then construct pde loss for the network to optimize to build PhyGNNet model.
  arXiv  Detail & Related papers  (2022-08-07T13:33:34Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Enforcing Continuous Physical Symmetries in Deep Learning Network for Solving Partial Differential Equations [3.6317085868198467]
  We introduce a new method, symmetry-enhanced physics informed neural network (SPINN) where the invariant surface conditions induced by the Lie symmetries of PDEs are embedded into the loss function of PINN. We show that SPINN performs better than PINN with fewer training points and simpler architecture of neural network.
  arXiv  Detail & Related papers  (2022-06-19T00:44:22Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• The Spectral Bias of Polynomial Neural Networks [63.27903166253743]
  Polynomial neural networks (PNNs) have been shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Previous studies have revealed that neural networks demonstrate a $textitspectral bias$ towards low-frequency functions, which yields faster learning of low-frequency components during training. Inspired by such studies, we conduct a spectral analysis of the Tangent Kernel (NTK) of PNNs. We find that the $Pi$-Net family, i.e., a recently proposed parametrization of PNNs, speeds up the
  arXiv  Detail & Related papers  (2022-02-27T23:12:43Z)
• Physics-informed Neural Network for Nonlinear Dynamics in Fiber Optics [10.335960060544904]
  A physics-informed neural network (PINN) that combines deep learning with physics is studied to solve the nonlinear Schr"odinger equation for learning nonlinear dynamics in fiber optics. PINN is not only an effective partial differential equation solver, but also a prospective technique to advance the scientific computing and automatic modeling in fiber optics.
  arXiv  Detail & Related papers  (2021-09-01T12:19:32Z)
• How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks [80.55378250013496]
  We study how neural networks trained by gradient descent extrapolate what they learn outside the support of the training distribution. Graph Neural Networks (GNNs) have shown some success in more complex tasks.
  arXiv  Detail & Related papers  (2020-09-24T17:48:59Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.