Deep Parallel Spectral Neural Operators for Solving Partial Differential Equations with Enhanced Low-Frequency Learning Capability

• URL: http://arxiv.org/abs/2409.19976v4
• Date: Fri, 21 Feb 2025 05:53:49 GMT
• Title: Deep Parallel Spectral Neural Operators for Solving Partial Differential Equations with Enhanced Low-Frequency Learning Capability
• Authors: Qinglong Ma, Peizhi Zhao, Sen Wang, Tao Song,
• Abstract summary: We propose a Deep Parallel Spectral Neural Operator (DPNO) to enhance the ability to learn low-frequency information.<n>Our method enhances the neural operator's ability to learn low-frequency information through parallel modules.<n>We smooth this information through convolutional mappings, thereby reducing high-frequency errors.
• Score: 11.121415128908566
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Designing universal artificial intelligence (AI) solver for partial differential equations (PDEs) is an open-ended problem and a significant challenge in science and engineering. Currently, data-driven solvers have achieved great success, such as neural operators. However, the ability of various neural operator solvers to learn low-frequency information still needs improvement. In this study, we propose a Deep Parallel Spectral Neural Operator (DPNO) to enhance the ability to learn low-frequency information. Our method enhances the neural operator's ability to learn low-frequency information through parallel modules. In addition, due to the presence of truncation coefficients, some high-frequency information is lost during the nonlinear learning process. We smooth this information through convolutional mappings, thereby reducing high-frequency errors. We selected several challenging partial differential equation datasets for experimentation, and DPNO performed exceptionally well. As a neural operator, DPNO also possesses the capability of resolution invariance.

Related papers

• LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators [20.77877474840923]
  High-frequency information is a critical challenge in machine learning. Deep neural nets exhibit the so-called spectral bias toward learning low-frequency components. We propose a novel frequency-sensitive loss term based on radially binned spectral errors.
  arXiv  Detail & Related papers  (2025-04-05T19:35:04Z)
• Dilated convolution neural operator for multiscale partial differential equations [11.093527996062058]
  We propose the Dilated Convolutional Neural Operator (DCNO) for multiscale partial differential equations. The DCNO architecture effectively captures both high-frequency and low-frequency features while maintaining a low computational cost. We show that DCNO strikes an optimal balance between accuracy and computational cost and offers a promising solution for multiscale operator learning.
  arXiv  Detail & Related papers  (2024-07-16T08:17:02Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• PICL: Physics Informed Contrastive Learning for Partial Differential Equations [7.136205674624813]
  We develop a novel contrastive pretraining framework that improves neural operator generalization across multiple governing equations simultaneously. A combination of physics-informed system evolution and latent-space model output are anchored to input data and used in our distance function. We find that physics-informed contrastive pretraining improves accuracy for the Fourier Neural Operator in fixed-future and autoregressive rollout tasks for the 1D and 2D Heat, Burgers', and linear advection equations.
  arXiv  Detail & Related papers  (2024-01-29T17:32:22Z)
• Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning [0.0]
  Physics-informed neural network (PINN) is a data-driven solver for partial and ordinary differential equations(ODEs/PDEs) We propose using transfer learning to boost the robustness and convergence of training PINN. We elaborately described our training strategy, including selection, and suggested guidelines for using transfer learning to train neural networks for solving more complex problems.
  arXiv  Detail & Related papers  (2024-01-05T13:45:08Z)
• D2NO: Efficient Handling of Heterogeneous Input Function Spaces with Distributed Deep Neural Operators [7.119066725173193]
  We propose a novel distributed approach to deal with input functions that exhibit heterogeneous properties. A central neural network is used to handle shared information across all output functions. We demonstrate that the corresponding neural network is a universal approximator of continuous nonlinear operators.
  arXiv  Detail & Related papers  (2023-10-29T03:29:59Z)
• Hyena Neural Operator for Partial Differential Equations [9.438207505148947]
  Recent advances in deep learning have provided a new approach to solving partial differential equations that involves the use of neural operators. This study utilizes a neural operator called Hyena, which employs a long convolutional filter that is parameterized by a multilayer perceptron. Our findings indicate Hyena can serve as an efficient and accurate model for partial learning differential equations solution operator.
  arXiv  Detail & Related papers  (2023-06-28T19:45:45Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs [86.35471039808023]
  We introduce the Incremental Fourier Neural Operator (iFNO), which progressively increases the number of frequency modes used by the model. We show that iFNO reduces total training time while maintaining or improving generalization performance across various datasets. Our method demonstrates a 10% lower testing error, using 20% fewer frequency modes compared to the existing Fourier Neural Operator, while also achieving a 30% faster training.
  arXiv  Detail & Related papers  (2022-11-28T09:57:15Z)
• Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport [2.512625172084287]
  We develop a multifidelity neural operator based on a deep operator network (DeepONet) A multifidelity DeepONet significantly reduces the required amount of high-fidelity data and achieves one order of magnitude smaller error when using the same amount of high-fidelity data. We apply a multifidelity DeepONet to learn the phonon Boltzmann transport equation (BTE), a framework to compute nanoscale heat transport.
  arXiv  Detail & Related papers  (2022-04-14T01:01:24Z)
• Pseudo-Differential Neural Operator: Generalized Fourier Neural Operator for Learning Solution Operators of Partial Differential Equations [14.43135909469058]
  We propose a novel textitpseudo-differential integral operator (PDIO) to analyze and generalize the Fourier integral operator in FNO. We experimentally validate the effectiveness of the proposed model by utilizing Darcy flow and the Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-01-28T07:22:32Z)
• Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State [66.2457134675891]
  Spiking neural networks (SNNs) are brain-inspired models that enable energy-efficient implementation on neuromorphic hardware. Most existing methods imitate the backpropagation framework and feedforward architectures for artificial neural networks. We propose a novel training method that does not rely on the exact reverse of the forward computation.
  arXiv  Detail & Related papers  (2021-09-29T07:46:54Z)
• Neural Operator: Learning Maps Between Function Spaces [75.93843876663128]
  We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations.
  arXiv  Detail & Related papers  (2021-08-19T03:56:49Z)
• Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics [73.77459272878025]
  We propose to enhance the supervised signal in learning dynamics by pre-training a neural differential operator (NDO) NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives. We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library.
  arXiv  Detail & Related papers  (2021-06-08T08:04:47Z)
• Conditional physics informed neural networks [85.48030573849712]
  We introduce conditional PINNs (physics informed neural networks) for estimating the solution of classes of eigenvalue problems. We show that a single deep neural network can learn the solution of partial differential equations for an entire class of problems.
  arXiv  Detail & Related papers  (2021-04-06T18:29:14Z)
• Fourier Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  We formulate a new neural operator by parameterizing the integral kernel directly in Fourier space. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. It is up to three orders of magnitude faster compared to traditional PDE solvers.
  arXiv  Detail & Related papers  (2020-10-18T00:34:21Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Neural Operator: Graph Kernel Network for Partial Differential Equations [57.90284928158383]
  This work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators) We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
  arXiv  Detail & Related papers  (2020-03-07T01:56:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.