Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery

• URL: http://arxiv.org/abs/2505.23106v1
• Date: Thu, 29 May 2025 05:18:30 GMT
• Title: Neural Interpretable PDEs: Harmonizing Fourier Insights with Attention for Scalable and Interpretable Physics Discovery
• Authors: Ning Liu, Yue Yu,
• Abstract summary: We introduce Neural Interpretable PDEs (NIPS), a novel neural operator architecture that builds upon and enhances Nonlocal Attention Operators (NAO)<n>NIPS employs a linear attention mechanism to enable scalable learning and integrates a learnable kernel network that acts as a channel-independent convolution in Fourier space.<n> Empirical evaluations demonstrate that NIPS consistently surpasses NAO and other baselines across diverse benchmarks.
• Score: 15.29112632863168
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Attention mechanisms have emerged as transformative tools in core AI domains such as natural language processing and computer vision. Yet, their largely untapped potential for modeling intricate physical systems presents a compelling frontier. Learning such systems often entails discovering operators that map between functional spaces using limited instances of function pairs -- a task commonly framed as a severely ill-posed inverse PDE problem. In this work, we introduce Neural Interpretable PDEs (NIPS), a novel neural operator architecture that builds upon and enhances Nonlocal Attention Operators (NAO) in both predictive accuracy and computational efficiency. NIPS employs a linear attention mechanism to enable scalable learning and integrates a learnable kernel network that acts as a channel-independent convolution in Fourier space. As a consequence, NIPS eliminates the need to explicitly compute and store large pairwise interactions, effectively amortizing the cost of handling spatial interactions into the Fourier transform. Empirical evaluations demonstrate that NIPS consistently surpasses NAO and other baselines across diverse benchmarks, heralding a substantial leap in scalable, interpretable, and efficient physics learning. Our code and data accompanying this paper are available at https://github.com/fishmoon1234/Nonlocal-Attention-Operator.

Related papers

• Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning [78.88684753303794]
  Deep learning has predominantly advanced through applications in computer vision and natural language processing.<n>Neural operators are a principled way to generalize neural networks to mappings between function spaces.<n>This paper identifies and distills the key principles for constructing practical implementations of mappings between infinite-dimensional function spaces.
  arXiv  Detail & Related papers  (2025-06-12T17:59:31Z)
• DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [60.58067866537143]
  We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis.<n>To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers.<n> Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
  arXiv  Detail & Related papers  (2024-10-08T10:48:50Z)
• Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discovery [25.75410883895742]
  We propose a novel neural operator architecture based on the attention mechanism, which we coin Nonlocal Attention Operator (NAO) NAO can address ill-posedness and rank deficiency in inverse PDE problems by encoding regularization and achieving generalizability.
  arXiv  Detail & Related papers  (2024-08-14T05:57:56Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Inferring Relational Potentials in Interacting Systems [56.498417950856904]
  We propose Neural Interaction Inference with Potentials (NIIP) as an alternative approach to discover such interactions. NIIP assigns low energy to the subset of trajectories which respect the relational constraints observed. It allows trajectory manipulation, such as interchanging interaction types across separately trained models, as well as trajectory forecasting.
  arXiv  Detail & Related papers  (2023-10-23T00:44:17Z)
• Convolutional Neural Operators for robust and accurate learning of PDEs [11.562748612983956]
  We present novel adaptations for convolutional neural networks to process functions as inputs and outputs. The resulting architecture is termed as convolutional neural operators (CNOs) We prove a universality theorem to show that CNOs can approximate operators arising in PDEs to desired accuracy.
  arXiv  Detail & Related papers  (2023-02-02T15:54:45Z)
• ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware [105.54048699217668]
  We show a competitive performance in accuracy with a clear advantage in the computational complexity for Event-Based Three-factor Local Plasticity (ETLP) We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learntemporal patterns with a rich temporal structure.
  arXiv  Detail & Related papers  (2023-01-19T19:45:42Z)
• DOSnet as a Non-Black-Box PDE Solver: When Deep Learning Meets Operator Splitting [12.655884541938656]
  We develop a learning-based PDE solver, which we name Deep Operator-Splitting Network (DOSnet) DOSnet is constructed from the physical rules and operators governing the underlying dynamics contains learnable parameters. We train and validate it on several types of operator-decomposable differential equations.
  arXiv  Detail & Related papers  (2022-12-11T18:23:56Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks [0.0]
  We show that neural networks (PINNs) struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. We construct novel architectures that employ multi-scale random observational features and justify how such coordinate embedding layers can lead to robust and accurate PINN models.
  arXiv  Detail & Related papers  (2020-12-18T04:19:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.