Towards Universal Neural Operators through Multiphysics Pretraining

• URL: http://arxiv.org/abs/2511.10829v1
• Date: Thu, 13 Nov 2025 22:04:38 GMT
• Title: Towards Universal Neural Operators through Multiphysics Pretraining
• Authors: Mikhail Masliaev, Dmitry Gusarov, Ilya Markov, Alexander Hvatov,
• Abstract summary: We investigate transformer-based neural operators, which have previously been applied only to specific problems.<n>We evaluate their performance across diverse PDE problems, including extrapolation to unseen parameters, incorporation of new variables, and transfer from datasets.
• Score: 40.321164373223475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although neural operators are widely used in data-driven physical simulations, their training remains computationally expensive. Recent advances address this issue via downstream learning, where a model pretrained on simpler problems is fine-tuned on more complex ones. In this research, we investigate transformer-based neural operators, which have previously been applied only to specific problems, in a more general transfer learning setting. We evaluate their performance across diverse PDE problems, including extrapolation to unseen parameters, incorporation of new variables, and transfer from multi-equation datasets. Our results demonstrate that advanced neural operator architectures can effectively transfer knowledge across PDE problems.

Related papers

• Convolutional-neural-operator-based transfer learning for solving PDEs [0.4125187280299247]
  Convolutional neural operator is a CNN-based architecture recently proposed to enforce structure-preserving continuous-discrete equivalence.<n>We extend the model to few-shot learning scenarios by first pre-training a convolutional neural operator using a source dataset.<n>We find that the neuron linear transformation strategy enjoys the highest surrogate accuracy in solving PDEs.
  arXiv  Detail & Related papers  (2025-12-19T03:55:20Z)
• Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning [78.88684753303794]
  Deep learning has predominantly advanced through applications in computer vision and natural language processing.<n>Neural operators are a principled way to generalize neural networks to mappings between function spaces.<n>This paper identifies and distills the key principles for constructing practical implementations of mappings between infinite-dimensional function spaces.
  arXiv  Detail & Related papers  (2025-06-12T17:59:31Z)
• Pseudo-Physics-Informed Neural Operators: Enhancing Operator Learning from Limited Data [17.835190275166408]
  We propose the Pseudo Physics-Informed Neural Operator (PPI-NO) framework.<n> PPI-NO constructs a surrogate physics system for the target system using partial differential equations (PDEs) derived from basic differential operators.<n>This framework significantly improves the accuracy of standard operator learning models in data-scarce scenarios.
  arXiv  Detail & Related papers  (2025-02-04T19:50:06Z)
• DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning [6.635683993472882]
  We propose a novel fine-tuning method to achieve multi-operator learning. Our approach combines distributed learning to integrate data from various operators in pre-training, while physics-informed methods enable zero-shot fine-tuning.
  arXiv  Detail & Related papers  (2024-11-11T18:58:46Z)
• DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving [54.605760146540234]
  DeltaPhi is a novel learning framework that transforms the PDE solving task from learning direct input-output mappings to learning the residuals between similar physical states.<n>Extensive experiments demonstrate consistent and significant improvements across diverse physical systems.
  arXiv  Detail & Related papers  (2024-06-14T07:45:07Z)
• Neural Operators for Accelerating Scientific Simulations and Design [85.89660065887956]
  An AI framework, known as Neural Operators, presents a principled framework for learning mappings between functions defined on continuous domains. Neural Operators can augment or even replace existing simulators in many applications, such as computational fluid dynamics, weather forecasting, and material modeling.
  arXiv  Detail & Related papers  (2023-09-27T00:12:07Z)
• GNOT: A General Neural Operator Transformer for Operator Learning [34.79481320566005]
  General neural operator transformer (GNOT) is a scalable and effective framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems.
  arXiv  Detail & Related papers  (2023-02-28T07:58:49Z)
• Physics-guided Data Augmentation for Learning the Solution Operator of Linear Differential Equations [2.1850269949775663]
  We propose a physics-guided data augmentation (PGDA) method to improve the accuracy and generalization of neural operator models. We demonstrate the advantage of PGDA on a variety of linear differential equations, showing that PGDA can improve the sample complexity and is robust to distributional shift.
  arXiv  Detail & Related papers  (2022-12-08T06:29:15Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.