Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs

• URL: http://arxiv.org/abs/2403.12553v3
• Date: Fri, 01 Nov 2024 06:45:41 GMT
• Title: Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
• Authors: Md Ashiqur Rahman, Robert Joseph George, Mogab Elleithy, Daniel Leibovici, Zongyi Li, Boris Bonev, Colin White, Julius Berner, Raymond A. Yeh, Jean Kossaifi, Kamyar Azizzadenesheli, Anima Anandkumar,
• Abstract summary: We propose Codomain Attention Neural Operator (CoDA-NO) to solve multiphysics problems with PDEs. CoDA-NO tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. We find CoDA-NO to outperform existing methods by over 36% on complex downstream tasks with limited data.
• Score: 85.40198664108624
• License:
• Abstract: Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data. To address these issues, we propose Codomain Attention Neural Operator (CoDA-NO), which tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. Specifically, we extend positional encoding, self-attention, and normalization layers to function spaces. CoDA-NO can learn representations of different PDE systems with a single model. We evaluate CoDA-NO's potential as a backbone for learning multiphysics PDEs over multiple systems by considering few-shot learning settings. On complex downstream tasks with limited data, such as fluid flow simulations, fluid-structure interactions, and Rayleigh-B\'enard convection, we found CoDA-NO to outperform existing methods by over 36%.

Related papers

• Advancing Generalization in PINNs through Latent-Space Representations [71.86401914779019]
  Physics-informed neural networks (PINNs) have made significant strides in modeling dynamical systems governed by partial differential equations (PDEs) We propose PIDO, a novel physics-informed neural PDE solver designed to generalize effectively across diverse PDE configurations. We validate PIDO on a range of benchmarks, including 1D combined equations and 2D Navier-Stokes equations.
  arXiv  Detail & Related papers  (2024-11-28T13:16:20Z)
• ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations [40.6591136324878]
  We present ConDiff, a novel dataset for scientific machine learning. ConDiff focuses on the parametric diffusion equation with space dependent coefficients, a fundamental problem in many applications of partial differential equations (PDEs) This class of problems is not only of great academic interest, but is also the basis for describing various environmental and industrial problems. In this way, ConDiff shortens the gap with real-world problems while remaining fully synthetic and easy to use.
  arXiv  Detail & Related papers  (2024-06-07T07:35:14Z)
• Latent Neural PDE Solver: a reduced-order modelling framework for partial differential equations [5.949599220326208]
  We propose to learn the dynamics of the system in the latent space with much coarser discretizations. A non-linear autoencoder is first trained to project the full-order representation of the system onto the mesh-reduced space. We showcase that it has competitive accuracy and efficiency compared to the neural PDE solver that operates on full-order space.
  arXiv  Detail & Related papers  (2024-02-27T19:36:27Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• KoopmanLab: machine learning for solving complex physics equations [7.815723299913228]
  We present KoopmanLab, an efficient module of the Koopman neural operator family, for learning PDEs without analytic solutions or closed forms. Our module consists of multiple variants of the Koopman neural operator (KNO), a kind of mesh-independent neural-network-based PDE solvers. The compact variants of KNO can accurately solve PDEs with small model sizes while the large variants of KNO are more competitive in predicting highly complicated dynamic systems.
  arXiv  Detail & Related papers  (2023-01-03T13:58:39Z)
• Semi-supervised Learning of Partial Differential Operators and Dynamical Flows [68.77595310155365]
  We present a novel method that combines a hyper-network solver with a Fourier Neural Operator architecture. We test our method on various time evolution PDEs, including nonlinear fluid flows in one, two, and three spatial dimensions. The results show that the new method improves the learning accuracy at the time point of supervision point, and is able to interpolate and the solutions to any intermediate time.
  arXiv  Detail & Related papers  (2022-07-28T19:59:14Z)
• Physics-Informed Neural Operator for Learning Partial Differential Equations [55.406540167010014]
  PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families.
  arXiv  Detail & Related papers  (2021-11-06T03:41:34Z)
• One-shot learning for solution operators of partial differential equations [3.559034814756831]
  Learning and solving governing equations of a physical system, represented by partial differential equations (PDEs), from data is a central challenge in a variety of areas of science and engineering. Traditional numerical methods for solving PDEs can be computationally expensive for complex systems and require the complete PDEs of the physical system. Here, we propose the first solution operator learning method that only requires one PDE solution, i.e., one-shot learning.
  arXiv  Detail & Related papers  (2021-04-06T17:35:10Z)
• Learning to Control PDEs with Differentiable Physics [102.36050646250871]
  We present a novel hierarchical predictor-corrector scheme which enables neural networks to learn to understand and control complex nonlinear physical systems over long time frames. We demonstrate that our method successfully develops an understanding of complex physical systems and learns to control them for tasks involving PDEs.
  arXiv  Detail & Related papers  (2020-01-21T11:58:41Z)
• Solving inverse-PDE problems with physics-aware neural networks [0.0]
  We propose a novel framework to find unknown fields in the context of inverse problems for partial differential equations. We blend the high expressibility of deep neural networks as universal function estimators with the accuracy and reliability of existing numerical algorithms.
  arXiv  Detail & Related papers  (2020-01-10T18:46:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.