Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers

• URL: http://arxiv.org/abs/2405.17527v2
• Date: Sat, 1 Jun 2024 13:28:27 GMT
• Title: Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers
• Authors: Hang Zhou, Yuezhou Ma, Haixu Wu, Haowen Wang, Mingsheng Long,
• Abstract summary: We present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components. Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks.
• Score: 55.0876373185983
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep models have recently emerged as a promising tool to solve partial differential equations (PDEs), known as neural PDE solvers. While neural solvers trained from either simulation data or physics-informed loss can solve the PDEs reasonably well, they are mainly restricted to a specific set of PDEs, e.g. a certain equation or a finite set of coefficients. This bottleneck limits the generalizability of neural solvers, which is widely recognized as its major advantage over numerical solvers. In this paper, we present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs by leveraging a Transformer pre-trained on diverse data and conditioned on diverse PDEs. Instead of simply scaling up data and parameters, Unisolver stems from the theoretical analysis of the PDE-solving process. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components, e.g. equation symbols, coefficients, and initial and boundary conditions. Inspired by the mathematical structure of PDEs, we define a complete set of PDE components and correspondingly embed them as domain-wise (e.g. equation symbols) and point-wise (e.g. boundaries) conditions for Transformer PDE solvers. Integrating physical insights with recent Transformer advances, Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks, showing impressive gains and endowing favorable generalizability and scalability.

Related papers

• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Elucidating the solution space of extended reverse-time SDE for diffusion models [54.23536653351234]
  Diffusion models (DMs) demonstrate potent image generation capabilities in various generative modeling tasks. Their primary limitation lies in slow sampling speed, requiring hundreds or thousands of sequential function evaluations to generate high-quality images. We formulate the sampling process as an extended reverse-time SDE, unifying prior explorations into ODEs and SDEs. We devise fast and training-free samplers, ER-SDE-rs, achieving state-of-the-art performance across all samplers.
  arXiv  Detail & Related papers  (2023-09-12T12:27:17Z)
• Weak-PDE-LEARN: A Weak Form Based Approach to Discovering PDEs From Noisy, Limited Data [0.0]
  We introduce Weak-PDE-LEARN, a discovery algorithm that can identify non-linear PDEs from noisy, limited measurements of their solutions. We demonstrate the efficacy of Weak-PDE-LEARN by learning several benchmark PDEs.
  arXiv  Detail & Related papers  (2023-09-09T06:45:15Z)
• Neural Control of Parametric Solutions for High-dimensional Evolution PDEs [6.649496716171139]
  We develop a novel computational framework to approximate solution operators of evolution partial differential equations (PDEs) We propose to approximate the solution operator of the PDE by learning the control vector field in the parameter space. This allows for substantially reduced computational cost to solve the evolution PDE with arbitrary initial conditions.
  arXiv  Detail & Related papers  (2023-01-31T19:26:25Z)
• Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh [24.572840023107574]
  Partial differential equations (PDEs) are often computationally challenging to solve. We present a meta-learning based method which learns to rapidly solve problems from a distribution of related PDEs.
  arXiv  Detail & Related papers  (2022-11-03T06:17:52Z)
• Learning differentiable solvers for systems with hard constraints [48.54197776363251]
  We introduce a practical method to enforce partial differential equation (PDE) constraints for functions defined by neural networks (NNs) We develop a differentiable PDE-constrained layer that can be incorporated into any NN architecture. Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective.
  arXiv  Detail & Related papers  (2022-07-18T15:11:43Z)
• Neural Q-learning for solving PDEs [0.0]
  We develop a new numerical method for solving elliptic-type PDEs by adapting the Q-learning algorithm in reinforcement learning. Our "Q-PDE" algorithm is mesh-free and therefore has the potential to overcome the curse of dimensionality. The numerical performance of the Q-PDE algorithm is studied for several elliptic PDEs.
  arXiv  Detail & Related papers  (2022-03-31T15:52:44Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• Meta-Auto-Decoder for Solving Parametric Partial Differential Equations [32.46080264991759]
  Partial Differential Equations (PDEs) are ubiquitous in many disciplines of science and engineering and notoriously difficult to solve. Our proposed approach, called Meta-Auto-Decoder (MAD), treats solving parametric PDEs as a meta-learning problem. MAD exhibits faster convergence speed without losing the accuracy compared with other deep learning methods.
  arXiv  Detail & Related papers  (2021-11-15T02:51:42Z)
• dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
  We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
  arXiv  Detail & Related papers  (2021-03-15T19:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.