Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving

• URL: http://arxiv.org/abs/2408.02198v1
• Date: Mon, 5 Aug 2024 02:50:58 GMT
• Title: Synergistic Learning with Multi-Task DeepONet for Efficient PDE Problem Solving
• Authors: Varun Kumar, Somdatta Goswami, Katiana Kontolati, Michael D. Shields, George Em Karniadakis,
• Abstract summary: Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs) We present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session.
• Score: 5.692133861249929
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.

Related papers

• DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [63.5925701087252]
  We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis. To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers. Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
  arXiv  Detail & Related papers  (2024-10-08T10:48:50Z)
• Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs [85.40198664108624]
  We propose Codomain Attention Neural Operator (CoDA-NO) to solve multiphysics problems with PDEs. CoDA-NO tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. We find CoDA-NO to outperform existing methods by over 36% on complex downstream tasks with limited data.
  arXiv  Detail & Related papers  (2024-03-19T08:56:20Z)
• GIT-Net: Generalized Integral Transform for Operator Learning [58.13313857603536]
  This article introduces GIT-Net, a deep neural network architecture for approximating Partial Differential Equation (PDE) operators. GIT-Net harnesses the fact that differential operators commonly used for defining PDEs can often be represented parsimoniously when expressed in specialized functional bases. Numerical experiments demonstrate that GIT-Net is a competitive neural network operator, exhibiting small test errors and low evaluations across a range of PDE problems.
  arXiv  Detail & Related papers  (2023-12-05T03:03:54Z)
• Physics-informed neural networks for transformed geometries and manifolds [0.0]
  We propose a novel method for integrating geometric transformations within PINNs to robustly accommodate geometric variations. We demonstrate the enhanced flexibility over traditional PINNs, especially under geometric variations. The proposed framework presents an outlook for training deep neural operators over parametrized geometries.
  arXiv  Detail & Related papers  (2023-11-27T15:47:33Z)
• A foundational neural operator that continuously learns without forgetting [1.0878040851638]
  We introduce the concept of the Neural Combinatorial Wavelet Neural Operator (NCWNO) as a foundational model for scientific computing. The NCWNO is specifically designed to excel in learning from a diverse spectrum of physics and continuously adapt to the solution operators associated with parametric partial differential equations (PDEs) The proposed foundational model offers two key advantages: (i) it can simultaneously learn solution operators for multiple parametric PDEs, and (ii) it can swiftly generalize to new parametric PDEs with minimal fine-tuning.
  arXiv  Detail & Related papers  (2023-10-29T03:20:10Z)
• Multi-Objective Optimization for Sparse Deep Multi-Task Learning [0.0]
  We present a Multi-Objective Optimization algorithm using a modified Weighted Chebyshev scalarization for training Deep Neural Networks (DNNs) Our work aims to address the (economical and also ecological) sustainability issue of DNN models, with particular focus on Deep Multi-Task models.
  arXiv  Detail & Related papers  (2023-08-23T16:42:27Z)
• iPINNs: Incremental learning for Physics-informed neural networks [66.4795381419701]
  Physics-informed neural networks (PINNs) have recently become a powerful tool for solving partial differential equations (PDEs) We propose incremental PINNs that can learn multiple tasks sequentially without additional parameters for new tasks and improve performance for every equation in the sequence. Our approach learns multiple PDEs starting from the simplest one by creating its own subnetwork for each PDE and allowing each subnetwork to overlap with previously learnedworks.
  arXiv  Detail & Related papers  (2023-04-10T20:19:20Z)
• Neural Partial Differential Equations with Functional Convolution [30.35306295442881]
  We present a lightweighted neural PDE representation to discover the hidden structure and predict the solution of different nonlinear PDEs. We leverage the prior of translational similarity'' of numerical PDE differential operators to drastically reduce the scale of learning model and training data.
  arXiv  Detail & Related papers  (2023-03-10T04:25:38Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Neural TMDlayer: Modeling Instantaneous flow of features via SDE Generators [37.92379202320938]
  We study how differential equation (SDE) based ideas can inspire new modifications to existing algorithms for a set of problems in computer vision. We show promising experiments on a number of vision tasks including few shot learning, point cloud transformers and deep variational segmentation.
  arXiv  Detail & Related papers  (2021-08-19T19:54:04Z)
• ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
  In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
  arXiv  Detail & Related papers  (2021-02-16T04:07:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.