Generating synthetic data for neural operators

• URL: http://arxiv.org/abs/2401.02398v3
• Date: Fri, 02 May 2025 17:02:22 GMT
• Title: Generating synthetic data for neural operators
• Authors: Erisa Hasani, Rachel A. Ward,
• Abstract summary: We introduce a "backward" data generation method that avoids solving the PDE numerically.<n>This produces training pairs $(f_j, u_j)$ by computing derivatives rather than solving a PDE numerically for each data point.<n>Experiments indicate that models trained on this synthetic data generalize well when tested on data produced by standard solvers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in the literature show promising potential of deep learning methods, particularly neural operators, in obtaining numerical solutions to partial differential equations (PDEs) beyond the reach of current numerical solvers. However, existing data-driven approaches often rely on training data produced by numerical PDE solvers (e.g., finite difference or finite element methods). We introduce a "backward" data generation method that avoids solving the PDE numerically: by randomly sampling candidate solutions $u_j$ from the appropriate solution space (e.g., $H_0^1(\Omega)$), we compute the corresponding right-hand side $f_j$ directly from the equation by differentiation. This produces training pairs ${(f_j, u_j)}$ by computing derivatives rather than solving a PDE numerically for each data point, enabling fast, large-scale data generation consisting of exact solutions. Experiments indicate that models trained on this synthetic data generalize well when tested on data produced by standard solvers. While the idea is simple, we hope this method will expand the potential of neural PDE solvers that do not rely on classical numerical solvers to generate their data.

Related papers

• Scale-Consistent Learning for Partial Differential Equations [79.48661503591943]
  We propose a data augmentation scheme based on scale-consistency properties of PDEs.<n>We then design a scale-informed neural operator that can model a wide range of scales.<n>With scale-consistency, the model trained on $Re$ of 1000 can generalize to $Re$ ranging from 250 to 10000.
  arXiv  Detail & Related papers  (2025-07-24T21:29:52Z)
• A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions [13.48986376824454]
  PDE foundation models utilize neural networks to train approximations to multiple differential equations simultaneously. We propose a novel multimodal deep learning approach that leverages a transformer-based architecture to approximate solution operators. Our approach generates interpretable scientific text descriptions, offering deeper insights into the underlying dynamics and solution properties.
  arXiv  Detail & Related papers  (2025-02-09T20:50:28Z)
• Method of data forward generation with partial differential equations for machine learning modeling in fluid mechanics [1.9688252014450927]
  This study proposes a high-efficient data forward generation method from the partial differential equations (PDEs)<n>A Poisson neural network (Poisson-NN) embedded in projection method and a wavelet transform convolutional neuro network (WTCNN) embedded in multigrid numerical simulation for solving incompressible Navier-Stokes equations is respectively proposed.
  arXiv  Detail & Related papers  (2025-01-06T15:17:13Z)
• Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations [5.308435208832696]
  A computed approximation of the solution operator to a system of partial differential equations (PDEs) is needed in various areas of science and engineering.<n>We propose that in order to learn a PDE solution operator that can generalize across multiple domains without needing to sample enough data expressive enough, we can train instead a latent neural operator on just a few ground truth solution fields.
  arXiv  Detail & Related papers  (2024-11-27T03:16:00Z)
• Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization [65.8915778873691]
  conditional distributions is a central problem in machine learning. We propose a new learning paradigm that integrates both paired and unpaired data. Our approach also connects intriguingly with inverse entropic optimal transport (OT)
  arXiv  Detail & Related papers  (2024-10-03T16:12:59Z)
• Active Learning for Neural PDE Solvers [18.665448858377694]
  Active learning could help surrogate models reach the same accuracy with smaller training sets.<n>We introduce AL4PDE, a modular and active learning benchmark.<n>We show that AL reduces the average error by up to 71% compared to random sampling.
  arXiv  Detail & Related papers  (2024-08-02T18:48:58Z)
• On the estimation rate of Bayesian PINN for inverse problems [10.100602879566782]
  Solving partial differential equations (PDEs) and their inverse problems using Physics-informed neural networks (PINNs) is a rapidly growing approach in the physics and machine learning community. We study the behavior of a Bayesian PINN estimator of the solution of a PDE from $n$ independent noisy measurement of the solution.
  arXiv  Detail & Related papers  (2024-06-21T01:13:18Z)
• Solving partial differential equations with sampled neural networks [1.8590821261905535]
  Approximation of solutions to partial differential equations (PDE) is an important problem in computational science and engineering. We discuss how sampling the hidden weights and biases of the ansatz network from data-agnostic and data-dependent probability distributions allows us to progress on both challenges.
  arXiv  Detail & Related papers  (2024-05-31T14:24:39Z)
• Learning High-Dimensional Nonparametric Differential Equations via Multivariate Occupation Kernel Functions [0.31317409221921133]
  Learning a nonparametric system of ordinary differential equations requires learning $d$ functions of $d$ variables. Explicit formulations scale quadratically in $d$ unless additional knowledge about system properties, such as sparsity and symmetries, is available. We propose a linear approach to learning using the implicit formulation provided by vector-valued Reproducing Kernel Hilbert Spaces.
  arXiv  Detail & Related papers  (2023-06-16T21:49:36Z)
• Learning the Solution Operator of Boundary Value Problems using Graph Neural Networks [0.0]
  We design a general solution operator for two different time-independent PDEs using graph neural networks (GNNs) and spectral graph convolutions. We train the networks on simulated data from a finite elements solver on a variety of shapes and inhomogeneities. We find that training on a diverse dataset with lots of variation in the finite element meshes is a key ingredient for achieving good generalization results.
  arXiv  Detail & Related papers  (2022-06-28T15:39:06Z)
• Improved Training of Physics-Informed Neural Networks with Model Ensembles [81.38804205212425]
  We propose to expand the solution interval gradually to make the PINN converge to the correct solution. All ensemble members converge to the same solution in the vicinity of observed data. We show experimentally that the proposed method can improve the accuracy of the found solution.
  arXiv  Detail & Related papers  (2022-04-11T14:05:34Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Computational characteristics of feedforward neural networks for solving a stiff differential equation [0.0]
  We study the solution of a simple but fundamental stiff ordinary differential equation modelling a damped system. We show that it is possible to identify preferable choices to be made for parameters and methods. Overall we extend the current literature in the field by showing what can be done in order to obtain reliable and accurate results by the neural network approach.
  arXiv  Detail & Related papers  (2020-12-03T12:22:24Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Deep-learning of Parametric Partial Differential Equations from Sparse and Noisy Data [2.4431531175170362]
  In this work, a new framework, which combines neural network, genetic algorithm and adaptive methods, is put forward to address all of these challenges simultaneously. A trained neural network is utilized to calculate derivatives and generate a large amount of meta-data, which solves the problem of sparse noisy data. Next, genetic algorithm is utilized to discover the form of PDEs and corresponding coefficients with an incomplete candidate library. A two-step adaptive method is introduced to discover parametric PDEs with spatially- or temporally-varying coefficients.
  arXiv  Detail & Related papers  (2020-05-16T09:09:57Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.