CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problems

• URL: http://arxiv.org/abs/2312.10527v2
• Date: Sun, 20 Oct 2024 15:18:03 GMT
• Title: CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problems
• Authors: Christian Jacobsen, Yilin Zhuang, Karthik Duraisamy,
• Abstract summary: This work extends the reach of generative models into physical problem domains. We present an efficient approach to promote consistency with the underlying PDE. We showcase the potential and versatility of score-based generative models in various physics tasks.
• Score: 1.0923877073891446
• License:
• Abstract: Recent advances in generative artificial intelligence have had a significant impact on diverse domains spanning computer vision, natural language processing, and drug discovery. This work extends the reach of generative models into physical problem domains, particularly addressing the efficient enforcement of physical laws and conditioning for forward and inverse problems involving partial differential equations (PDEs). Our work introduces two key contributions: firstly, we present an efficient approach to promote consistency with the underlying PDE. By incorporating discretized information into score-based generative models, our method generates samples closely aligned with the true data distribution, showcasing residuals comparable to data generated through conventional PDE solvers, significantly enhancing fidelity. Secondly, we showcase the potential and versatility of score-based generative models in various physics tasks, specifically highlighting surrogate modeling as well as probabilistic field reconstruction and inversion from sparse measurements. A robust foundation is laid by designing unconditional score-based generative models that utilize reversible probability flow ordinary differential equations. Leveraging conditional models that require minimal training, we illustrate their flexibility when combined with a frozen unconditional model. These conditional models generate PDE solutions by incorporating parameters, macroscopic quantities, or partial field measurements as guidance. The results illustrate the inherent flexibility of score-based generative models and explore the synergy between unconditional score-based generative models and the present physically-consistent sampling approach, emphasizing the power and flexibility in solving for and inverting physical fields governed by differential equations, and in other scientific machine learning tasks.

Related papers

• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolation [4.286691905364396]
  We introduce a multi-modal foundation model for scientific problems, named PROSE-PDE. Our model is a multi-operator learning approach which can predict future states of systems while concurrently learning the underlying governing equations of the physical system.
  arXiv  Detail & Related papers  (2024-04-18T17:34:20Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Learning Space-Time Continuous Neural PDEs from Partially Observed States [13.01244901400942]
  We introduce a grid-independent model learning partial differential equations (PDEs) from noisy and partial observations on irregular grids. We propose a space-time continuous latent neural PDE model with an efficient probabilistic framework and a novel design encoder for improved data efficiency and grid independence.
  arXiv  Detail & Related papers  (2023-07-09T06:53:59Z)
• Score-based Generative Modeling Through Backward Stochastic Differential Equations: Inversion and Generation [6.2255027793924285]
  The proposed BSDE-based diffusion model represents a novel approach to diffusion modeling, which extends the application of differential equations (SDEs) in machine learning. We demonstrate the theoretical guarantees of the model, the benefits of using Lipschitz networks for score matching, and its potential applications in various areas such as diffusion inversion, conditional diffusion, and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-04-26T01:15:35Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Surrogate Modeling for Physical Systems with Preserved Properties and Adjustable Tradeoffs [0.0]
  We present a model-based and a data-driven strategy to generate surrogate models. The latter generates interpretable surrogate models by fitting artificial relations to a presupposed topological structure. Our framework is compatible with various spatial discretization schemes for distributed parameter models.
  arXiv  Detail & Related papers  (2022-02-02T17:07:02Z)
• Approximate Latent Force Model Inference [1.3927943269211591]
  latent force models offer an interpretable alternative to purely data driven tools for inference in dynamical systems. We show that a neural operator approach can scale our model to thousands of instances, enabling fast, distributed computation.
  arXiv  Detail & Related papers  (2021-09-24T09:55:00Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.