Physics-Informed Diffusion Models

• URL: http://arxiv.org/abs/2403.14404v2
• Date: Thu, 23 May 2024 09:34:29 GMT
• Title: Physics-Informed Diffusion Models
• Authors: Jan-Hendrik Bastek, WaiChing Sun, Dennis M. Kochmann,
• Abstract summary: We present a framework to inform denoising diffusion models of underlying constraints on generated samples during model training. Our approach improves the alignment of the generated samples with the imposed constraints and significantly outperforms existing methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative models such as denoising diffusion models are quickly advancing their ability to approximate highly complex data distributions. They are also increasingly leveraged in scientific machine learning, where samples from the implied data distribution are expected to adhere to specific governing equations. We present a framework to inform denoising diffusion models of underlying constraints on such generated samples during model training. Our approach improves the alignment of the generated samples with the imposed constraints and significantly outperforms existing methods without affecting inference speed. Additionally, our findings suggest that incorporating such constraints during training provides a natural regularization against overfitting. Our framework is easy to implement and versatile in its applicability for imposing equality and inequality constraints as well as auxiliary optimization objectives.

Related papers

• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution. Our contribution is to provide the first general estimation technique for transportability problems. We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• Symmetry-Preserving Diffusion Models via Target Symmetrization [43.83899968118655]
  We propose a novel approach that enforces equivariance through a symmetrized loss function. Our method uses Monte Carlo sampling to estimate the average, incurring minimal computational overhead. Experiments show improved sample quality compared to existing methods.
  arXiv  Detail & Related papers  (2025-02-14T03:26:57Z)
• Elucidating Flow Matching ODE Dynamics with Respect to Data Geometries [10.947094609205765]
  Diffusion-based generative models have become the standard for image generation. ODE-based samplers and flow matching models improve efficiency, in comparison to diffusion models, by reducing sampling steps through learned vector fields. We advance the theory of flow matching models through a comprehensive analysis of sample trajectories, centered on the denoiser that drives ODE dynamics. Our analysis reveals how trajectories evolve from capturing global data features to local structures, providing the geometric characterization of per-sample behavior in flow matching models.
  arXiv  Detail & Related papers  (2024-12-25T01:17:15Z)
• Learning Structural Causal Models from Ordering: Identifiable Flow Models [19.99352354910655]
  We introduce a set of flow models that can recover component-wise, invertible transformation of variables. We propose design improvements that enable simultaneous learning of all causal mechanisms. Our method achieves a significant reduction in computational time compared to existing diffusion-based techniques.
  arXiv  Detail & Related papers  (2024-12-13T04:25:56Z)
• Generalized Diffusion Model with Adjusted Offset Noise [1.7767466724342067]
  We propose a generalized diffusion model that naturally incorporates additional noise within a rigorous probabilistic framework. We derive a loss function based on the evidence lower bound, establishing its theoretical equivalence to offset noise with certain adjustments. Experiments on synthetic datasets demonstrate that our model effectively addresses brightness-related challenges and outperforms conventional methods in high-dimensional scenarios.
  arXiv  Detail & Related papers  (2024-12-04T08:57:03Z)
• Statistical guarantees for denoising reflected diffusion models [1.9116784879310031]
  In recent years, denoising diffusion models have become a crucial area of research due to their abundance in the rapidly expanding field of generative AI. In this paper, we study statistical guarantees of denoising reflected diffusion models. Our main contributions include the statistical analysis of this novel class of denoising reflected diffusion models and a refined score approximation method in both time and space.
  arXiv  Detail & Related papers  (2024-11-03T13:26:35Z)
• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Provable Statistical Rates for Consistency Diffusion Models [87.28777947976573]
  Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved. This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem.
  arXiv  Detail & Related papers  (2024-06-23T20:34:18Z)
• Constraint-Aware Diffusion Models for Trajectory Optimization [9.28162057044835]
  This paper presents a constraint-aware diffusion model for trajectory optimization. We introduce a novel hybrid loss function for training that minimizes the constraint violation of diffusion samples. Our model is demonstrated on tabletop manipulation and two-car reach-avoid problems.
  arXiv  Detail & Related papers  (2024-06-03T04:53:20Z)
• Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling [2.91204440475204]
  Diffusion Probabilistic Models (DPMs) have emerged as a powerful class of deep generative models. They rely on sequential denoising steps during sample generation. We propose a novel method that integrates denoising phases directly into the model's architecture.
  arXiv  Detail & Related papers  (2024-05-31T08:19:44Z)
• Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory [87.00653989457834]
  Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. Despite the empirical success, theory of conditional diffusion models is largely missing. This paper bridges the gap by presenting a sharp statistical theory of distribution estimation using conditional diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:08:24Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• Information-Theoretic Diffusion [18.356162596599436]
  Denoising diffusion models have spurred significant gains in density modeling and image generation. We introduce a new mathematical foundation for diffusion models inspired by classic results in information theory.
  arXiv  Detail & Related papers  (2023-02-07T23:03:07Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Surrogate Modeling for Physical Systems with Preserved Properties and Adjustable Tradeoffs [0.0]
  We present a model-based and a data-driven strategy to generate surrogate models. The latter generates interpretable surrogate models by fitting artificial relations to a presupposed topological structure. Our framework is compatible with various spatial discretization schemes for distributed parameter models.
  arXiv  Detail & Related papers  (2022-02-02T17:07:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.