No-Regret Generative Modeling via Parabolic Monge-Ampère PDE

• URL: http://arxiv.org/abs/2504.09279v1
• Date: Sat, 12 Apr 2025 16:51:02 GMT
• Title: No-Regret Generative Modeling via Parabolic Monge-Ampère PDE
• Authors: Nabarun Deb, Tengyuan Liang,
• Abstract summary: We introduce a novel generative modeling framework based on a discretized parabolic Monge-Ampere PDE.<n>We establish theoretical guarantees for generative modeling through the lens of no-regret analysis.<n>As a technical contribution, we derive a new Evolution Variational Inequality tailored to the parabolic Monge-Ampere PDE.
• Score: 8.006362366562936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel generative modeling framework based on a discretized parabolic Monge-Amp\`ere PDE, which emerges as a continuous limit of the Sinkhorn algorithm commonly used in optimal transport. Our method performs iterative refinement in the space of Brenier maps using a mirror gradient descent step. We establish theoretical guarantees for generative modeling through the lens of no-regret analysis, demonstrating that the iterates converge to the optimal Brenier map under a variety of step-size schedules. As a technical contribution, we derive a new Evolution Variational Inequality tailored to the parabolic Monge-Amp\`ere PDE, connecting geometry, transportation cost, and regret. Our framework accommodates non-log-concave target distributions, constructs an optimal sampling process via the Brenier map, and integrates favorable learning techniques from generative adversarial networks and score-based diffusion models. As direct applications, we illustrate how our theory paves new pathways for generative modeling and variational inference.

Related papers

• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• A Stein Gradient Descent Approach for Doubly Intractable Distributions [5.63014864822787]
  We propose a novel Monte Carlo Stein variational gradient descent (MC-SVGD) approach for inference for doubly intractable distributions. The proposed method achieves substantial computational gains over existing algorithms, while providing comparable inferential performance for the posterior distributions.
  arXiv  Detail & Related papers  (2024-10-28T13:42:27Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• Reflected Schr\"odinger Bridge for Constrained Generative Modeling [16.72888494254555]
  Reflected diffusion models have become the go-to method for large-scale generative models in real-world applications. We introduce the Reflected Schrodinger Bridge algorithm: an entropy-regularized optimal transport approach tailored generating data within diverse bounded domains. Our algorithm yields robust generative modeling in diverse domains, and its scalability is demonstrated in real-world constrained generative modeling through standard image benchmarks.
  arXiv  Detail & Related papers  (2024-01-06T14:39:58Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Protein Design with Guided Discrete Diffusion [67.06148688398677]
  A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling. We propose diffusioN Optimized Sampling (NOS), a guidance method for discrete diffusion models. NOS makes it possible to perform design directly in sequence space, circumventing significant limitations of structure-based methods.
  arXiv  Detail & Related papers  (2023-05-31T16:31:24Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• An optimal control perspective on diffusion-based generative modeling [9.806130366152194]
  We establish a connection between optimal control and generative models based on differential equations (SDEs) In particular, we derive a Hamilton-Jacobi-Bellman equation that governs the evolution of the log-densities of the underlying SDE marginals. We develop a novel diffusion-based method for sampling from unnormalized densities.
  arXiv  Detail & Related papers  (2022-11-02T17:59:09Z)
• Unveiling the Latent Space Geometry of Push-Forward Generative Models [24.025975236316846]
  Many deep generative models are defined as a push-forward of a Gaussian measure by a continuous generator, such as Generative Adversarial Networks (GANs) or Variational Auto-Encoders (VAEs) This work explores the latent space of such deep generative models. A key issue with these models is their tendency to output samples outside of the support of the target distribution when learning disconnected distributions.
  arXiv  Detail & Related papers  (2022-07-21T15:29:35Z)
• Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations [57.15855198512551]
  We propose a novel score-based generative model for graphs with a continuous-time framework. We show that our method is able to generate molecules that lie close to the training distribution yet do not violate the chemical valency rule.
  arXiv  Detail & Related papers  (2022-02-05T08:21:04Z)
• Spatiotemporal Imaging with Diffeomorphic Optimal Transportation [1.5464123983408111]
  We propose a variational model with diffeomorphic optimal transportation for joint image reconstruction and motion estimation. The proposed model is compared against several existing alternatives theoretically. Several important issues on the proposed model and associated algorithms are also discussed.
  arXiv  Detail & Related papers  (2020-11-24T05:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.