Hessian-Informed Flow Matching

• URL: http://arxiv.org/abs/2410.11433v1
• Date: Tue, 15 Oct 2024 09:34:52 GMT
• Title: Hessian-Informed Flow Matching
• Authors: Christopher Iliffe Sprague, Arne Elofsson, Hossein Azizpour,
• Abstract summary: Hessian-Informed Flow Matching is a novel approach that integrates the Hessian of an energy function into conditional flows. This integration allows HI-FM to account for local curvature and anisotropic covariance structures. Empirical evaluations on the MNIST and Lennard-Jones particles datasets demonstrate that HI-FM improves the likelihood of test samples.
• Score: 4.542719108171107
• License:
• Abstract: Modeling complex systems that evolve toward equilibrium distributions is important in various physical applications, including molecular dynamics and robotic control. These systems often follow the stochastic gradient descent of an underlying energy function, converging to stationary distributions around energy minima. The local covariance of these distributions is shaped by the energy landscape's curvature, often resulting in anisotropic characteristics. While flow-based generative models have gained traction in generating samples from equilibrium distributions in such applications, they predominately employ isotropic conditional probability paths, limiting their ability to capture such covariance structures. In this paper, we introduce Hessian-Informed Flow Matching (HI-FM), a novel approach that integrates the Hessian of an energy function into conditional flows within the flow matching framework. This integration allows HI-FM to account for local curvature and anisotropic covariance structures. Our approach leverages the linearization theorem from dynamical systems and incorporates additional considerations such as time transformations and equivariance. Empirical evaluations on the MNIST and Lennard-Jones particles datasets demonstrate that HI-FM improves the likelihood of test samples.

Related papers

• Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models [59.331993845831946]
  Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties. This paper provides the first theoretical study towards understanding the influence of guidance on diffusion models in the context of Gaussian mixture models.
  arXiv  Detail & Related papers  (2024-03-03T23:15:48Z)
• Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation [19.71452214879951]
  conditional generation is one of the most important applications of generative models. We show that we can introduce inductive bias to the conditional generation through the matrix field. We will present our theory along with experimental results that support the competitiveness of EFM in conditional generation.
  arXiv  Detail & Related papers  (2024-02-29T04:12:32Z)
• Equivariant flow matching [0.9208007322096533]
  We introduce equivariant flow matching, a new training objective for equivariant continuous normalizing flows (CNFs) Equivariant flow matching exploits the physical symmetries of the target energy for efficient, simulation-free training of equivariant CNFs. Our results show that the equivariant flow matching objective yields flows with shorter integration paths, improved sampling efficiency, and higher scalability compared to existing methods.
  arXiv  Detail & Related papers  (2023-06-26T19:40:10Z)
• Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning [60.02391969049972]
  We introduce a novel deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. DiG employs deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system.
  arXiv  Detail & Related papers  (2023-06-08T17:12:08Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• GANs and Closures: Micro-Macro Consistency in Multiscale Modeling [0.0]
  We present an approach that couples physics-based simulations and biasing methods for sampling conditional distributions with Machine Learning-based conditional generative adversarial networks. We show that this framework can improve multiscale SDE dynamical systems sampling, and even shows promise for systems of increasing complexity.
  arXiv  Detail & Related papers  (2022-08-23T03:45:39Z)
• GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation [102.85440102147267]
  We propose a novel generative model named GeoDiff for molecular conformation prediction. We show that GeoDiff is superior or comparable to existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-06T09:47:01Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densities [1.7188280334580197]
  Normalizing flows are exact-likelihood generative neural networks which transform samples from a simple prior distribution to samples of the probability distribution of interest. Recent work showed that such generative models can be utilized in statistical mechanics to sample equilibrium states of many-body systems in physics and chemistry.
  arXiv  Detail & Related papers  (2020-06-03T17:54:26Z)
• AI Giving Back to Statistics? Discovery of the Coordinate System of Univariate Distributions by Beta Variational Autoencoder [0.0]
  The article discusses experiences of training neural networks to classify univariate empirical distributions and to represent them on the two-dimensional latent space forcing disentanglement based on the inputs of cumulative distribution functions (CDF) The representation on the latent two-dimensional coordinate system can be seen as an additional metadata of the real-world data that disentangles important distribution characteristics, such as shape of the CDF, classification probabilities of underlying theoretical distributions and their parameters, information entropy, and skewness.
  arXiv  Detail & Related papers  (2020-04-06T14:11:13Z)
• Training Deep Energy-Based Models with f-Divergence Minimization [113.97274898282343]
  Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging. We propose a general variational framework termed f-EBM to train EBMs using any desired f-divergence. Experimental results demonstrate the superiority of f-EBM over contrastive divergence, as well as the benefits of training EBMs using f-divergences other than KL.
  arXiv  Detail & Related papers  (2020-03-06T23:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.