Unbalanced Sobolev Descent

• URL: http://arxiv.org/abs/2009.14148v1
• Date: Tue, 29 Sep 2020 16:43:38 GMT
• Title: Unbalanced Sobolev Descent
• Authors: Youssef Mroueh, Mattia Rigotti
• Abstract summary: We introduce Unbalanced Sobolev Descent (USD), a particle descent algorithm for transporting a high dimensional source distribution to a target distribution that does not necessarily have the same mass. USD transports particles along flows of the witness function of the Sobolev-Fisher discrepancy (advection step) and reweighs the mass of particles with respect to this witness function (reaction step) We show on synthetic examples that USD transports distributions with or without conservation of mass faster than previous particle descent algorithms.
• Score: 31.777218621726284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Unbalanced Sobolev Descent (USD), a particle descent algorithm for transporting a high dimensional source distribution to a target distribution that does not necessarily have the same mass. We define the Sobolev-Fisher discrepancy between distributions and show that it relates to advection-reaction transport equations and the Wasserstein-Fisher-Rao metric between distributions. USD transports particles along gradient flows of the witness function of the Sobolev-Fisher discrepancy (advection step) and reweighs the mass of particles with respect to this witness function (reaction step). The reaction step can be thought of as a birth-death process of the particles with rate of growth proportional to the witness function. When the Sobolev-Fisher witness function is estimated in a Reproducing Kernel Hilbert Space (RKHS), under mild assumptions we show that USD converges asymptotically (in the limit of infinite particles) to the target distribution in the Maximum Mean Discrepancy (MMD) sense. We then give two methods to estimate the Sobolev-Fisher witness with neural networks, resulting in two Neural USD algorithms. The first one implements the reaction step with mirror descent on the weights, while the second implements it through a birth-death process of particles. We show on synthetic examples that USD transports distributions with or without conservation of mass faster than previous particle descent algorithms, and finally demonstrate its use for molecular biology analyses where our method is naturally suited to match developmental stages of populations of differentiating cells based on their single-cell RNA sequencing profile. Code is available at https://github.com/ibm/usd .

Related papers

• Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data [55.54827581105283]
  We show that the concrete score in absorbing diffusion can be expressed as conditional probabilities of clean data. We propose a dedicated diffusion model without time-condition that characterizes the time-independent conditional probabilities. Our models achieve SOTA performance among diffusion models on 5 zero-shot language modeling benchmarks.
  arXiv  Detail & Related papers  (2024-06-06T04:22:11Z)
• Amortizing intractable inference in diffusion models for vision, language, and control [89.65631572949702]
  This paper studies amortized sampling of the posterior over data, $mathbfxsim prm post(mathbfx)propto p(mathbfx)r(mathbfx)$, in a model that consists of a diffusion generative model prior $p(mathbfx)$ and a black-box constraint or function $r(mathbfx)$. We prove the correctness of a data-free learning objective, relative trajectory balance, for training a diffusion model that samples from
  arXiv  Detail & Related papers  (2024-05-31T16:18:46Z)
• Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective [18.331374727331077]
  We provide a theoretical understanding of the Lipschitz continuity and second momentum properties of the diffusion process. Our results provide deeper theoretical insights into the dynamics of the diffusion process under common data distributions.
  arXiv  Detail & Related papers  (2024-05-26T03:32:27Z)
• Computing Marginal and Conditional Divergences between Decomposable Models with Applications [7.89568731669979]
  We propose an approach to compute the exact alpha-beta divergence between any marginal or conditional distribution of two decomposable models. We show how our method can be used to analyze distributional changes by first applying it to a benchmark image dataset. Based on our framework, we propose a novel way to quantify the error in contemporary superconducting quantum computers.
  arXiv  Detail & Related papers  (2023-10-13T14:17:25Z)
• Bayesian Inference for Jump-Diffusion Approximations of Biochemical Reaction Networks [26.744964200606784]
  We develop a tractable Bayesian inference algorithm based on Markov chain Monte Carlo. The algorithm is numerically evaluated for a partially observed multi-scale birth-death process example.
  arXiv  Detail & Related papers  (2023-04-13T14:57:22Z)
• Dilute neutron star matter from neural-network quantum states [58.720142291102135]
  Low-density neutron matter is characterized by the formation of Cooper pairs and the onset of superfluidity. We model this density regime by capitalizing on the expressivity of the hidden-nucleon neural-network quantum states combined with variational Monte Carlo and reconfiguration techniques.
  arXiv  Detail & Related papers  (2022-12-08T17:55:25Z)
• Interference dynamics of matter-waves of SU($N$) fermions [0.0]
  We analyze the two main physical observables related to the momenta of strongly correlated SU($N$) fermions in ring-shaped lattices pierced by an effective magnetic flux. We find that both homodyne and self-heterodyne interference display a specific dependence on the structure of the Fermi distribution and particles' correlations.
  arXiv  Detail & Related papers  (2022-06-06T18:00:01Z)
• Flexible Amortized Variational Inference in qBOLD MRI [56.4324135502282]
  Oxygen extraction fraction (OEF) and deoxygenated blood volume (DBV) are more ambiguously determined from the data. Existing inference methods tend to yield very noisy and underestimated OEF maps, while overestimating DBV. This work describes a novel probabilistic machine learning approach that can infer plausible distributions of OEF and DBV.
  arXiv  Detail & Related papers  (2022-03-11T10:47:16Z)
• A blob method method for inhomogeneous diffusion with applications to multi-agent control and sampling [0.6562256987706128]
  We develop a deterministic particle method for the weighted porous medium equation (WPME) and prove its convergence on bounded time intervals. Our method has natural applications to multi-agent coverage algorithms and sampling probability measures.
  arXiv  Detail & Related papers  (2022-02-25T19:49:05Z)
• Mechanism for particle fractionalization and universal edge physics in quantum Hall fluids [58.720142291102135]
  We advance a second-quantization framework that helps reveal an exact fusion mechanism for particle fractionalization in FQH fluids. We also uncover the fundamental structure behind the condensation of non-local operators characterizing topological order in the lowest-Landau-level (LLL)
  arXiv  Detail & Related papers  (2021-10-12T18:00:00Z)
• Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization [106.70006655990176]
  A distributional optimization problem arises widely in machine learning and statistics. We propose a novel particle-based algorithm, dubbed as variational transport, which approximately performs Wasserstein gradient descent. We prove that when the objective function satisfies a functional version of the Polyak-Lojasiewicz (PL) (Polyak, 1963) and smoothness conditions, variational transport converges linearly.
  arXiv  Detail & Related papers  (2020-12-21T18:33:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.