To discretize continually: Mean shift interacting particle systems for Bayesian inference

• URL: http://arxiv.org/abs/2605.14142v1
• Date: Wed, 13 May 2026 21:48:00 GMT
• Title: To discretize continually: Mean shift interacting particle systems for Bayesian inference
• Abstract summary: We introduce new methods for approximating integration against a probability distribution given its unnormalized density.<n>These methods extend the classical mean shift algorithm, as well as recent algorithms for optimal quantization of empirical distributions.<n>We demonstrate their performance on a wide range of benchmark sampling problems, including multi-modal mixtures, Bayesian hierarchical models, PDE-constrained inverse problems, and beyond.
• Score: 9.150567695563067
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Integration against a probability distribution given its unnormalized density is a central task in Bayesian inference and other fields. We introduce new methods for approximating such expectations with a small set of weighted samples -- i.e., a quadrature rule -- constructed via an interacting particle system that minimizes maximum mean discrepancy (MMD) to the target distribution. These methods extend the classical mean shift algorithm, as well as recent algorithms for optimal quantization of empirical distributions, to the case of continuous distributions. Crucially, our approach creates dynamics for MMD minimization that are invariant to the unknown normalizing constant; they also admit both gradient-free and gradient-informed implementations. The resulting mean shift interacting particle systems converge quickly, capture anisotropy and multi-modality, avoid mode collapse, and scale to high dimensions. We demonstrate their performance on a wide range of benchmark sampling problems, including multi-modal mixtures, Bayesian hierarchical models, PDE-constrained inverse problems, and beyond.

Related papers

• Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms [0.29494468099506904]
  We propose a continuous-time formulation of persistent contrastive divergence (PCD) for maximum likelihood estimation (MLE) of unnormalised densities.<n>We are able to derive explicit bounds for the error between the PCD and the MLE solution for the model parameter.
  arXiv  Detail & Related papers  (2025-10-02T12:12:33Z)
• Sampling by averaging: A multiscale approach to score estimation [4.003851730099099]
  We introduce a novel framework for efficient sampling from complex, unnormalised target distributions by exploiting multiscale dynamics.<n>Two algorithms are developed: MultALMC and MultCDiff, based on multiscale controlled diffusions for the reverse-time Ornstein-Uhlenbeck process.<n>The framework is extended to handle heavy-dimensional target distributions using Student's t-based noise models and tailored fast-process dynamics.
  arXiv  Detail & Related papers  (2025-08-20T21:09:34Z)
• Steering Large Agent Populations using Mean-Field Schrodinger Bridges with Gaussian Mixture Models [13.03355083378673]
  Mean-Field Schrodinger Bridge (MFSB) problem is an optimization problem aiming to find the minimum effort control policy.<n>In the context of multi-agent control, the objective is to control the configuration of a swarm of identical, interacting cooperative agents.
  arXiv  Detail & Related papers  (2025-03-31T04:01:04Z)
• Weighted quantization using MMD: From mean field to mean shift via gradient flows [5.216151302783165]
  We show that a Wasserstein-Fisher-Rao gradient flow is well-suited for designing quantizations optimal under MMD.<n>We derive a new fixed-point algorithm called mean shift interacting particles (MSIP)<n>Our unification of gradient flows, mean shift, and MMD-optimal quantization yields algorithms more robust than state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-02-14T23:13:20Z)
• Multi-Agent Path Finding in Continuous Spaces with Projected Diffusion Models [57.45019514036948]
  Multi-Agent Path Finding (MAPF) is a fundamental problem in robotics.<n>This work proposes a novel approach that integrates constrained optimization with diffusion models for MAPF in continuous spaces.
  arXiv  Detail & Related papers  (2024-12-23T21:27:19Z)
• Go With the Flow: Fast Diffusion for Gaussian Mixture Models [13.836464287004505]
  Schrodinger Bridges (SBs) are diffusion processes that steer in finite time, a given initial distribution to another final one while minimizing a suitable cost functional.<n>We propose an analytic parametrization of a set of feasible policies for solving low-to-dimensional problems.<n>We showcase the potential of this approach in low-to-dimensional problems such as image-to- latent translation in the space of an autoencoder, learning of cellular dynamics using multi-marginal momentum SBs, and various other examples.
  arXiv  Detail & Related papers  (2024-12-12T08:40:22Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.