Nesterov acceleration in optimizing over probability measures

• URL: http://arxiv.org/abs/2607.23008v1
• Date: Sat, 25 Jul 2026 02:53:53 GMT
• Title: Nesterov acceleration in optimizing over probability measures
• Abstract summary: Motivated by Nesterov's accelerated method in Euclidean space, we develop Heavy-ball and Nesterov acceleration methods over the probability measure space.<n>We establish non-asymptotic convergence guarantees that match their Euclidean counterparts.
• Score: 2.1838525584229362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimization over probability measures has become an increasingly important paradigm in modern machine learning, scientific computing, and uncertainty quantification. Motivated by Nesterov's accelerated gradient method in Euclidean space, we develop Heavy-ball and Nesterov acceleration methods over the probability measure space $\mathcal{P}_2$ and establish non-asymptotic convergence guarantees that match their Euclidean counterparts. In particular, we derive convergence rates with respect to both the number of iterations and the number of particles used to represent the underlying probability distributions. Extending accelerated optimization from Euclidean space to probability measures is challenging. The natural notion of momentum requires concepts such as tangent bundles of the set of probability space and they are hard to operate numerically. To overcome these difficulties, we introduce two complementary lifting procedures. The first lifts probability measures to phase space through a Hamiltonian formulation, introducing momentum variables into the dynamics. The second lifts probability measures to a common Hilbert space, restoring the linear structure required for convergence analysis while simultaneously yielding executable particle dynamics. Together, these two complementary lifting procedures provide a systematic methodology for designing, analyzing, and implementing momentum-based accelerated optimization methods over probability measure spaces.

Related papers

• Fitting Unknown Number of Hyperplanes with Manifold Optimization [57.48093263119306]
  Fitting unknown number of linearplanes to data is a fundamental yet challenging problem in machine learning.<n>Existing approaches often struggle with optimum optima or lack geometric consistency.
  arXiv  Detail & Related papers  (2026-05-27T14:02:20Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Hamiltonian Simulation for Advection-Diffusion Equation with arbitrary transport field [0.6608945629704324]
  We present a novel approach to solve the advection-diffusion equation under arbitrary transporting fields using a quantum-inspired 'Schrodingerisation' technique for Hamiltonian simulation.<n>Building on this potential, our quantum algorithm is designed to accommodate non-trivial, spatially varying transport fields.<n>We demonstrate the algorithm's effectiveness on benchmark scenarios involving coupled rotational, shear, and diffusive transport in two and three dimensions.
  arXiv  Detail & Related papers  (2025-08-22T18:02:08Z)
• Learning from Samples: Inverse Problems over measures via Sharpened Fenchel-Young Losses [20.246040671823557]
  We introduce a new class of loss functions, called sharpened Fenchel-Young losses, which measure the sub-optimality gap of the optimization problem over the space of probability measures.<n>We provide explicit stability guarantees for two relevant settings in the context of optimal transport.<n>We present optimization algorithms specifically tailored to efficiently solve iUOT and iJKO problems.
  arXiv  Detail & Related papers  (2025-05-11T21:26:44Z)
• A Unified Theory of Stochastic Proximal Point Methods without Smoothness [52.30944052987393]
  Proximal point methods have attracted considerable interest owing to their numerical stability and robustness against imperfect tuning. This paper presents a comprehensive analysis of a broad range of variations of the proximal point method (SPPM)
  arXiv  Detail & Related papers  (2024-05-24T21:09:19Z)
• Momentum Particle Maximum Likelihood [2.4561590439700076]
  We propose an analogous dynamical-systems-inspired approach to minimizing the free energy functional. By discretizing the system, we obtain a practical algorithm for Maximum likelihood estimation in latent variable models. The algorithm outperforms existing particle methods in numerical experiments and compares favourably with other MLE algorithms.
  arXiv  Detail & Related papers  (2023-12-12T14:53:18Z)
• Accelerating optimization over the space of probability measures [17.32262527237843]
  We introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order.
  arXiv  Detail & Related papers  (2023-10-06T04:32:15Z)
• Probabilistic deconstruction of a theory of gravity, Part I: flat space [0.0]
  We show that Einstein's equations of the theory arise in the semi-classical limit of the quantum evolution of probability under the process. In particular, in flat Jackiw-Teitelboim gravity, the area of compactified space solved for by Einstein's equations can be identified as a probability density evolving under the Markovian process.
  arXiv  Detail & Related papers  (2021-08-24T18:52:31Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)
• Accelerated Jarzynski Estimator with Deterministic Virtual Trajectories [5.482532589225552]
  The Jarzynski estimator is a powerful tool that uses nonequilibrium statistical physics to numerically obtain partition functions of probability distributions. We present a method to significantly accelerate the convergence by deterministic virtual trajectories generated in augmented state space under Hamiltonian dynamics.
  arXiv  Detail & Related papers  (2021-02-28T14:59:03Z)
• Bernstein-Greene-Kruskal approach for the quantum Vlasov equation [91.3755431537592]
  The one-dimensional stationary quantum Vlasov equation is analyzed using the energy as one of the dynamical variables. In the semiclassical case where quantum tunneling effects are small, an infinite series solution is developed.
  arXiv  Detail & Related papers  (2021-02-18T20:55:04Z)
• Acceleration Methods [57.202881673406324]
  We first use quadratic optimization problems to introduce two key families of acceleration methods. We discuss momentum methods in detail, starting with the seminal work of Nesterov. We conclude by discussing restart schemes, a set of simple techniques for reaching nearly optimal convergence rates.
  arXiv  Detail & Related papers  (2021-01-23T17:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.