Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamics

• URL: http://arxiv.org/abs/2108.08247v1
• Date: Wed, 18 Aug 2021 17:11:29 GMT
• Title: Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamics
• Abstract summary: We introduce a novel geometry-informed irreversible perturbation that accelerates convergence of the Langevin algorithm for Bayesian computation. We show that this new irreversible perturbation can improve performance of the estimator over reversible perturbations that do not take the geometry into account.
• Score: 1.675857332621569
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel geometry-informed irreversible perturbation that accelerates convergence of the Langevin algorithm for Bayesian computation. It is well documented that there exist perturbations to the Langevin dynamics that preserve its invariant measure while accelerating its convergence. Irreversible perturbations and reversible perturbations (such as Riemannian manifold Langevin dynamics (RMLD)) have separately been shown to improve the performance of Langevin samplers. We consider these two perturbations simultaneously by presenting a novel form of irreversible perturbation for RMLD that is informed by the underlying geometry. Through numerical examples, we show that this new irreversible perturbation can improve performance of the estimator over reversible perturbations that do not take the geometry into account. Moreover we demonstrate that irreversible perturbations generally can be implemented in conjunction with the stochastic gradient version of the Langevin algorithm. Lastly, while continuous-time irreversible perturbations cannot impair the performance of a Langevin estimator, the situation can sometimes be more complicated when discretization is considered. To this end, we describe a discrete-time example in which irreversibility increases both the bias and variance of the resulting estimator.

Related papers

• Variance Reduction for Stochastic Gradient Generalized Non-reversible Langevin Monte Carlo Algorithms [7.336637417508747]
  We study the leading-order fluctuation of gradient-Maruyama estimators for non-reversible Langevin dynamics.<n>We prove that the empirical average over a horizon of order the inverse squared stepsize satisfies a central limit theorem in the vanishing-stepsize regime.
  arXiv  Detail & Related papers  (2026-06-27T08:25:48Z)
• Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation [73.84423888025171]
  Autoregressive (AR) video diffusion has recently emerged as a promising paradigm for long video generation.<n>We show that extending generation to minute-scale horizons remains challenging due to progressive temporal degradation.<n>We introduce Relax Forcing, a structured temporal memory mechanism for AR diffusion.
  arXiv  Detail & Related papers  (2026-03-22T18:59:24Z)
• SSR: A Training-Free Approach for Streaming 3D Reconstruction [53.19807901585702]
  Self-expressive Sequence Regularization (SSR) is a plug-and-play operator that enforces Grassmannian sequence regularity during inference.<n>We show that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.
  arXiv  Detail & Related papers  (2026-03-16T02:55:14Z)
• Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance [2.336538451986937]
  We show that deliberately breaking detailed balance in generative diffusion processes can accelerate the reverse process without changing the stationary distribution.<n>We analyze how such non-reversible control reshapes the macroscopic dynamical regimes of the phase transitions recently identified in generative diffusion models.
  arXiv  Detail & Related papers  (2026-02-17T03:05:37Z)
• An Inverse Scattering Inspired Fourier Neural Operator for Time-Dependent PDE Learning [0.0]
  We introduce an inverse scattering inspired Fourier Neural Operator (IS-FNO)<n>IS-FNO achieves lower short-term errors and substantially improved long-horizon stability in non-stiff regimes.<n>Overall, this work shows that incorporating physical structure -- particularly reversibility and spectral evolution -- into neural operator design significantly enhances robustness and long-term predictive fidelity for nonlinear PDE dynamics.
  arXiv  Detail & Related papers  (2025-12-22T14:40:13Z)
• Regulated reconstruction of long-time spin--boson dynamics and emergent zero-bias transverse measurement primitive [0.0]
  We reconstruct a time-local correlatative generator's map around a Davies reference semigroup.<n>An exact rotating-wave benchmark links generator growth to interference-induced near-zeros of the coherence.<n>The effect disappears in the rotating-wave approximation and in the Davies weak-coupling limit.
  arXiv  Detail & Related papers  (2025-12-15T21:11:00Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Fisher information dissipation for time inhomogeneous stochastic differential equations [7.076726009680242]
  We provide a Lyapunov convergence analysis for time-inhomogeneous variable coefficient differential equations. Three typical examples include overdamped, irreversible drift, and underdamped Langevin dynamics.
  arXiv  Detail & Related papers  (2024-02-01T21:49:50Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction [49.66486092259376]
  The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. We provide a framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and gradient approximation.
  arXiv  Detail & Related papers  (2023-06-12T16:28:11Z)
• Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability [69.01076284478151]
  In machine learning optimization, gradient descent (GD) often operates at the edge of stability (EoS) This paper studies the convergence and implicit bias of constant-stepsize GD for logistic regression on linearly separable data in the EoS regime.
  arXiv  Detail & Related papers  (2023-05-19T16:24:47Z)
• Transport map unadjusted Langevin algorithms: learning and discretizing perturbed samplers [1.993607565985189]
  We study the use of transport maps that normalize a target distribution as a way to precondition and accelerate the convergence of Langevin dynamics. We also show that applying a transport map to an irreversibly-perturbed ULA results in a geometry-informed irreversible perturbation (GiIrr) of the original dynamics.
  arXiv  Detail & Related papers  (2023-02-14T18:13:19Z)
• Interpolation can hurt robust generalization even when there is no noise [76.3492338989419]
  We show that avoiding generalization through ridge regularization can significantly improve generalization even in the absence of noise. We prove this phenomenon for the robust risk of both linear regression and classification and hence provide the first theoretical result on robust overfitting.
  arXiv  Detail & Related papers  (2021-08-05T23:04:15Z)
• The Connection between Discrete- and Continuous-Time Descriptions of Gaussian Continuous Processes [60.35125735474386]
  We show that discretizations yielding consistent estimators have the property of invariance under coarse-graining' This result explains why combining differencing schemes for derivatives reconstruction and local-in-time inference approaches does not work for time series analysis of second or higher order differential equations.
  arXiv  Detail & Related papers  (2021-01-16T17:11:02Z)
• Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction [24.794221009364772]
  We study the reduction for a noisy energy estimators variance, which promotes much more effective analysis. We obtain the state-of-the-art results in optimization and uncertainty estimates for synthetic experiments and image data.
  arXiv  Detail & Related papers  (2020-10-02T16:23:35Z)
• Fast Mixing of Multi-Scale Langevin Dynamics under the Manifold Hypothesis [85.65870661645823]
  We show how the Langevin dimension algorithm allows for the considerable reduction of time depending on the (much) smaller data. Second, the high of the sampling space significantly hurts the performance of Langevin Dynamics.
  arXiv  Detail & Related papers  (2020-06-19T14:52:40Z)
• Stochastic Gradient Langevin with Delayed Gradients [29.6870062491741]
  We show that the rate of convergence in measure is not significantly affected by the error caused by the delayed gradient information used for computation. We show that the rate of convergence in measure is not significantly affected by the error caused by the delayed gradient information used for computation, suggesting significant potential for speedup in wall clock time.
  arXiv  Detail & Related papers  (2020-06-12T17:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.