Implicit Bias of the JKO Scheme

• URL: http://arxiv.org/abs/2511.14827v1
• Date: Tue, 18 Nov 2025 18:48:37 GMT
• Title: Implicit Bias of the JKO Scheme
• Abstract summary: We study emphJKO-Flow, Wasserstein gradient flow on $J$, in several simple numerical examples.<n>We show that $_k$ are approximated to order $2$ by Wasserstein gradient flow on a emphmodified energy.
• Score: 9.659199680879262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Wasserstein gradient flow provides a general framework for minimizing an energy functional $J$ over the space of probability measures on a Riemannian manifold $(M,g)$. Its canonical time-discretization, the Jordan-Kinderlehrer-Otto (JKO) scheme, produces for any step size $η>0$ a sequence of probability distributions $ρ_k^η$ that approximate to first order in $η$ Wasserstein gradient flow on $J$. But the JKO scheme also has many other remarkable properties not shared by other first order integrators, e.g. it preserves energy dissipation and exhibits unconditional stability for $λ$-geodesically convex functionals $J$. To better understand the JKO scheme we characterize its implicit bias at second order in $η$. We show that $ρ_k^η$ are approximated to order $η^2$ by Wasserstein gradient flow on a \emph{modified} energy \[ J^η(ρ) = J(ρ) - \fracη{4}\int_M \Big\lVert \nabla_g \frac{δJ}{δρ} (ρ) \Big\rVert_{2}^{2} \,ρ(dx), \] obtained by subtracting from $J$ the squared metric curvature of $J$ times $η/4$. The JKO scheme therefore adds at second order in $η$ a \textit{deceleration} in directions where the metric curvature of $J$ is rapidly changing. This corresponds to canonical implicit biases for common functionals: for entropy the implicit bias is the Fisher information, for KL-divergence it is the Fisher-Hyv{ä}rinen divergence, and for Riemannian gradient descent it is the kinetic energy in the metric $g$. To understand the differences between minimizing $J$ and $J^η$ we study \emph{JKO-Flow}, Wasserstein gradient flow on $J^η$, in several simple numerical examples. These include exactly solvable Langevin dynamics on the Bures-Wasserstein space and Langevin sampling from a quartic potential in 1D.

Related papers

• JKO-RAG: Distributional Retrieval as Wasserstein Free-Energy Gradient Flow [0.0]
  We propose jko, which frames reranking as minimising a free-energy functional $F(p)=textrelevance+textentropy+textredundancy$ under Wasserstein-2 gradient flow.<n>Four extensions are introduced: textbfnmjko (learned ground metric), textbfbwjko ($W2$-KL), textbfsamjko ($2times$ speedup), and textbfdualrank (OT dual potentials as confidence signals
  arXiv  Detail & Related papers  (2026-06-15T04:34:49Z)
• OAM-Induced Lattice Rotation Reveals a Fractional Optimum in Fault-Tolerant GKP Quantum Sensing [0.0]
  We establish that orbital-angular-momentum encoding and Gottesman-Kitaev-Preskill (GKP) lattice geometry are structurally coupled.<n>Using an end-to-end differentiable Strawberry Fields--TensorFlow circuit, we jointly optimise $ell$, the lattice aspect ratio $r$, and the finite-energy envelope $$ to maximise quantum Fisher information.
  arXiv  Detail & Related papers  (2026-05-13T09:49:16Z)
• Which Coherence Decoheres? Basis-Dependent Decoherence Rates in Symmetry-Broken Collective Spin Systems [0.0]
  Lindblad dephasing rates differ by a factor approaching $2$ as $Ntoinfty$ and reaching $2.42$ near the quantum-critical crossover.<n>The discrepancy has a single algebraic origin: parity forces $langle E_i|hatJ_z|E_irangle=0$ exactly, eliminating the cross-term that doubles the localised-state rate.
  arXiv  Detail & Related papers  (2026-05-01T11:39:51Z)
• Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition [50.36362492608702]
  We study entrywise scalar quantization of two matrices prior to multiplication.<n>We obtain a closed-form optimal point density [ star(u) propto exp!left(-fracu26right)bigl( (1-2)+2u22bigr), qquad u=fracx_X,, and prove a correlation-driven phase transition.
  arXiv  Detail & Related papers  (2026-03-20T01:53:44Z)
• Quantitative Convergence of Wasserstein Gradient Flows of Kernel Mean Discrepancies [10.511277414974613]
  We study the quantitative convergence of Wasserstein gradient flows of Kernel Mean Discrepancy functionals.<n>Our setting covers in particular the training dynamics of shallow neural networks in the infinite-width and continuous time limit.
  arXiv  Detail & Related papers  (2026-03-02T15:32:54Z)
• Wigner quasi-probability distribution for symmetric multi-quDit systems and their generalized heat kernel [0.0]
  We analyze the phase-space structure of Schr"odinger $U(D)$-spin cat states.<n>We compute the generalized heat kernel relating two quasi-probability distributions $mathcalF(s)_rho$ and $mathcalF(s')_rho$.
  arXiv  Detail & Related papers  (2025-07-20T08:26:28Z)
• Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction [57.93371273485736]
  We consider a centralized distributed learning setup where all workers jointly find an unbiased bound LDeltaepsilon2,$ better poly-logarithmically in $n$, even in the homogeneous (i.i.d.) case, where all workers access the same distribution.
  arXiv  Detail & Related papers  (2025-06-30T13:27:39Z)
• Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent [14.890609936348277]
  We provide finite-particle convergence rates for the Stein Variational Gradient Descent algorithm in the Kernelized Stein Discrepancy ($mathsfKSD$) and Wasserstein-2 metrics.<n>Our key insight is that the time derivative of the relative entropy between the joint density of $N$ particle locations splits into a dominant negative part' proportional to $N$ times the expected $mathsfKSD2$ and a smaller positive part'
  arXiv  Detail & Related papers  (2024-09-13T01:49:19Z)
• A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing [68.80803866919123]
  Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
  arXiv  Detail & Related papers  (2023-09-25T17:54:19Z)
• Measurement-induced phase transition for free fermions above one dimension [46.176861415532095]
  Theory of the measurement-induced entanglement phase transition for free-fermion models in $d>1$ dimensions is developed. Critical point separates a gapless phase with $elld-1 ln ell$ scaling of the second cumulant of the particle number and of the entanglement entropy.
  arXiv  Detail & Related papers  (2023-09-21T18:11:04Z)
• Projected Langevin dynamics and a gradient flow for entropic optimal transport [0.8057006406834466]
  We introduce analogous diffusion dynamics that sample from an entropy-regularized optimal transport. By studying the induced Wasserstein geometry of the submanifold $Pi(mu,nu)$, we argue that the SDE can be viewed as a Wasserstein gradient flow on this space of couplings.
  arXiv  Detail & Related papers  (2023-09-15T17:55:56Z)
• Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models [91.22420505636006]
  This paper deals with the problem of efficient sampling from a differential equation, given the drift function and the diffusion matrix. It is possible to obtain independent and identically distributed (i.i.d.) samples at precision $varepsilon$ with a cost that is $m2 d log (1/varepsilon)$ Our results suggest that as the true solution gets smoother, we can circumvent the curse of dimensionality without requiring any sort of convexity.
  arXiv  Detail & Related papers  (2023-03-30T02:50:49Z)
• ReSQueing Parallel and Private Stochastic Convex Optimization [59.53297063174519]
  We introduce a new tool for BFG convex optimization (SCO): a Reweighted Query (ReSQue) estimator for the gradient of a function convolved with a (Gaussian) probability density. We develop algorithms achieving state-of-the-art complexities for SCO in parallel and private settings.
  arXiv  Detail & Related papers  (2023-01-01T18:51:29Z)
• Local Max-Entropy and Free Energy Principles Solved by Belief Propagation [0.0]
  A statistical system is classically defined on a set of microstates $E$ by a global energy function $H : E to mathbbR$, yielding Gibbs probability measures $rhobeta(H)$ for every inverse temperature $beta = T-1$. We show that the generalized belief propagation algorithm solves a collection of local variational principles, by converging to critical points of Bethe-Kikuchi approximations of the free energy $F(beta)$, the Shannon entropy $S(cal U)$, and the variational free energy
  arXiv  Detail & Related papers  (2022-07-02T14:20:40Z)
• Learning Green's functions associated with parabolic partial differential equations [1.5293427903448025]
  We derive the first theoretically rigorous scheme for learning the associated Green's function $G$. We extend the low-rank theory of Bebendorf and Hackbusch from elliptic PDEs in dimension $1leq nleq 3$ to parabolic PDEs in any dimensions.
  arXiv  Detail & Related papers  (2022-04-27T09:23:39Z)
• Sparse sketches with small inversion bias [79.77110958547695]
  Inversion bias arises when averaging estimates of quantities that depend on the inverse covariance. We develop a framework for analyzing inversion bias, based on our proposed concept of an $(epsilon,delta)$-unbiased estimator for random matrices. We show that when the sketching matrix $S$ is dense and has i.i.d. sub-gaussian entries, the estimator $(epsilon,delta)$-unbiased for $(Atop A)-1$ with a sketch of size $m=O(d+sqrt d/
  arXiv  Detail & Related papers  (2020-11-21T01:33:15Z)
• Data-driven Efficient Solvers for Langevin Dynamics on Manifold in High Dimensions [12.005576001523515]
  We study the Langevin dynamics of a physical system with manifold structure $mathcalMsubsetmathbbRp$ We leverage the corresponding Fokker-Planck equation on the manifold $mathcalN$ in terms of the reaction coordinates $mathsfy$. We propose an implementable, unconditionally stable, data-driven finite volume scheme for this Fokker-Planck equation.
  arXiv  Detail & Related papers  (2020-05-22T16:55:38Z)
• Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness [151.67113334248464]
  We show that extending the smoothing technique to defend against other attack models can be challenging. We present experimental results on CIFAR to validate our theory.
  arXiv  Detail & Related papers  (2020-02-08T22:02:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.