Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective

• URL: http://arxiv.org/abs/2603.09936v1
• Date: Tue, 10 Mar 2026 17:30:35 GMT
• Title: Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
• Abstract summary: Generative Modeling via Drifting has recently achieved state-of-the-art one-step image generation through a kernel-based drift operator.<n>We make the following observation: emphunder a Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions.
• Score: 33.2175112734242
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generative Modeling via Drifting has recently achieved state-of-the-art one-step image generation through a kernel-based drift operator, yet the success is largely empirical and its theoretical foundations remain poorly understood. In this paper, we make the following observation: \emph{under a Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions}. This insight allows us to answer all three key questions left open in the original work: (1) whether a vanishing drift guarantees equality of distributions ($V_{p,q}=0\Rightarrow p=q$), (2) how to choose between kernels, and (3) why the stop-gradient operator is indispensable for stable training. Our observations position drifting within the well-studied score-matching family and enable a rich theoretical perspective. By linearizing the McKean-Vlasov dynamics and probing them in Fourier space, we reveal frequency-dependent convergence timescales comparable to \emph{Landau damping} in plasma kinetic theory: the Gaussian kernel suffers an exponential high-frequency bottleneck, explaining the empirical preference for the Laplacian kernel. We also propose an exponential bandwidth annealing schedule $σ(t)=σ_0 e^{-rt}$ that reduces convergence time from $\exp(O(K_{\max}^2))$ to $O(\log K_{\max})$. Finally, by formalizing drifting as a Wasserstein gradient flow of the smoothed KL divergence, we prove that the stop-gradient operator is derived directly from the frozen-field discretization mandated by the JKO scheme, and removing it severs training from any gradient-flow guarantee. This variational perspective further provides a general template for constructing novel drift operators, demonstrated with a Sinkhorn divergence drift.

Related papers

• LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction [59.32834240526981]
  Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE)<n>Existing GRPO methods for flow models replace the Euler-time ODE with aotropic differential equation (SDE) during training.<n>We introduce LC-GRPO, a flow-based GRPO framework with Langevin correction targeting the marginal distribution at the resulting timestep.
  arXiv  Detail & Related papers  (2026-08-06T04:47:57Z)
• Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models [5.256913964154542]
  We propose and analyze a conservative drifting method for one-step generative modeling.<n>The method replaces the original displacement-based drifting velocity by a kernel density estimator (KDE)-gradient velocity.<n>We also analyze the non-conservative drifting method with Laplace kernel, corresponding to the original displacement-based velocity proposed in Deng et al., 2026.
  arXiv  Detail & Related papers  (2026-05-21T17:49:09Z)
• Attraction, Repulsion, and Friction: Introducing DMF, a Friction-Augmented Drifting Model [37.847379736905175]
  We show that a linearly-scheduled friction gives a finite-horizon bound on the error trajectory.<n>Our friction-augmented model, DMF, matches or exceeds Optimal Flow Matching on FFHQ coefficient adult-to-child domain translation.
  arXiv  Detail & Related papers  (2026-04-20T12:47:27Z)
• On Interpolation Formulas Describing Neural Network Generalization [8.154667543705687]
  We introduce a gradient kernel that extends the deterministic version via a continuous-time diffusion.<n>It reveals that training samples contribute through loss-dependent weights and gradient alignment along the trajectory.<n>We visualize then link the generalization error to the null space of the integral operator induced by the gradient kernel.
  arXiv  Detail & Related papers  (2026-03-14T10:03:29Z)
• Sinkhorn-Drifting Generative Models [27.763380775285466]
  "Drifting" generative dynamics and gradient flows induced by Sinkhorn divergence.<n>Drifting reduces sensitivity to kernel temperature and improves one-step generative quality.<n>These theoretical gains translate to strong low-temperature improvements in practice.
  arXiv  Detail & Related papers  (2026-03-12T18:33:36Z)
• A Unified View of Drifting and Score-Based Models [92.51878018325031]
  We make our relationship to the score-matching principle behind diffusion models precise by showing that drifting admits a score-based formulation on kernel-smoothed distributions.<n>We derive an exact decomposition for general radial kernels, and for the Laplace kernel we prove rigorous error bounds showing that drifting remains an accurate proxy for score matching in low-temperature and high-dimensional regimes.
  arXiv  Detail & Related papers  (2026-03-08T07:41:36Z)
• Renormalizable Spectral-Shell Dynamics as the Origin of Neural Scaling Laws [2.779943773196378]
  We show that deep-network training obeys a simple macroscopic structure despite highly nonlinear optimization dynamics.<n>For mean-squared error loss, the training error evolves as $dot e_t=-M(t)e_t$ with $M(t)=J_(t)J_(t)!*$, a time-dependent self-adjoint operator induced by the network Jacobian.<n>This framework explains neural scaling laws and double descent, and unifies lazy (NTK-like) training and feature learning as two limits of the same spectral-shell
  arXiv  Detail & Related papers  (2025-12-11T08:38:46Z)
• Contraction and entropy production in continuous-time Sinkhorn dynamics [0.6423239719448169]
  We give an exact identity for the entropy production rate of the Sinkhorn flow, which was previously known only to be nonpositive.<n>We show that the flow induces a reversible Markov dynamics on the target marginal as an Onsager gradient flow.<n>We give for illustration two immediate practical use-cases for the Sinkhorn LSI: as a design principle for the latent space in which generative models are trained, and as a stopping algorithm for discrete-time algorithms.
  arXiv  Detail & Related papers  (2025-10-14T15:32:15Z)
• Driving a Quantum Phase Transition at Arbitrary Rate: Exact solution of the Transverse-Field Ising model [0.0]
  We study the crossing of the quantum phase transition in the transverse-field Ising model after the magnetic field at an arbitrary rate.<n>We analyze the defect density, the complete kink number distribution, and its cumulants upon completion of a linearized quench.
  arXiv  Detail & Related papers  (2025-01-16T19:03:37Z)
• On the Wasserstein Convergence and Straightness of Rectified Flow [54.580605276017096]
  Rectified Flow (RF) is a generative model that aims to learn straight flow trajectories from noise to data.<n>We provide a theoretical analysis of the Wasserstein distance between the sampling distribution of RF and the target distribution.<n>We present general conditions guaranteeing uniqueness and straightness of 1-RF, which is in line with previous empirical findings.
  arXiv  Detail & Related papers  (2024-10-19T02:36:11Z)
• Accelerated Bayesian imaging by relaxed proximal-point Langevin sampling [4.848683707039751]
  This paper presents a new accelerated proximal Markov chain Monte Carlo methodology to perform Bayesian inference in imaging inverse problems. For models that are smooth or regularised by Moreau-Yosida smoothing, the midpoint is equivalent to an implicit discretisation of an overdamped Langevin diffusion. For targets that are $kappa$-strongly log-concave, the provided non-asymptotic convergence analysis also identifies the optimal time step.
  arXiv  Detail & Related papers  (2023-08-18T10:55:49Z)
• An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flow [8.052709336750823]
  We show that when $pirhollback$ exhibits multiple modes, $pirhollback$ is that textitindependent of the potential function. We provide an explicit expansion of $textKL. KL. KL. KL. KL. KL. KL. KL. KL. KL. KL. KL. KL.
  arXiv  Detail & Related papers  (2023-02-23T18:47:54Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Fast Margin Maximization via Dual Acceleration [52.62944011696364]
  We present and analyze a momentum-based method for training linear classifiers with an exponentially-tailed loss. This momentum-based method is derived via the convex dual of the maximum-margin problem, and specifically by applying Nesterov acceleration to this dual.
  arXiv  Detail & Related papers  (2021-07-01T16:36:39Z)
• Debiased Sinkhorn barycenters [110.79706180350507]
  Entropy regularization in optimal transport (OT) has been the driver of many recent interests for Wasserstein metrics and barycenters in machine learning. We show how this bias is tightly linked to the reference measure that defines the entropy regularizer. We propose debiased Wasserstein barycenters that preserve the best of both worlds: fast Sinkhorn-like iterations without entropy smoothing.
  arXiv  Detail & Related papers  (2020-06-03T23:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.