Kernel-Gradient Drifting Models

• URL: http://arxiv.org/abs/2605.10727v1
• Date: Mon, 11 May 2026 15:33:21 GMT
• Title: Kernel-Gradient Drifting Models
• Abstract summary: kernel-gradient drifting is a one-step generative modeling framework.<n>It replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself.
• Score: 34.79469799316397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose kernel-gradient drifting, a one-step generative modeling framework that replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself. Standard drifting is attractive because it enables fast, high-quality generation without distilling a large pretrained diffusion model, but its theory is currently understood mainly for Gaussian kernels, where the drift coincides with smoothed score matching and is identifiable. Our gradient-based reformulation exposes this score-based structure for general kernels: the resulting drift is the score difference between kernel-smoothed data and model distributions, yielding identifiability for characteristic kernels and a smoothed-KL descent interpretation of the drifting dynamics. Since kernel gradients are intrinsic tangent vectors, the same construction extends naturally to Riemannian manifolds and to discrete data via the Fisher-Rao geometry of the probability simplex. Across spherical geospatial data, promoter DNA and molecule generation, kernel-gradient drifting enables state-of-the-art one-step generation beyond the Euclidean setting without distillation.

Related papers

• Second Order Drifting Models [9.446439690148713]
  Drifting models evolve the model distribution during training using a predefined sample-based drift field.<n>We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables.<n>We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control.
  arXiv  Detail & Related papers  (2026-08-08T05:02:55Z)
• Distilling Drifting Transformers with Representation Autoencoders [23.990618033500695]
  Representation Autoencoders (RAEs) have improved diffusion and flow models by semantically richer latent space.<n>In this work, we argue that the RAE latent space is compatible with distillation via the newly proposed Drifting Models.
  arXiv  Detail & Related papers  (2026-06-14T02:46:50Z)
• DriftXpress: Faster Drifting Models via Projected RKHS Fields [7.80855192041271]
  DriftXpress is an accelerated formulation of drifting models based on RKHS fields.<n>It preserves the attraction-repulsion structure of the original drifting field while reducing the cost of field evaluation.<n>Across image-generation benchmarks, DriftXpress achieves comparable FID to standard drifting while reducing wall-clock training cost.
  arXiv  Detail & Related papers  (2026-05-12T14:26:31Z)
• Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective [33.2175112734242]
  Generative Modeling via Drifting has recently achieved state-of-the-art one-step image generation through a kernel-based drift operator.<n>We make the following observation: emphunder a Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions.
  arXiv  Detail & Related papers  (2026-03-10T17:30:35Z)
• A Unified View of Drifting and Score-Based Models [92.51878018325031]
  We make our relationship to the score-matching principle behind diffusion models precise by showing that drifting admits a score-based formulation on kernel-smoothed distributions.<n>We derive an exact decomposition for general radial kernels, and for the Laplace kernel we prove rigorous error bounds showing that drifting remains an accurate proxy for score matching in low-temperature and high-dimensional regimes.
  arXiv  Detail & Related papers  (2026-03-08T07:41:36Z)
• KROM: Kernelized Reduced Order Modeling [3.988493458010939]
  KROM formulates PDE solution as a minimum-norm (Gaussian-process) recovery problem in an RKHS.<n>A central ingredient is an empirical kernel constructed from a snapshot library of PDE solutions.
  arXiv  Detail & Related papers  (2026-02-27T22:52:22Z)
• Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel [55.82768375605861]
  We establish a generalization bound for gradient flow that aligns with the classical Rademacher complexity for kernel methods.<n>Unlike static kernels such as NTK, the LPK captures the entire training trajectory, adapting to both data and optimization dynamics.
  arXiv  Detail & Related papers  (2025-06-12T23:17:09Z)
• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum [18.10812063219831]
  We introduce Modified Spectrum Kernels (MSKs) to approximate kernels with desired eigenvalues. We propose a preconditioned gradient descent method, which alters the trajectory of gradient descent. Our method is both computationally efficient and simple to implement.
  arXiv  Detail & Related papers  (2023-07-26T22:39:47Z)
• Neural Fields as Learnable Kernels for 3D Reconstruction [101.54431372685018]
  We present a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression. Our technique achieves state-of-the-art results when reconstructing 3D objects and large scenes from sparse oriented points.
  arXiv  Detail & Related papers  (2021-11-26T18:59:04Z)
• Flow-based Kernel Prior with Application to Blind Super-Resolution [143.21527713002354]
  Kernel estimation is generally one of the key problems for blind image super-resolution (SR) This paper proposes a normalizing flow-based kernel prior (FKP) for kernel modeling. Experiments on synthetic and real-world images demonstrate that the proposed FKP can significantly improve the kernel estimation accuracy.
  arXiv  Detail & Related papers  (2021-03-29T22:37:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.