Second Order Drifting Models

• URL: http://arxiv.org/abs/2608.07924v1
• Date: Sat, 08 Aug 2026 05:02:55 GMT
• Title: Second Order Drifting Models
• Abstract summary: Drifting models evolve the model distribution during training using a predefined sample-based drift field.<n>We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables.<n>We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control.
• Score: 9.446439690148713
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field. Although they avoid iterative inference, their kernel-based drift fields induce frequency-dependent training dynamics: In the linearized regime, each Fourier mode of the density residual decays at a rate determined by the kernel spectrum, leading to slow recovery of fine-scale structure. We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables. We show that the resulting density perturbations obey accelerated second-order dynamics in Fourier space, connecting drifting models to the celebrated Nesterov acceleration from optimization theory. This provides a principled mechanism for mitigating the spectral stiffness of first-order drifting while preserving one-step inference. We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control. Across these settings, the second-order drifting model improves convergence behavior and achieves competitive or superior performance over first-order drifting baselines.

Related papers

• PedestrianDiffusion: Multimodal Generative Denoising and Dense State Estimation for Inertial Navigation [1.8620637029128544]
  PedestrianDiffusion is a generative framework that reformulates dense 6D state estimation as a continuous conditional denoising process.<n>It achieves state-of-the-art performance, exhibiting unprecedented robustness to impulse perturbations and coupled 6D kinematic drift.<n>This work provides a rigorous algorithmic blueprint for next-generation Neural Inertial Measurement Units (N-IMUs)
  arXiv  Detail & Related papers  (2026-07-03T14:04:30Z)
• Drift Flow Matching [69.16550765154163]
  We propose Drift Flow Matching, a framework that connects drifting generative modeling with flow-based iterative generation.<n>DFM preserves the efficiency of direct transport maps while enabling generation to be refined through multiple inference steps.
  arXiv  Detail & Related papers  (2026-05-17T03:58:01Z)
• DriftXpress: Faster Drifting Models via Projected RKHS Fields [7.80855192041271]
  DriftXpress is an accelerated formulation of drifting models based on RKHS fields.<n>It preserves the attraction-repulsion structure of the original drifting field while reducing the cost of field evaluation.<n>Across image-generation benchmarks, DriftXpress achieves comparable FID to standard drifting while reducing wall-clock training cost.
  arXiv  Detail & Related papers  (2026-05-12T14:26:31Z)
• A solver-in-the-loop framework for end-to-end differentiable coastal hydrodynamics [4.010598744735379]
  We introduce AegirJAX, a differentiable hydrodynamic solver based on the depth-integrated, non-static shallow-water equations.<n>AegirJAX treats the time-marching physics loop as a continuous computational graph.<n>We demonstrate the framework's versatility across a suite of scientific machine learning tasks.
  arXiv  Detail & Related papers  (2026-04-08T14:21:47Z)
• Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Framework [52.632217600064344]
  Full-waveform inversion (FWI) estimates physical parameters in the wave equation from limited measurements.<n>FWI methods are limited by their notorious sensitivity to the accuracy of the initial models.<n>Recent progress in continuous representation FWI (CR-FWI) demonstrates that representing parameter models with a coordinate-based neural network, such as implicit neural representation (INR), can mitigate the dependence on initial models.
  arXiv  Detail & Related papers  (2026-03-23T03:22:33Z)
• Towards Fast Coarse-graining and Equation Discovery with Foundation Inference Models [6.403678133359229]
  latent dynamics in high-dimensional recordings are often characterized by a much smaller set of effective variables.<n>Most machine learning approaches tackle these tasks jointly by training autoencoders together with models that enforce dynamical consistency.<n>We propose to decouple the two problems by leveraging the recently introduced Foundation Inference Models (FIMs)<n>A proof of concept on a double-well system with semicircle diffusion, embedded into synthetic video data, illustrates the potential of this approach for fast and reusable coarse-graining pipelines.
  arXiv  Detail & Related papers  (2025-10-14T15:17:23Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Improving Consistency Models with Generator-Augmented Flows [16.049476783301724]
  Consistency models imitate the multi-step sampling of score-based diffusion in a single forward pass of a neural network.<n>They can be learned in two ways: consistency distillation and consistency training.<n>We propose a novel flow that transports noisy data towards their corresponding outputs derived from a consistency model.
  arXiv  Detail & Related papers  (2024-06-13T20:22:38Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion [29.489737359897312]
  We study the limiting dynamics of deep neural networks trained with gradient descent (SGD) We show that the key ingredient driving these dynamics is not the original training loss, but rather the combination of a modified loss, which implicitly regularizes the velocity and probability currents, which cause oscillations in phase space.
  arXiv  Detail & Related papers  (2021-07-19T20:18:57Z)
• Haar Wavelet based Block Autoregressive Flows for Trajectories [129.37479472754083]
  Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents. We introduce a novel Haar wavelet based block autoregressive model leveraging split couplings. We illustrate the advantages of our approach for generating diverse and accurate trajectories on two real-world datasets.
  arXiv  Detail & Related papers  (2020-09-21T13:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.