FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling

• URL: http://arxiv.org/abs/2604.06779v1
• Date: Wed, 08 Apr 2026 07:50:00 GMT
• Title: FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling
• Abstract summary: We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers.<n>Inspired by Fleming-Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment.
• Score: 18.997552937110548
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomial resampling or closely related resampling schemes, which can still reduce diversity and lead to lineage collapse under strong selection pressure. Inspired by Fleming-Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment. To handle cases where rewards are only approximately available and naive rebirth would collapse deterministic trajectories, FVD integrates independent reward-based survival decisions with stochastic rebirth noise. This yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions, all without requiring value function approximation or costly rollouts. FVD is fully parallelizable and scales efficiently with inference compute. Empirically, it achieves substantial gains across settings: on DrawBench it outperforms prior methods by 7% in ImageReward, while on class-conditional tasks it improves FID by roughly 14-20% over strong baselines and is up to 66 times faster than value-based approaches.

Related papers

• DiffIE: Diffusion-based Open Information Extraction [4.37919186746582]
  We introduce DIFFIE, which treats conditional discrete diffusion as the extraction mechanism itself.<n>DIFFIE achieves the new state of the art in CaRB, and outperforms the strongest rule-based system (ClausIE) in BenchIE.
  arXiv  Detail & Related papers  (2026-09-02T09:01:42Z)
• KANResDiff: Learning Local Residual Diffusion via Kolmogorov-Arnold Network for Ambiguous Medical Image Segmentation [10.667399216244169]
  We propose KANResDiff to learn local residual diffusion with Kolmogorov-Arnold Network.<n>We show that KANResDiff achieves SOTA performance on GED and HM-IoU, with maximum improvements of 16.8% and 7.7%, respectively.
  arXiv  Detail & Related papers  (2026-08-12T03:54:27Z)
• Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion [33.94204857658877]
  We introduce Contrastive Distribution Matching (CDM), a novel framework that amortizes the cost of SMC inference by learning a parameterized twist function via positive and negative samples.<n>In practice, evaluating our learned twist function incurs less than 5% additional computational overhead compared to a single forward pass of the base model.<n>We validate the effectiveness and versatility of our approach across a diverse range of applications, including toxic text generation, regulatory DNA sequence design, protein designability, and diffusion large language model alignment.
  arXiv  Detail & Related papers  (2026-05-22T08:06:52Z)
• Continuous-Time Distribution Matching for Few-Step Diffusion Distillation [57.28746398500951]
  We introduce Continuous-Time Distribution Matching (CDM), migrating the DMD framework from discrete anchoring to continuous optimization.<n>CDM achieves this through two continuous-time designs. First, we replace the fixed discrete schedule with a dynamic continuous schedule of random length.<n>Second, we propose a continuous-time alignment objective that performs active off-trajectory matching on latents extrapolated via the student's velocity field.
  arXiv  Detail & Related papers  (2026-05-07T14:56:39Z)
• DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving [65.7087560656003]
  Generative diffusion models for end-to-end autonomous driving often suffer from mode collapse.<n>We propose DiffusionDriveV2, which leverages reinforcement learning to constrain low-quality modes and explore for superior trajectories.<n>This significantly enhances the overall output quality while preserving the inherent multimodality of its core Gaussian Mixture Model.
  arXiv  Detail & Related papers  (2025-12-08T17:29:52Z)
• ReDDiT: Rehashing Noise for Discrete Visual Generation [53.813067778912]
  We propose a rehashing noise approach for discrete diffusion transformer (termed ReDDiT)<n>We aim to extend absorbing states and improve expressive capacity of discrete diffusion models.<n>Experiments show that ReDDiT significantly outperforms the baseline model.
  arXiv  Detail & Related papers  (2025-05-26T08:17:20Z)
• Sequential Controlled Langevin Diffusions [94.82767690147865]
  Two popular methods are (1) Sequential Monte Carlo (SMC), where the transport is performed through successive densities via prescribed Markov chains and resampling steps, and (2) recently developed diffusion-based sampling methods, where a learned dynamical transport is used.<n>We present a principled framework for combining SMC with diffusion-based samplers by viewing both methods in continuous time and considering measures on path space.<n>This culminates in the new Sequential Controlled Langevin Diffusion (SCLD) sampling method, which is able to utilize the benefits of both methods and reaches improved performance on multiple benchmark problems, in many cases using only 10% of the training budget of previous diffusion-
  arXiv  Detail & Related papers  (2024-12-10T00:47:10Z)
• Improving Vector-Quantized Image Modeling with Latent Consistency-Matching Diffusion [55.185588994883226]
  We introduce VQ-LCMD, a continuous-space latent diffusion framework within the embedding space that stabilizes training.<n>VQ-LCMD uses a novel training objective combining the joint embedding-diffusion variational lower bound with a consistency-matching (CM) loss.<n>Experiments show that the proposed VQ-LCMD yields superior results on FFHQ, LSUN Churches, and LSUN Bedrooms compared to discrete-state latent diffusion models.
  arXiv  Detail & Related papers  (2024-10-18T09:12:33Z)
• Generative Fractional Diffusion Models [53.36835573822926]
  We introduce the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics. Our evaluations on real image datasets demonstrate that GFDM achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID.
  arXiv  Detail & Related papers  (2023-10-26T17:53:24Z)
• Fast Diffusion Model [122.36693015093041]
  Diffusion models (DMs) have been adopted across diverse fields with their abilities in capturing intricate data distributions. In this paper, we propose a Fast Diffusion Model (FDM) to significantly speed up DMs from a DM optimization perspective.
  arXiv  Detail & Related papers  (2023-06-12T09:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.