Integration Matters: Rollout-Based Training for Constrained Diffusion Models

• URL: http://arxiv.org/abs/2607.14398v1
• Date: Wed, 15 Jul 2026 22:25:59 GMT
• Title: Integration Matters: Rollout-Based Training for Constrained Diffusion Models
• Abstract summary: Constrained generative models aim to produce samples that satisfy complex feasibility constraints while remaining faithful to the data distribution.<n>Existing constrained generation methods enforce constraints either through training-time optimization or sampling-time correction.<n>We propose a fine-tuning framework that incorporates constraint guidance obtained through online rollout into the training process.
• Score: 16.46255955344152
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Constrained generative models aim to produce samples that satisfy complex feasibility constraints while remaining faithful to the data distribution. Existing constrained generation methods typically enforce constraints either through training-time optimization or sampling-time correction. Training-time optimization approaches optimize on states induced by the training distribution, which can differ substantially from those encountered during sampling. Sampling-time correction methods instead modify the sampling process at inference, introducing distribution shift and requiring expensive tuning, particularly for few-step sampling. We propose a fine-tuning framework that incorporates constraint guidance obtained through online rollout into the training process, which aligns training with sampling by differentiating through the fixed noise schedule used to numerically integrate the denoising process. This exposes the model to violations that arise along the denoising trajectory and aligns diffusion learning with the sampling process. Experiments across multiple tasks show that our method improves constraint satisfaction while maintaining competitive sampling quality compared to prior methods.

Related papers

• Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding [56.38477453373368]
  LazyMCoT is a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty.<n>LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency.
  arXiv  Detail & Related papers  (2026-06-15T03:17:44Z)
• Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling [45.70806419699897]
  Deep generative models provide state-of-the-art performance across a wide array of applications.<n>Existing approaches fail to enforce strict constraint satisfaction while maintaining sample quality.<n>This paper proposes Constraint-Aware Flow Matching, a novel end-to-end framework that explicitly incorporates constraint projections into the training objective.
  arXiv  Detail & Related papers  (2026-05-12T21:07:13Z)
• Inference-Time Attribute Distribution Alignment for Unconditional Diffusion [54.28942858023809]
  We formalize the inference-time attribute distributional alignment problem for pretrained unconditional diffusion models.<n>To address this, we cast inference-time attribute distributional alignment as an optimal control problem over the reverse diffusion process.<n>We solve for the perturbations using an optimal-control-based algorithm to optimize a differentiable distribution-matching objective.
  arXiv  Detail & Related papers  (2026-05-08T09:02:47Z)
• Few-Step Diffusion Sampling Through Instance-Aware Discretizations [16.93105640007355]
  Diffusion and flow matching models generate high-fidelity data by simulating paths defined by Ordinary or Differential Equations (ODEs/SDEs)<n>We propose an instance-aware discretization framework, which learns to adapt timestep allocations based on input-dependent priors.<n>Our method consistently improves generation quality with marginal tuning cost compared to training and negligible inference overhead.
  arXiv  Detail & Related papers  (2026-03-18T12:49:38Z)
• Coarse-Guided Visual Generation via Weighted h-Transform Sampling [12.670888779100972]
  Coarse-guided visual generation is essential for various real-world applications.<n>Recent training-free works propose to leverage pretrained diffusion models and incorporate guidance during the sampling process.<n>We propose a novel guided method by using the htransform, a tool that can constrain processes (e.g., sampling process) under desired conditions.
  arXiv  Detail & Related papers  (2026-03-12T15:26:19Z)
• TFTF: Training-Free Targeted Flow for Conditional Sampling [1.4151684142137693]
  We propose a training-free conditional sampling method for flow matching models based on importance sampling.<n>Because a nave application of importance sampling suffers from weighteneracy in high-dimensional settings, we modify and incorporate a resampling technique in sequential Monte Carlo.<n>Our framework requires no additional training, while providing theoretical guarantees of accuracy.
  arXiv  Detail & Related papers  (2026-02-13T13:41:35Z)
• Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning [43.678382510171986]
  Diffusion models generate samples through an iterative denoising process, guided by a neural network.<n>We introduce an inverse reinforcement learning framework for learning sampling strategies without retraining the denoiser.<n>We provide experimental evidence that this approach can improve the quality of samples generated by pretrained diffusion models.
  arXiv  Detail & Related papers  (2026-02-09T14:10:44Z)
• Learnable Chernoff Baselines for Inference-Time Alignment [64.81256817158851]
  We introduce Learnable Chernoff Baselines as a method for efficiently and approximately sampling from exponentially tilted kernels.<n>We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling.
  arXiv  Detail & Related papers  (2026-02-08T00:09:40Z)
• Combating Noisy Labels through Fostering Self- and Neighbor-Consistency [120.4394402099635]
  Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning.<n>We propose a noise-robust method named Jo-SNC (textbfJoint sample selection and model regularization based on textbfSelf- and textbfNeighbor-textbfConsistency)<n>We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds.
  arXiv  Detail & Related papers  (2026-01-19T07:55:29Z)
• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Fast T2T: Optimization Consistency Speeds Up Diffusion-Based Training-to-Testing Solving for Combinatorial Optimization [83.65278205301576]
  We propose to learn direct mappings from different noise levels to the optimal solution for a given instance, facilitating high-quality generation with minimal shots.<n>This is achieved through an optimization consistency training protocol, which minimizes the difference among samples.<n>Experiments on two popular tasks, the Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS), demonstrate the superiority of Fast T2T regarding both solution quality and efficiency.
  arXiv  Detail & Related papers  (2025-02-05T07:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.