PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model

• URL: http://arxiv.org/abs/2608.06794v1
• Date: Fri, 07 Aug 2026 04:35:26 GMT
• Title: PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model
• Abstract summary: PAST enhances computational efficiency of existing RL fine-tuning methods by up to 66.7%.<n>It also improves preference optimization quality by up to 29.5% through its dual adaptive regulation mechanism.
• Score: 11.180362540642689
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: While diffusion models have made significant progress in text-to-image tasks, they still exhibit limitations when directly optimizing downstream objectives. Although Reinforcement Learning (RL) enables targeted optimization, existing methods are generally constrained by low-efficiency fine-tuning and sparse rewards. To address these challenges, we propose PAST, which provides differentiated rewards while adaptively regulating training episode length by jointly perceiving denoising progress and prompt difficulty. Specifically, we design an intrinsic reward paradigm to compensate for sparse extrinsic rewards and guide the model to explore paths that diverge more efficiently from noise patterns. We further provide theoretical justification for intrinsic rewards. Then, PAST dynamically monitors denoising completion and semantic alignment between image structures and prompt semantics. When both metrics satisfy generation requirements, the system adaptively terminates training. This enables appropriate allocation of episode lengths based on prompt difficulty and the current generation process. Finally, based on the predicted residual noise level, we establish a dual adaptive coordination mechanism. Specifically, it not only balances the extrinsic and intrinsic rewards but also balances the exploration and convergence. Experimental results demonstrate that PAST enhances computational efficiency of existing RL fine-tuning methods by up to 66.7%, while improving preference optimization quality by up to 29.5% through its dual adaptive regulation mechanism.

Related papers

• Explore or Converge? Stage-Guided Per-Step Optimization for Diffusion Models [11.180362540642689]
  Reinforcement learning for preference alignment in diffusion models is promising but limited by reward sparsity.<n>We propose Stage-Guided Per-Step Optimization (SGPO) for diffusion models.<n>Our method achieves 26.7% average gains in generative quality and 36.7% higher convergence speed.
  arXiv  Detail & Related papers  (2026-08-07T03:41:58Z)
• Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models? [30.438505697106496]
  Most studies apply RL to the full denoising trajectory, making it computationally costly and weakening preference alignment.<n>We propose AdaScope, an RL-enhanced plug-in that improves generation quality while reducing computational cost.<n>Compared with state-of-the-art methods, AdaScope improves performance by 66% while cutting computational cost by 59%.
  arXiv  Detail & Related papers  (2026-05-15T11:14:13Z)
• TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignment [52.570581883709345]
  We propose Trajectory Matching Policy Balance Optimization (TMPO), which replaces reward with human-level reward distribution matching.<n>TMPO improves generative diversity over state-of-the-art methods by 9.1%, and achieves competitive performance in all downstream and efficiency metrics.<n>To further reduce multi-trajectory training time on large-scale flow- prefixes, TMPO incorporates Dynamic Tree Sampling models, where trajectories share denoising and branch at dynamically scheduled steps.
  arXiv  Detail & Related papers  (2026-05-09T04:41:02Z)
• HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models [23.070399327132737]
  We propose a novel framework that trains a hypernetwork for efficient and effective test-time alignment.<n>Instead of modifying latent states, HyperAlign dynamically generates low-rank adaptation weights to modulate the diffusion model's generation operators.<n>It significantly outperforms existing fine-tuning and test-time scaling baselines in enhancing semantic consistency and visual appeal.
  arXiv  Detail & Related papers  (2026-01-22T13:49:47Z)
• SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimization [62.958457694151384]
  We introduce preference optimization into precipitation nowcasting for the first time, motivated by the success of reinforcement learning from human feedback in large language models.<n>In the first stage, the framework focuses on reducing FAR, training the model to effectively suppress false alarms.
  arXiv  Detail & Related papers  (2025-10-22T16:11:22Z)
• Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectories [58.988535279557546]
  We introduce textbf sycophancy Mitigation through Adaptive Reasoning Trajectories.<n>We show that SMART significantly reduces sycophantic behavior while preserving strong performance on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2025-09-20T17:09:14Z)
• Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design [87.58981407469977]
  We propose a novel framework for inference-time reward optimization with diffusion models inspired by evolutionary algorithms.<n>Our approach employs an iterative refinement process consisting of two steps in each iteration: noising and reward-guided denoising.
  arXiv  Detail & Related papers  (2025-02-20T17:48:45Z)
• Inference-Time Alignment of Diffusion Models with Direct Noise Optimization [45.77751895345154]
  We propose a novel alignment approach, named Direct Noise Optimization (DNO), that optimize the injected noise during the sampling process of diffusion models. By design, DNO operates at inference-time, and thus is tuning-free and prompt-agnostic, with the alignment occurring in an online fashion during generation. We conduct extensive experiments on several important reward functions and demonstrate that the proposed DNO approach can achieve state-of-the-art reward scores within a reasonable time budget for generation.
  arXiv  Detail & Related papers  (2024-05-29T08:39:39Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.