Diffusion Controller: Framework, Algorithms and Parameterization

• URL: http://arxiv.org/abs/2603.06981v1
• Date: Sat, 07 Mar 2026 01:49:59 GMT
• Title: Diffusion Controller: Framework, Algorithms and Parameterization
• Abstract summary: We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
• Score: 54.82539154511621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Controllable diffusion generation often relies on various heuristics that are seemingly disconnected without a unified understanding. We bridge this gap with Diffusion Controller (DiffCon), a unified control-theoretic view that casts reverse diffusion sampling as state-only stochastic control within (generalized) linearly-solvable Markov Decision Processes (LS-MDPs). Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost. From the resulting optimality conditions, we derive practical reinforcement learning methods for diffusion fine-tuning: (i) f-divergence-regularized policy-gradient updates, including a PPO-style rule, and (ii) a regularizer-determined reward-weighted regression objective with a minimizer-preservation guarantee under the Kullback-Leibler (KL) divergence. The LS-MDP framework further implies a principled model form: the optimal score decomposes into a fixed pretrained baseline plus a lightweight control correction, motivating a side-network parameterization conditioned on exposed intermediate denoising outputs, enabling effective gray-box adaptation with a frozen backbone. Experiments on Stable Diffusion v1.4 across supervised and reward-driven finetuning show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs versus gray-box baselines and even the parameter-efficient white-box adapter LoRA.

Related papers

• Trajectory-Regularized Stochastic Optimal Control via KL Divergence [8.729531978655737]
  We introduce trajectory-regularized optimal control (TRSOC), which augments standard optimal control (SOC)<n>We derive the corresponding Hamilton-Jacobi-Bellman equation and characterize the optimal policy.<n>Experiments show that the regularization parameter induces a trade-off between performance-driven and reference-preserving behavior.
  arXiv  Detail & Related papers  (2026-07-24T11:15:27Z)
• From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation [108.02425846269234]
  controllable image generation has made significant strides by incorporating visual reference conditions.<n>Existing methods predominantly operate as open-loop systems.<n>We propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem.
  arXiv  Detail & Related papers  (2026-07-06T05:36:30Z)
• Improved techniques for fine-tuning flow models via adjoint matching: a deterministic control pipeline [7.967038299436285]
  We propose a deterministic adjoint matching framework that formulates human preference alignment for flow-based generative models.<n>Building on this perspective, we introduce a truncated adjoint scheme that focuses on the terminal portion of the trajectory.<n>We further generalize the framework beyond standard KL-based regularization, allowing more flexible trade-offs between alignment strength and distributional preservation.
  arXiv  Detail & Related papers  (2026-05-07T17:12:47Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Discrete Flow Matching Policy Optimization [12.864514553126186]
  We introduce a unified framework for Reinforcement Learning (RL) fine-tuning Discrete Flow Matching (DFM) models.<n>Our key idea is to view the DFM sampling procedure as a multi-step Markov Decision Process.<n>DoMinO achieves stronger predicted enhancer activity and better sequence naturalness than the previous best reward-driven baselines.
  arXiv  Detail & Related papers  (2026-04-07T21:49:29Z)
• Diffusion Reinforcement Learning via Centered Reward Distillation [35.979608265594685]
  We present textbf Reward Distillation (CRD), a diffusion RL framework derived from KL-regularized reward models built on forward-process fine-tuning.<n>To enable reliable text-to-image fine-tuning, we introduce techniques that explicitly control distribution drift.<n>Experiments on text-to-image post-training with textttGenEval and textttOCR rewards show that competitive SOTA reward optimization results with fast convergence and reduced reward hacking preference.
  arXiv  Detail & Related papers  (2026-03-14T21:29:33Z)
• CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance [31.552164852288325]
  We introduce Sliding Mode Control CFG (SMC-CFG), which enforces the generative flow toward a rapidly convergent sliding manifold.<n>SMC-CFG outperforms standard CFG in semantic alignment and robustness across a wide range of guidance scales.
  arXiv  Detail & Related papers  (2026-03-03T18:59:48Z)
• Causal Autoregressive Diffusion Language Model [70.7353007255797]
  CARD reformulates the diffusion process within a strictly causal attention mask, enabling dense, per-token supervision in a single forward pass.<n>Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation.
  arXiv  Detail & Related papers  (2026-01-29T17:38:29Z)
• Goal inference with Rao-Blackwellized Particle Filters [5.633221187382381]
  Inferring the eventual goal of a mobile agent from noisy observations of its trajectory is a fundamental estimation problem.<n>We study such intent inference using a variant of a Rao-Blackwellized Particle Filter (RBPF)<n>We quantify how well the adversary can recover the agent's intent using information-theoretic leakage metrics.
  arXiv  Detail & Related papers  (2025-12-10T02:48:55Z)
• DPAC: Distribution-Preserving Adversarial Control for Diffusion Sampling [0.7866885337535715]
  Adversarially guided diffusion sampling often achieves the target class, but sample quality degrades as deviations between the adversarially controlled and nominal trajectories accumulate.<n>We formalize this degradation as a path-space Kullback-Leibler divergence(path-KL) between controlled and nominal (uncontrolled) diffusion processes.<n>We show that minimizing this path-KL simultaneously tightens upper bounds on both the Wasserstein distance and Fréchet Inception Distance (FID), revealing a connection between adversarial control energy and perceptual fidelity.
  arXiv  Detail & Related papers  (2025-12-01T00:15:05Z)
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning [56.47948583452555]
  We introduce the Stepwise Flow Policy (SWFP) framework, founded on the key insight that discretizing the flow matching inference process via a fixed-step Euler scheme aligns it with the variational Jordan-Kinderlehrer-Otto principle from optimal transport.<n>SWFP decomposes the global flow into a sequence of small, incremental transformations between proximate distributions.<n>This decomposition yields an efficient algorithm that fine-tunes pre-trained flows via a cascade of small flow blocks, offering significant advantages.
  arXiv  Detail & Related papers  (2025-10-17T07:43:51Z)
• Optimal Control Meets Flow Matching: A Principled Route to Multi-Subject Fidelity [35.95129874095729]
  Text-to-image (T2I) models excel on single-entity prompts but struggle with multi-subject descriptions.<n>We introduce the first theoretical framework with principled optimizable objective for steering sampling dynamics toward multi-subject fidelity.
  arXiv  Detail & Related papers  (2025-10-02T17:59:58Z)
• DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Models [22.823183347642132]
  We study inference-time scaling for diffusion models, where the goal is to adapt a pre-trained model to new target distributions without retraining.<n>We introduce DriftLite, a lightweight, training-free particle-based approach that steers the inference dynamics on the fly with provably optimal stability.
  arXiv  Detail & Related papers  (2025-09-25T22:21:59Z)
• DiffusionNFT: Online Diffusion Reinforcement with Forward Process [99.94852379720153]
  Diffusion Negative-aware FineTuning (DiffusionNFT) is a new online RL paradigm that optimize diffusion models directly on the forward process via flow matching.<n>DiffusionNFT is up to $25times$ more efficient than FlowGRPO in head-to-head comparisons, while being CFG-free.
  arXiv  Detail & Related papers  (2025-09-19T16:09:33Z)
• Adding Additional Control to One-Step Diffusion with Joint Distribution Matching [58.37264951734603]
  JDM is a novel approach that minimizes the reverse KL divergence between image-condition joint distributions.<n>By deriving a tractable upper bound, JDM decouples fidelity learning from condition learning.<n>This asymmetric distillation scheme enables our one-step student to handle controls unknown to the teacher model.
  arXiv  Detail & Related papers  (2025-03-09T15:06:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.