Composition and Alignment of Diffusion Models using Constrained Learning

• URL: http://arxiv.org/abs/2508.19104v1
• Date: Tue, 26 Aug 2025 15:06:30 GMT
• Title: Composition and Alignment of Diffusion Models using Constrained Learning
• Authors: Shervin Khalafi, Ignacio Hounie, Dongsheng Ding, Alejandro Ribeiro,
• Abstract summary: Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions.<n>Two commonly used methods are: (i) alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs.<n>We propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models.
• Score: 79.36736636241564
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions. To improve the quality of generated samples and their compliance with user requirements, two commonly used methods are: (i) Alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) Composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs. However, trade-offs often arise when optimizing for multiple rewards or combining multiple models, as they can often represent competing properties. Existing methods cannot guarantee that the resulting model faithfully generates samples with all the desired properties. To address this gap, we propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models. We provide a theoretical characterization of the solutions to the constrained alignment and composition problems and develop a Lagrangian-based primal-dual training algorithm to approximate these solutions. Empirically, we demonstrate the effectiveness and merits of our proposed approach in image generation, applying it to alignment and composition, and show that our aligned or composed model satisfies constraints effectively, and improves on the equally-weighted approach. Our implementation can be found at https://github.com/shervinkhalafi/constrained_comp_align.

Related papers

• Alignment of Diffusion Model and Flow Matching for Text-to-Image Generation [39.484148941369234]
  Diffusion models and flow matching have demonstrated remarkable success in text-to-image generation.<n>We propose a novel alignment framework by leveraging the underlying nature of the alignment problem.<n>We achieve comparable performance to finetuning-based models with one-step generation with at least a 60% reduction in computational cost.
  arXiv  Detail & Related papers  (2026-01-31T00:06:55Z)
• Distribution Matching via Generalized Consistency Models [4.3343955642269805]
  We propose a novel approach for distribution matching inspired by the consistency models employed in Continuous Normalizing Flow (CNF)<n>Our model inherits the advantages of CNF models, such as having a straight forward norm minimization objective, while remaining adaptable to different constraints similar to GANs.
  arXiv  Detail & Related papers  (2025-08-17T03:37:57Z)
• Bridging the inference gap in Mutimodal Variational Autoencoders [6.246098300155483]
  Multimodal Variational Autoencoders offer versatile and scalable methods for generating unobserved modalities from observed ones.<n>Recent models using mixturesof-experts aggregation suffer from theoretically grounded limitations that restrict their generation quality on complex datasets.<n>We propose a novel interpretable model able to learn both joint and conditional distributions without introducing mixture aggregation.
  arXiv  Detail & Related papers  (2025-02-06T10:43:55Z)
• Variational Search Distributions [16.609027794680213]
  We develop VSD, a method for conditioning a generative model of discrete, variational designs on a rare desired class.<n>We empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems.
  arXiv  Detail & Related papers  (2024-09-10T01:33:31Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion [53.90516061351706]
  We present InterHandGen, a novel framework that learns the generative prior of two-hand interaction. For sampling, we combine anti-penetration and synthesis-free guidance to enable plausible generation. Our method significantly outperforms baseline generative models in terms of plausibility and diversity.
  arXiv  Detail & Related papers  (2024-03-26T06:35:55Z)
• Physics-Informed Diffusion Models [0.0]
  We present a framework that unifies generative modeling and partial differential equation fulfillment.<n>Our approach reduces the residual error by up to two orders of magnitude compared to previous work in a fluid flow case study.
  arXiv  Detail & Related papers  (2024-03-21T13:52:55Z)
• Diffusion Model for Data-Driven Black-Box Optimization [54.25693582870226]
  We focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization. We study two practical types of labels: 1) noisy measurements of a real-valued reward function and 2) human preference based on pairwise comparisons. Our proposed method reformulates the design optimization problem into a conditional sampling problem, which allows us to leverage the power of diffusion models.
  arXiv  Detail & Related papers  (2024-03-20T00:41:12Z)
• Lp-Norm Constrained One-Class Classifier Combination [18.27510863075184]
  We consider the one-class classification problem by modelling the sparsity/uniformity of the ensemble. We present an effective approach to solve formulated convex constrained problem efficiently.
  arXiv  Detail & Related papers  (2023-12-25T16:32:34Z)
• Control as Hybrid Inference [62.997667081978825]
  We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference. We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines.
  arXiv  Detail & Related papers  (2020-07-11T19:44:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.