Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models

• URL: http://arxiv.org/abs/2605.30038v1
• Date: Thu, 28 May 2026 14:57:01 GMT
• Title: Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models
• Abstract summary: Diffusion models generate highly realistic images but often struggle with precise text-image alignment.<n>We propose a lightweight, reward-free post-training method that refines soft tokens by integrating contrastive alignment guidance.<n>Our method achieves over 35% improvement in counting accuracy on the GenEval benchmark.
• Score: 46.98979357654374
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion models generate highly realistic images but often struggle with precise text-image alignment. While recent post-training methods improve alignment using external rewards or human preference signals, their performance heavily depends on reward quality and does not directly address alignment within the diffusion process itself. Recent reward-free approaches such as SoftREPA demonstrate that optimizing soft text tokens via contrastive learning can effectively improve text-image representation alignment, outperforming standard parameter-efficient fine-tuning baselines. However, the contrastive formulation can excessively penalize negative pairs, which manifests as characteristic failure cases such as over-counting and repetition. To address this issue, we propose a lightweight, reward-free post-training method that refines soft tokens by integrating contrastive alignment guidance directly into the score-matching objective of diffusion models. By assigning alignment directions at the score level, our approach mitigates these limitations and yields more coherent and semantically faithful generations. Experiments show that our method matches SoftREPA while substantially improving its failure cases, achieving over 35% improvement in counting accuracy on the GenEval benchmark. Our method is seamlessly applicable to existing diffusion backbones (SD1.5, SDXL, and SD3), and is complementary to existing RL-based diffusion post-training methods. Project page: https://jaayeon.github.io/AGSM

Related papers

• DebFilter: Eradicating Biases Stashed in Value [4.060731229044571]
  We propose DebFilter to mitigate social and semantic biases in text-to-image models.<n>We apply a bias-correction strategy that adjusts the value components within cross-attention.<n>Our results demonstrate that this method effectively reconfigures social biases in generated images.
  arXiv  Detail & Related papers  (2026-05-27T08:49:41Z)
• Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models [73.08789211016567]
  Diffusion LAIR is a reward-aware listwise preference optimization method for diffusion models.<n>Experiments show it outperforms strong preference optimization baselines on text-to-image generation, compositional generation, and image editing benchmarks.
  arXiv  Detail & Related papers  (2026-05-26T03:09:24Z)
• FAIL: Flow Matching Adversarial Imitation Learning for Image Generation [52.643484089126844]
  Post-training of flow matching models-aligning the output distribution with a high-quality target-is mathematically equivalent to Imitation learning.<n>We propose Flow Matching Adrial Learning (FAIL), which minimizes policy-expert divergence through adversarial training without explicit rewards or pairwise comparisons.
  arXiv  Detail & Related papers  (2026-02-12T16:36:33Z)
• Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision [14.612317970237436]
  Diffusion models have achieved impressive results in generative tasks such as text-to-image synthesis.<n>They often struggle to fully align outputs with nuanced user intent and maintain consistent aesthetic quality.<n>Existing preference-based training methods help address these issues but rely on costly and potentially noisy human datasets.
  arXiv  Detail & Related papers  (2025-12-29T12:46:07Z)
• LightFair: Towards an Efficient Alternative for Fair T2I Diffusion via Debiasing Pre-trained Text Encoders [84.39846443122853]
  This paper explores a novel lightweight approach to achieve fair text-to-image diffusion models (T2I DMs) by addressing the adverse effects of the text encoder.<n>Since T2I DMs consist of multiple components, with the text encoder being the most fine-tunable and front-end module, this paper focuses on mitigating bias by fine-tuning text embeddings.<n>Our method achieves SOTA debiasing at just $1/4$ of the training burden, with virtually no increase in sampling burden.
  arXiv  Detail & Related papers  (2025-09-28T04:46:39Z)
• SAGA: Learning Signal-Aligned Distributions for Improved Text-to-Image Generation [9.212970624261272]
  State-of-the-art text-to-image models produce visually impressive results but often struggle with precise alignment to text prompts.<n>We propose a novel approach that learns a high-success-rate distribution conditioned on a target prompt.<n>Our method explicitly models the signal component during the denoising process, offering fine-grained control that mitigates over-optimization.
  arXiv  Detail & Related papers  (2025-08-19T14:31:15Z)
• Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards [52.90573877727541]
  reinforcement learning (RL) has been considered for diffusion model fine-tuning.<n>RL's effectiveness is limited by the challenge of sparse reward.<n>$textB2text-DiffuRL$ is compatible with existing optimization algorithms.
  arXiv  Detail & Related papers  (2025-03-14T09:45:19Z)
• Aligning Text to Image in Diffusion Models is Easier Than You Think [47.623236425067326]
  SoftREPA is a lightweight contrastive fine-tuning strategy that leverages soft text tokens for representation alignment.<n>Our method explicitly increases the mutual information between text and image representations, leading to enhanced semantic consistency.
  arXiv  Detail & Related papers  (2025-03-11T10:14:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.