CTCal: Rethinking Text-to-Image Diffusion Models via Cross-Timestep Self-Calibration

• URL: http://arxiv.org/abs/2603.20741v1
• Date: Sat, 21 Mar 2026 10:00:35 GMT
• Title: CTCal: Rethinking Text-to-Image Diffusion Models via Cross-Timestep Self-Calibration
• Abstract summary: We introduce Cross-Timestep Self-Calibration ( CTCal) to achieve precise alignment between text prompts and generated images.<n> CTCal is model-agnostic and can be seamlessly integrated into existing text-to-image diffusion models.
• Score: 39.59945414053394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in text-to-image synthesis have been largely propelled by diffusion-based models, yet achieving precise alignment between text prompts and generated images remains a persistent challenge. We find that this difficulty arises primarily from the limitations of conventional diffusion loss, which provides only implicit supervision for modeling fine-grained text-image correspondence. In this paper, we introduce Cross-Timestep Self-Calibration (CTCal), founded on the supporting observation that establishing accurate text-image alignment within diffusion models becomes progressively more difficult as the timestep increases. CTCal leverages the reliable text-image alignment (i.e., cross-attention maps) formed at smaller timesteps with less noise to calibrate the representation learning at larger timesteps with more noise, thereby providing explicit supervision during training. We further propose a timestep-aware adaptive weighting to achieve a harmonious integration of CTCal and diffusion loss. CTCal is model-agnostic and can be seamlessly integrated into existing text-to-image diffusion models, encompassing both diffusion-based (e.g., SD 2.1) and flow-based approaches (e.g., SD 3). Extensive experiments on T2I-Compbench++ and GenEval benchmarks demonstrate the effectiveness and generalizability of the proposed CTCal. Our code is available at https://github.com/xiefan-guo/ctcal.

Related papers

• ImageRAGTurbo: Towards One-step Text-to-Image Generation with Retrieval-Augmented Diffusion Models [33.09645476860831]
  We propose ImageRAGTurbo, a novel approach to efficiently finetune few-step diffusion models via retrieval augmentation.<n>Given a text prompt, we retrieve relevant text-image pairs from a database and use them to condition the generation process.<n>Experiments show that our approach produces high-fidelity images without compromising latency compared to existing methods.
  arXiv  Detail & Related papers  (2026-02-13T05:59:57Z)
• Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers [79.94246924019984]
  Multimodal Diffusion Transformers (MM-DiTs) have achieved remarkable progress in text-driven visual generation.<n>We propose textbfTemperature-Adjusted Cross-modal Attention (TACA), a parameter-efficient method that dynamically rebalances multimodal interactions.<n>Our findings highlight the importance of balancing cross-modal attention in improving semantic fidelity in text-to-image diffusion models.
  arXiv  Detail & Related papers  (2025-06-09T17:54:04Z)
• Fast constrained sampling in pre-trained diffusion models [80.99262780028015]
  We propose an algorithm that enables fast, high-quality generation under arbitrary constraints.<n>Our approach produces results that rival or surpass the state-of-the-art training-free inference methods.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Improving Consistency in Diffusion Models for Image Super-Resolution [28.945663118445037]
  We observe two kinds of inconsistencies in diffusion-based methods.<n>We introduce ConsisSR to handle both semantic and training-inference consistencies.<n>Our method demonstrates state-of-the-art performance among existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-17T17:41:52Z)
• Faster Diffusion via Temporal Attention Decomposition [77.90640748930178]
  We explore the role of attention mechanism during inference in text-conditional diffusion models.<n>We develop a training-free method known as temporally gating the attention (TGATE)<n>TGATE efficiently generates images by caching and reusing attention outputs at scheduled time steps.
  arXiv  Detail & Related papers  (2024-04-03T13:44:41Z)
• Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models [23.786473791344395]
  Cross-attention layers in diffusion models tend to disproportionately focus on certain tokens during the generation process. We introduce attention regulation, an on-the-fly optimization approach at inference time to align attention maps with the input text prompt. Experiment results show that our method consistently outperforms other baselines.
  arXiv  Detail & Related papers  (2024-03-11T02:18:27Z)
• Debiasing Text-to-Image Diffusion Models [84.46750441518697]
  Learning-based Text-to-Image (TTI) models have revolutionized the way visual content is generated in various domains. Recent research has shown that nonnegligible social bias exists in current state-of-the-art TTI systems.
  arXiv  Detail & Related papers  (2024-02-22T14:33:23Z)
• Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing [49.800746112114375]
  We propose a novel post-training quantization method (Progressive and Relaxing) for text-to-image diffusion models. We are the first to achieve quantization for Stable Diffusion XL while maintaining the performance.
  arXiv  Detail & Related papers  (2023-11-10T09:10:09Z)
• CoreDiff: Contextual Error-Modulated Generalized Diffusion Model for Low-Dose CT Denoising and Generalization [41.64072751889151]
  Low-dose computed tomography (LDCT) images suffer from noise and artifacts due to photon starvation and electronic noise. This paper presents a novel COntextual eRror-modulated gEneralized Diffusion model for low-dose CT (LDCT) denoising, termed CoreDiff.
  arXiv  Detail & Related papers  (2023-04-04T14:13:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.