Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives

• URL: http://arxiv.org/abs/2607.26735v1
• Date: Wed, 29 Jul 2026 10:29:07 GMT
• Title: Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives
• Abstract summary: We propose Dualin, a two-stage method that recovers the semantic prompt and latent noise of the target image.<n>In the first stage, we integrate vision-language model, CLIP and large language model to invert a faithful, human-interpretable hard prompt.<n>In the second stage, unconditional DDIM inversion reconstructs the exact latent noise of the target image, guaranteeing the consistency at the structural information level.
• Score: 24.71740058679873
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt inversion, as a typical reverse engineering technique, enables text-to-image (T2I) diffusion models to generate the desired target images without extensive prompt engineering. However, existing prompt inversion methods suffer from significant limitations: (1) gradient-based methods are unstable and uninterpretable, often resulting in generated images with severe artifacts; (2) gradient-free methods yield human-readable prompts but still fail to preserve visual fidelity due to the lack of fine-grained detail alignment. We contend that the limitations stem from treating prompt inversion as a sufficient condition for reverse engineering, ignoring the critical role of the latent noise that encodes structural information. Consequently, we propose Dualin (Dual inversion), a two-stage method that jointly recovers both the semantic prompt and latent noise of the target image. In the first stage, we integrate vision-language model, CLIP and large language model to invert a faithful, human-interpretable hard prompt. In the second stage, unconditional DDIM inversion reconstructs the exact latent noise of the target image, guaranteeing the consistency at the structural information level. Theoretically, we prove that the inverted noise enables flexible image editing without re-optimization. Extensive experiments on diverse datasets demonstrate that Dualin simultaneously generates high-quality inverted prompts and achieves state-of-the-art image fidelity. Additionally, Dualin can establish a robust foundation for the precise and controllable image editing.

Related papers

• Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers [55.15722080205737]
  Edit2Perceive is a unified diffusion framework that adapts editing models for depth, normal, and matting.<n>Our single-step deterministic inference yields up to faster runtime while training on relatively small datasets.
  arXiv  Detail & Related papers  (2025-11-24T01:13:51Z)
• EditInfinity: Image Editing with Binary-Quantized Generative Models [64.05135380710749]
  We investigate the parameter-efficient adaptation of binary-quantized generative models for image editing.<n>Specifically, we propose EditInfinity, which adapts emphInfinity, a binary-quantized generative model, for image editing.<n>We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation.
  arXiv  Detail & Related papers  (2025-10-23T05:06:24Z)
• Prompt-Guided Dual Latent Steering for Inversion Problems [16.58915166460579]
  Inverting corrupted images into the latent space of diffusion models is challenging.<n>Current methods, which encode an image into a single latent vector, struggle to balance structural fidelity with semantic accuracy.<n>We introduce Prompt-Guided Dual Latent Steering (PDLS), a novel framework built upon Rectified Flow models for their stable inversion paths.<n>PDLS decomposes the inversion process into two complementary streams: a structural path to preserve source integrity and a semantic path guided by a prompt.
  arXiv  Detail & Related papers  (2025-09-23T04:11:06Z)
• Single-Reference Text-to-Image Manipulation with Dual Contrastive Denoising Score [4.8677910801584385]
  Large-scale text-to-image generative models have shown remarkable ability to synthesize diverse and high-quality images.<n>We present Dual Contrastive Denoising Score, a framework that leverages the rich generative prior of text-to-image diffusion models.<n>Our method achieves both flexible content modification and structure preservation between input and output images, as well as zero-shot image-to-image translation.
  arXiv  Detail & Related papers  (2025-08-18T08:30:07Z)
• Dual Recursive Feedback on Generation and Appearance Latents for Pose-Robust Text-to-Image Diffusion [15.384896404310645]
  We propose a training-free Dual Recursive Feedback (DRF) system that properly reflects control conditions in controllable T2I models.<n>Our method produces high-quality, semantically coherent, and structurally consistent image generations.
  arXiv  Detail & Related papers  (2025-08-13T07:46:00Z)
• Accurate Latent Inversion for Generative Image Steganography via Rectified Flow [5.404219831398271]
  Steganography based on diffusion models has attracted increasing attention due to its ability to generate high-quality images and exhibit strong robustness.<n>We propose textbfRF-Stego, a novel generative image steganography method that enables accurate latent inversion and significantly improves extraction accuracy.<n> Experimental results show RF-Stego outperforms state-of-the-art methods in terms of extraction accuracy, image quality, robustness, security and generation efficiency.
  arXiv  Detail & Related papers  (2025-08-01T08:46:32Z)
• DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing [73.12011187146481]
  Inversion within Diffusion models aims to recover the latent noise representation for a real or generated image.<n>Most inversion approaches suffer from an intrinsic trade-off between reconstruction accuracy and editing flexibility.<n>We introduce Dual-Conditional Inversion (DCI), a novel framework that jointly conditions on the source prompt and reference image.
  arXiv  Detail & Related papers  (2025-06-03T07:46:44Z)
• NOFT: Test-Time Noise Finetune via Information Bottleneck for Highly Correlated Asset Creation [70.96827354717459]
  diffusion model has provided a strong tool for implementing text-to-image (T2I) and image-to-image (I2I) generation.<n>We propose a noise finetune NOFT module employed by Stable Diffusion to generate highly correlated and diverse images.
  arXiv  Detail & Related papers  (2025-05-18T05:09:47Z)
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.