There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation

• URL: http://arxiv.org/abs/2608.27885v1
• Date: Fri, 28 Aug 2026 03:43:53 GMT
• Title: There and Back Again: Bidirectional Diffusion Bridges for Multimodality Translation
• Abstract summary: Multimodality translation (e.g., text-to-image) is a core generative AI task.<n>We propose BIT: Bi Image-Text Diffusion Bridges.<n>In contrast to previous approaches, BIT starts directly from text and interpolates into images.
• Score: 39.33656400753428
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodality translation (e.g., text-to-image) is a core generative AI task. However, existing approaches (1) follow generative paths that do not directly represent the source modality, limiting the flexibility of some sampling algorithms; and (2) are unidirectional, preventing inversion (e.g., image-to-text). We propose BIT: Bidirectional Image-Text Diffusion Bridges. In contrast to previous approaches, BIT starts directly from text and interpolates into images, providing (1) a source-aware generative path that enables diverse and flexible sampling algorithms; and (2) an endpoint-conditioned process that can be traversed from image to text, providing a unified, bidirectional generative framework. BIT is derived through stochastic calculus, yielding SDE forms amenable to simulation and tractable loss functions that scale to high dimensions. Our experiments show that BIT is competitive with denoising-diffusion and deterministic-flow baselines, and outperforms them on several vision--language and natural-science evaluations.

Related papers

• Unified Text-Image Generation with Weakness-Targeted Post-Training [57.956648078400775]
  Unified multimodal generation architectures that jointly produce text and images have recently emerged as a promising direction for text-to-image (T2I) synthesis.<n>This work explores post-training to achieve fully unified text-image generation, where models autonomously transition from textual reasoning to visual synthesis.
  arXiv  Detail & Related papers  (2026-01-07T19:19:44Z)
• CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation [9.628074306577851]
  Current conditional autoregressive image generation methods have shown promising results, yet their potential remains largely unexplored in the practical unsupervised image translation domain.<n>A critical limitation stems from the discrete quantization inherent in traditional Vector Quantization-based frameworks.<n>We propose Softmax Relaxed Quantization, a novel approach that reformulates codebook selection as a continuous probability mixing process.
  arXiv  Detail & Related papers  (2025-06-29T17:43:04Z)
• Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation [55.45188329646137]
  Implicit Bridge Consistency Distillation (IBCD) enables single-step bidirectional unpaired translation without using adversarial loss.<n>IBCD achieves state-of-the-art performance on benchmark datasets in a single generation step.
  arXiv  Detail & Related papers  (2025-03-19T09:48:04Z)
• Origin Identification for Text-Guided Image-to-Image Diffusion Models [39.234894330025114]
  We propose origin IDentification for text-guided Image-to-image Diffusion models (ID$2$)<n>A straightforward solution to ID$2$ involves training a specialized deep embedding model to extract and compare features from both query and reference images.<n>To solve this challenge of the proposed ID$2$ task, we contribute the first dataset and a theoretically guaranteed method.
  arXiv  Detail & Related papers  (2025-01-04T20:34:53Z)
• Latent Schrodinger Bridge: Prompting Latent Diffusion for Fast Unpaired Image-to-Image Translation [58.19676004192321]
  Diffusion models (DMs), which enable both image generation from noise and inversion from data, have inspired powerful unpaired image-to-image (I2I) translation algorithms. We tackle this problem with Schrodinger Bridges (SBs), which are differential equations (SDEs) between distributions with minimal transport cost. Inspired by this observation, we propose Latent Schrodinger Bridges (LSBs) that approximate the SB ODE via pre-trained Stable Diffusion. We demonstrate that our algorithm successfully conduct competitive I2I translation in unsupervised setting with only a fraction of cost required by previous DM-
  arXiv  Detail & Related papers  (2024-11-22T11:24:14Z)
• Bidirectional Consistency Models [1.486435467709869]
  Diffusion models (DMs) are capable of generating remarkably high-quality samples by iteratively denoising a random vector.<n>DMs can also invert an input image to noise by moving backward along the probability flow ordinary differential equation (PF ODE)
  arXiv  Detail & Related papers  (2024-03-26T18:40:36Z)
• Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion [35.21106030549071]
  Diffusion Probabilistic Models (DPMs) are dominant force in text-to-image generation tasks. We propose an alternative view of state-of-the-art DPMs as a way of inverting advanced Vision-Language Models (VLMs) By directly optimizing images with the supervision of discriminative VLMs, the proposed method can potentially achieve a better text-image alignment.
  arXiv  Detail & Related papers  (2024-02-26T05:08:40Z)
• DiffDis: Empowering Generative Diffusion Model with Cross-Modal Discrimination Capability [75.9781362556431]
  We propose DiffDis to unify the cross-modal generative and discriminative pretraining into one single framework under the diffusion process. We show that DiffDis outperforms single-task models on both the image generation and the image-text discriminative tasks.
  arXiv  Detail & Related papers  (2023-08-18T05:03:48Z)
• How Image Generation Helps Visible-to-Infrared Person Re-Identification? [15.951145523749735]
  Flow2Flow is a framework that can jointly achieve training sample expansion and cross-modality image generation for V2I person ReID. For the purpose of identity alignment and modality alignment of generated images, we develop adversarial training strategies to train Flow2Flow. Experimental results on SYSU-MM01 and RegDB demonstrate that both training sample expansion and cross-modality image generation can significantly improve V2I ReID accuracy.
  arXiv  Detail & Related papers  (2022-10-04T13:09:29Z)
• BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models [50.39417112077254]
  A novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation.
  arXiv  Detail & Related papers  (2022-05-16T13:47:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.