Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences

• URL: http://arxiv.org/abs/2506.02095v1
• Date: Mon, 02 Jun 2025 17:42:58 GMT
• Title: Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences
• Authors: Hyojin Bahng, Caroline Chan, Fredo Durand, Phillip Isola,
• Abstract summary: We propose an alternative approach that leverages cycle consistency as a supervisory signal.<n>We map the text back to image space using a text-to-image model and compute the similarity between the original image and its reconstruction.<n>We use the cycle consistency score to rank candidates and construct a preference dataset of 866K comparison pairs.
• Score: 28.683767105094393
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Learning alignment between language and vision is a fundamental challenge, especially as multimodal data becomes increasingly detailed and complex. Existing methods often rely on collecting human or AI preferences, which can be costly and time-intensive. We propose an alternative approach that leverages cycle consistency as a supervisory signal. Given an image and generated text, we map the text back to image space using a text-to-image model and compute the similarity between the original image and its reconstruction. Analogously, for text-to-image generation, we measure the textual similarity between an input caption and its reconstruction through the cycle. We use the cycle consistency score to rank candidates and construct a preference dataset of 866K comparison pairs. The reward model trained on our dataset outperforms state-of-the-art alignment metrics on detailed captioning, with superior inference-time scalability when used as a verifier for Best-of-N sampling. Furthermore, performing DPO and Diffusion DPO using our dataset enhances performance across a wide range of vision-language tasks and text-to-image generation. Our dataset, model, and code are at https://cyclereward.github.io

Related papers

• Debiasing Vison-Language Models with Text-Only Training [15.069736314663352]
  We propose a Text-Only Debiasing framework called TOD, leveraging a text-as-image training paradigm to mitigate visual biases. To address the limitations, we propose a Text-Only Debiasing framework called TOD, leveraging a text-as-image training paradigm to mitigate visual biases.
  arXiv  Detail & Related papers  (2024-10-12T04:34:46Z)
• Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency [47.3163261953469]
  Current vision-language generative models rely on expansive corpora of paired image-text data to attain optimal performance and generalization capabilities. We introduce ITIT: an innovative training paradigm grounded in the concept of cycle consistency which allows vision-language training on unpaired image and text data. ITIT is comprised of a joint image-text encoder with disjoint image and text decoders that enable bidirectional image-to-text and text-to-image generation in a single framework.
  arXiv  Detail & Related papers  (2023-10-05T17:55:19Z)
• Learning to Generate Semantic Layouts for Higher Text-Image Correspondence in Text-to-Image Synthesis [37.32270579534541]
  We propose a novel approach for enhancing text-image correspondence by leveraging available semantic layouts. Our approach achieves higher text-image correspondence compared to existing text-to-image generation approaches in the Multi-Modal CelebA-HQ and the Cityscapes dataset.
  arXiv  Detail & Related papers  (2023-08-16T05:59:33Z)
• COSA: Concatenated Sample Pretrained Vision-Language Foundation Model [78.32081709802873]
  Most vision-language foundation models employ image-text datasets for pretraining. We propose COSA, a COncatenated SAmple pretrained vision-language foundation model. We achieve this by sequentially concatenating multiple image-text pairs as inputs for pretraining. This transformation effectively converts existing image-text corpora into a pseudo long-form video-paragraph corpus.
  arXiv  Detail & Related papers  (2023-06-15T12:29:42Z)
• Text-based Person Search without Parallel Image-Text Data [52.63433741872629]
  Text-based person search (TBPS) aims to retrieve the images of the target person from a large image gallery based on a given natural language description. Existing methods are dominated by training models with parallel image-text pairs, which are very costly to collect. In this paper, we make the first attempt to explore TBPS without parallel image-text data.
  arXiv  Detail & Related papers  (2023-05-22T12:13:08Z)
• Generative Negative Text Replay for Continual Vision-Language Pretraining [95.2784858069843]
  Vision-language pre-training has attracted increasing attention recently. Massive data are usually collected in a streaming fashion. We propose a multi-modal knowledge distillation between images and texts to align the instance-wise prediction between old and new models.
  arXiv  Detail & Related papers  (2022-10-31T13:42:21Z)
• CRIS: CLIP-Driven Referring Image Segmentation [71.56466057776086]
  We propose an end-to-end CLIP-Driven Referring Image framework (CRIS) CRIS resorts to vision-language decoding and contrastive learning for achieving the text-to-pixel alignment. Our proposed framework significantly outperforms the state-of-the-art performance without any post-processing.
  arXiv  Detail & Related papers  (2021-11-30T07:29:08Z)
• Cycle-Consistent Inverse GAN for Text-to-Image Synthesis [101.97397967958722]
  We propose a novel unified framework of Cycle-consistent Inverse GAN for both text-to-image generation and text-guided image manipulation tasks. We learn a GAN inversion model to convert the images back to the GAN latent space and obtain the inverted latent codes for each image. In the text-guided optimization module, we generate images with the desired semantic attributes by optimizing the inverted latent codes.
  arXiv  Detail & Related papers  (2021-08-03T08:38:16Z)
• Using Text to Teach Image Retrieval [47.72498265721957]
  We build on the concept of image manifold to represent the feature space of images, learned via neural networks, as a graph. We augment the manifold samples with geometrically aligned text, thereby using a plethora of sentences to teach us about images. The experimental results show that the joint embedding manifold is a robust representation, allowing it to be a better basis to perform image retrieval.
  arXiv  Detail & Related papers  (2020-11-19T16:09:14Z)
• Deep Multimodal Image-Text Embeddings for Automatic Cross-Media Retrieval [0.0]
  We introduce an end-to-end deep multimodal convolutional-recurrent network for learning both vision and language representations simultaneously. The model learns which pairs are a match (positive) and which ones are a mismatch (negative) using a hinge-based triplet ranking.
  arXiv  Detail & Related papers  (2020-02-23T23:58:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.