Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation

• URL: http://arxiv.org/abs/2604.18376v1
• Date: Mon, 20 Apr 2026 15:03:01 GMT
• Title: Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation
• Abstract summary: This paper proposes a semantic compensation framework (MVR) driven by Large Language Models (LLMs)<n>It enhances cross-modal representation consistency through multi-view semantic reformulation and feature compensation.<n> Experiments demonstrate that our method can improve the accuracy of the original model well without training and performs SOTA on three text-to-image person retrieval datasets.
• Score: 11.925230809051143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In text-to-image person retrieval tasks, the diversity of natural language expressions and the implicitness of visual semantics often lead to the problem of Expression Drift, where semantically equivalent texts exhibit significant feature discrepancies in the embedding space due to phrasing variations, thereby degrading the robustness of image-text alignment. This paper proposes a semantic compensation framework (MVR) driven by Large Language Models (LLMs), which enhances cross-modal representation consistency through multi-view semantic reformulation and feature compensation. The core methodology comprises three components: Multi-View Reformulation (MVR): A dual-branch prompting strategy combines key feature guidance (extracting visually critical components via feature similarity) and diversity-aware rewriting to generate semantically equivalent yet distributionally diverse textual variants; Textual Feature Robustness Enhancement: A training-free latent space compensation mechanism suppresses noise interference through multi-view feature mean-pooling and residual connections, effectively capturing "Semantic Echoes"; Visual Semantic Compensation: VLM generates multi-perspective image descriptions, which are further enhanced through shared text reformulation to address visual semantic gaps. Experiments demonstrate that our method can improve the accuracy of the original model well without training and performs SOTA on three text-to-image person retrieval datasets.

Related papers

• ViQ: Text-Aligned Visual Quantized Representations at Any Resolution [91.46185855575789]
  We present ViQ, a Visual Quantized Representations framework, which is designed to balance semantics and details in discrete representations.<n>Our approach structures quantization learning into two stages: text-aligned pre-training and feature discretization.<n>Experiments on multimodal tasks demonstrate that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders.
  arXiv  Detail & Related papers  (2026-06-25T17:29:27Z)
• Semantic Browsing: Controllable Diversity for Image Generation [51.503726779537]
  We introduce a method for controlled diversity that enables Semantic Browsing.<n>We exploit the fact that recent text-to-image models are trained on elaborated captions.<n>This enables a paradigm shift: instead of relying on variation within the text-to-image model, we induce diversity directly at the text level.
  arXiv  Detail & Related papers  (2026-06-22T17:59:17Z)
• Multimodal Large Language Models for Multi-Subject In-Context Image Generation [56.20395856287325]
  Music is the first MLLM specifically designed for textbfMUlti-textbfSubject textbfIn-textbfContext image generation.<n>We enhance the model's understanding of multi-subject semantic relationships through a vision chain-of-thought mechanism.<n>By incorporating complex subject images during training, we improve the model's capacity for chained reasoning.
  arXiv  Detail & Related papers  (2026-04-08T15:37:42Z)
• OS-HGAdapter: Open Semantic Hypergraph Adapter for Large Language Models Assisted Entropy-Enhanced Image-Text Alignment [8.625923727928752]
  Text-image alignment is a foundational challenge in multimedia content understanding.<n>We propose to use the open semantic knowledge of Large Language Model (LLM) to fill for the entropy gap.<n> Comprehensive evaluations on the Flickr30K and MS-COCO benchmarks validate the superiority of our Open Semantic Hypergraph Adapter (OS-HGAdapter)
  arXiv  Detail & Related papers  (2025-10-15T04:09:00Z)
• Context-Adaptive Multi-Prompt Embedding with Large Language Models for Vision-Language Alignment [33.152772648399846]
  We propose a novel approach to enrich semantic representations in vision-language contrastive learning.<n>We leverage a pretrained LLM as the text encoder within the CLIP framework, processing all prompts jointly in a single forward pass.<n>The resulting prompt embeddings are combined into a unified text representation, enabling semantically richer alignment with visual features.
  arXiv  Detail & Related papers  (2025-08-03T20:48:43Z)
• MegaSR: Mining Customized Semantics and Expressive Guidance for Image Super-Resolution [76.30559905769859]
  MegaSR mines customized block-wise semantics and expressive guidance for diffusion-based ISR.<n>We experimentally identify HED edge maps, depth maps, and segmentation maps as the most expressive guidance.<n>Extensive experiments demonstrate the superiority of MegaSR in terms of semantic richness and structural consistency.
  arXiv  Detail & Related papers  (2025-03-11T07:00:20Z)
• Data-Efficient Generalization for Zero-shot Composed Image Retrieval [67.46975191141928]
  ZS-CIR aims to retrieve the target image based on a reference image and a text description without requiring in-distribution triplets for training.<n>One prevalent approach follows the vision-language pretraining paradigm that employs a mapping network to transfer the image embedding to a pseudo-word token in the text embedding space.<n>We propose a Data-efficient Generalization (DeG) framework, including two novel designs, namely, Textual Supplement (TS) module and Semantic-Set (S-Set)
  arXiv  Detail & Related papers  (2025-03-07T07:49:31Z)
• Embedding and Enriching Explicit Semantics for Visible-Infrared Person Re-Identification [31.011118085494942]
  Visible-infrared person re-identification (VIReID) retrieves pedestrian images with the same identity across different modalities.<n>Existing methods learn visual content solely from images, lacking the capability to sense high-level semantics.<n>We propose an Embedding and Enriching Explicit Semantics framework to learn semantically rich cross-modality pedestrian representations.
  arXiv  Detail & Related papers  (2024-12-11T14:27:30Z)
• Dynamic Visual Semantic Sub-Embeddings and Fast Re-Ranking [0.5242869847419834]
  We propose a Dynamic Visual Semantic Sub-Embeddings framework (DVSE) to reduce the information entropy. To encourage the generated candidate embeddings to capture various semantic variations, we construct a mixed distribution. We compare the performance with existing set-based method using four image feature encoders and two text feature encoders on three benchmark datasets.
  arXiv  Detail & Related papers  (2023-09-15T04:39:11Z)
• Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval [12.057465578064345]
  The goal of Text-to-Image Person Retrieval (TIPR) is to retrieve specific person images according to the given textual descriptions.<n>We propose a novel TIPR framework to build fine-grained interactions and alignment between person images and the corresponding texts.
  arXiv  Detail & Related papers  (2023-07-18T08:23:46Z)
• RealignDiff: Boosting Text-to-Image Diffusion Model with Coarse-to-fine Semantic Re-alignment [112.45442468794658]
  We propose a two-stage coarse-to-fine semantic re-alignment method, named RealignDiff. In the coarse semantic re-alignment phase, a novel caption reward is proposed to evaluate the semantic discrepancy between the generated image caption and the given text prompt. The fine semantic re-alignment stage employs a local dense caption generation module and a re-weighting attention modulation module to refine the previously generated images from a local semantic view.
  arXiv  Detail & Related papers  (2023-05-31T06:59:21Z)
• Learning to Model Multimodal Semantic Alignment for Story Visualization [58.16484259508973]
  Story visualization aims to generate a sequence of images to narrate each sentence in a multi-sentence story. Current works face the problem of semantic misalignment because of their fixed architecture and diversity of input modalities. We explore the semantic alignment between text and image representations by learning to match their semantic levels in the GAN-based generative model.
  arXiv  Detail & Related papers  (2022-11-14T11:41:44Z)
• CRIS: CLIP-Driven Referring Image Segmentation [71.56466057776086]
  We propose an end-to-end CLIP-Driven Referring Image framework (CRIS) CRIS resorts to vision-language decoding and contrastive learning for achieving the text-to-pixel alignment. Our proposed framework significantly outperforms the state-of-the-art performance without any post-processing.
  arXiv  Detail & Related papers  (2021-11-30T07:29:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.