InterPartAbility: Text-Guided Part Matching for Interpretable Person Re-Identification

• URL: http://arxiv.org/abs/2604.27122v1
• Date: Wed, 29 Apr 2026 19:18:36 GMT
• Title: InterPartAbility: Text-Guided Part Matching for Interpretable Person Re-Identification
• Abstract summary: TI-ReID relies on natural-language text description to retrieve top matching individuals from a large gallery of images.<n>Existing interpretability approaches in TI-ReID rely solely on slot-attention to highlight attended regions.<n>This paper introduces InterPart, an interpretable TI-ReID method that performs explicit part-wise matching.
• Score: 15.419626926769595
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Text-to-image person re-identification (TI-ReID) relies on natural-language text description to retrieve top matching individuals from a large gallery of images. While recent large vision-language models (VLMs) achieve strong retrieval performance, their decisions remain largely uninterpretable. Existing interpretability approaches in TI-ReID rely solely on slot-attention to highlight attended regions, but fail to reliably bind visual regions to semantically meaningful concepts, limiting explanations to qualitative visualizations over a restricted vocabulary. This paper introduces InterPartAbility, an interpretable TI-ReID method that performs explicit part-wise matching and enables phrase-region grounding. A new open-vocabulary, lightweight supervision, patch-phrase interaction module (PPIM) is proposed to train a standard TI-ReID model with concept-level guidance. Concept-based part phrases provide evidence that encourages the model to attend to corresponding image regions. InterPartAbility further constrains CLIP ViT self-attention to produce spatially concentrated patch activations aligned with each part-level phrase, yielding grounded explanation maps. A quantitative interpretability protocol for TI-ReID is introduced by adapting perturbation-based evaluation metrics, including counterfactual region masking that measures retrieval degradation when top-ranked explanatory regions are removed. Empirical results\footnote{Our code is included in the supplementary materials and will be made public.} on challenging benchmarks like CUHK-PEDES and ICFG-PEDES show that InterPartAbility achieves state-of-the-art (SOTA) interpretability performance under these metrics, while sustaining competitive retrieval accuracy.

Related papers

• TCSA-UDA: Text-Driven Cross-Semantic Alignment for Unsupervised Domain Adaptation in Medical Image Segmentation [10.074858409073292]
  We propose a Text-driven Cross-Semantic Alignment framework to guide visual representation learning.<n>Our framework significantly reduces domain shift and consistently outperforms state-of-the-art UDA methods.
  arXiv  Detail & Related papers  (2025-11-08T00:56:52Z)
• LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance [54.683384204063934]
  Large multi-modal models (LMMs) struggle with inaccurate segmentation and hallucinated comprehension.<n>We propose LIRA, a framework that capitalizes on the complementary relationship between visual comprehension and segmentation.<n>LIRA achieves state-of-the-art performance in both segmentation and comprehension tasks.
  arXiv  Detail & Related papers  (2025-07-08T07:46:26Z)
• Semantic Token Reweighting for Interpretable and Controllable Text Embeddings in CLIP [46.53595526049201]
  A text encoder within Vision-Language Models (VLMs) like CLIP plays a crucial role in translating textual input into an embedding space shared with images. We propose a framework of Semantic Token Reweighting to build Interpretable text embeddings (SToRI) SToRI refines the text encoding process in CLIP by differentially weighting semantic elements based on contextual importance.
  arXiv  Detail & Related papers  (2024-10-11T02:42:13Z)
• Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation [100.81837601210597]
  We propose Concept Curation (CoCu) to bridge the gap between visual and textual semantics in pre-training data. CoCu achieves superb zero-shot transfer performance and greatly boosts language-supervised segmentation baseline by a large margin.
  arXiv  Detail & Related papers  (2023-09-24T00:05:39Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Fine-Grained Semantically Aligned Vision-Language Pre-Training [151.7372197904064]
  Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks. Existing methods mainly model the cross-modal alignment by the similarity of the global representations of images and texts. We introduce LO, a fine-grained semantically aLigned visiOn-langUage PrE-training framework, which learns fine-grained semantic alignment from the novel perspective of game-theoretic interactions.
  arXiv  Detail & Related papers  (2022-08-04T07:51:48Z)
• Vision-Language Pre-Training with Triple Contrastive Learning [45.80365827890119]
  We propose triple contrastive learning (TCL) for vision-language pre-training by leveraging both cross-modal and intra-modal self-supervision. Ours is the first work that takes into account local structure information for multi-modality representation learning.
  arXiv  Detail & Related papers  (2022-02-21T17:54:57Z)
• Referring Image Segmentation via Cross-Modal Progressive Comprehension [94.70482302324704]
  Referring image segmentation aims at segmenting the foreground masks of the entities that can well match the description given in the natural language expression. Previous approaches tackle this problem using implicit feature interaction and fusion between visual and linguistic modalities. We propose a Cross-Modal Progressive (CMPC) module and a Text-Guided Feature Exchange (TGFE) module to effectively address the challenging task.
  arXiv  Detail & Related papers  (2020-10-01T16:02:30Z)
• Towards Accurate Scene Text Recognition with Semantic Reasoning Networks [52.86058031919856]
  We propose a novel end-to-end trainable framework named semantic reasoning network (SRN) for accurate scene text recognition. GSRM is introduced to capture global semantic context through multi-way parallel transmission. Results on 7 public benchmarks, including regular text, irregular text and non-Latin long text, verify the effectiveness and robustness of the proposed method.
  arXiv  Detail & Related papers  (2020-03-27T09:19:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.