Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study

• URL: http://arxiv.org/abs/2606.09362v1
• Date: Mon, 08 Jun 2026 11:35:34 GMT
• Title: Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study
• Abstract summary: Re-Identification in autonomous driving is typically formulated as a visual matching problem, where observations of vehicles, pedestrians, and cyclists are associated across time, frames, or camera views.<n>We propose a baseline study of a zero-shot pipeline using Vision-Language Models (VLMs) to generate textual descriptions of detected traffic participants.<n>We show that zero-shot semantic descriptions can support effective object re-identification, achieving retrieval performance comparable to a supervised CNN baseline.
• Score: 0.5599792629509229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Re-Identification (ReID) in autonomous driving is typically formulated as a visual matching problem, where observations of vehicles, pedestrians, and cyclists are associated across time, frames, or camera views using learned appearance embeddings, often complemented by motion, geometric, or multimodal cues. However, purely visual representations may be sensitive to viewpoint, occlusion, illumination, and sensor-domain variations, limiting their interpretability and robustness in complex driving scenes. We propose a baseline study of a zero-shot pipeline using Vision-Language Models (VLMs) to generate textual descriptions of detected traffic participants and evaluate whether these descriptions can support identity matching across observations. Instead of relying only on low-level visual similarity, the proposed formulation represents each object through structured semantic attributes, including category, color, shape, pose, visible parts, spatial context, and distinctive visual cues. This study provides an initial benchmark for language-based re-identification in autonomous-driving scenarios, discussing and evaluating the strengths and limitations of current VLMs for this task. Results demonstrate that zero-shot semantic descriptions can support effective object re-identification, achieving retrieval performance comparable to a supervised CNN baseline while offering greater interpretability through explicit identity cues. However, the experiments also reveal important challenges, including attribute inconsistency across viewpoints and limited fine-grained discrimination between visually similar instances.

Related papers

• Diagnosing Visual Ignorance in Vision-Language Models [29.2851901986069]
  Vision-Language Models (VLMs) frequently rely on language priors, producing confident answers that are weakly grounded in visual evidence.<n>In this work, we study language-prior reliance from both mechanistic and behavioral perspectives.<n>We find that language-prior reliance is a systematic routing failure affecting both model internals and benchmark validity.
  arXiv  Detail & Related papers  (2026-06-05T04:16:01Z)
• Steerable Visual Representations [72.39044430620977]
  We introduce Steerable Visual Representations, whose global and local features can be steered with natural language.<n>Our method also matches or outperforms dedicated approaches on anomaly detection and personalized object discrimination.
  arXiv  Detail & Related papers  (2026-04-02T17:59:49Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Can Vision-Language Models Count? A Synthetic Benchmark and Analysis of Attention-Based Interventions [0.4934817254755008]
  Vision Language Models (VLMs) often rely on inherent biases learned during training when responding to queries about visual properties of images.<n>We build upon this research by developing a synthetic benchmark dataset and evaluation framework to determine how counting performance varies as image and prompt properties change.<n>We implement attention-based interventions to focus on visual tokens at different layers and evaluate their impact on counting performance across a range of visual conditions.
  arXiv  Detail & Related papers  (2025-11-21T19:18:41Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• Vision language models are unreliable at trivial spatial cognition [0.2902243522110345]
  Vision language models (VLMs) are designed to extract relevant visuospatial information from images.<n>We develop a benchmark dataset -- TableTest -- whose images depict 3D scenes of objects arranged on a table, and used it to evaluate state-of-the-art VLMs.<n>Results show that performance could be degraded by minor variations of prompts that use equivalent descriptions.
  arXiv  Detail & Related papers  (2025-04-22T17:38:01Z)
• Teaching VLMs to Localize Specific Objects from In-context Examples [56.797110842152]
  We find that present-day Vision-Language Models (VLMs) lack a fundamental cognitive ability: learning to localize specific objects in a scene by taking into account the context.<n>This work is the first to explore and benchmark personalized few-shot localization for VLMs.
  arXiv  Detail & Related papers  (2024-11-20T13:34:22Z)
• Unifying Vision-Language Representation Space with Single-tower Transformer [29.604520441315135]
  We train a model to learn a unified vision-language representation space that encodes both modalities at once in a modality-agnostic manner. We discover intriguing properties that distinguish OneR from the previous works that learn modality-specific representation spaces.
  arXiv  Detail & Related papers  (2022-11-21T02:34:21Z)
• Pluggable Weakly-Supervised Cross-View Learning for Accurate Vehicle Re-Identification [53.6218051770131]
  Cross-view consistent feature representation is key for accurate vehicle ReID. Existing approaches resort to supervised cross-view learning using extensive extra viewpoints annotations. We present a pluggable Weakly-supervised Cross-View Learning (WCVL) module for vehicle ReID.
  arXiv  Detail & Related papers  (2021-03-09T11:51:09Z)
• Parsing-based View-aware Embedding Network for Vehicle Re-Identification [138.11983486734576]
  We propose a parsing-based view-aware embedding network (PVEN) to achieve the view-aware feature alignment and enhancement for vehicle ReID. The experiments conducted on three datasets show that our model outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-04-10T13:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.