ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search

• URL: http://arxiv.org/abs/2606.01825v1
• Date: Mon, 01 Jun 2026 07:41:44 GMT
• Title: ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search
• Abstract summary: Text-Based Person Search models struggle with fine-grained understanding due to global representational bias and semantic sparsity inherited from training on short captions.<n>We propose ROGLE, a unified framework that overcomes reliance on costly manual annotations through an automated Region-to-Sentence Matching strategy.<n>We also introduce the P-VLG Benchmark, a large-scale dataset constructed by curating and enriching images from established public benchmarks.
• Score: 12.859616063162491
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-Based Person Search (TBPS) aims to retrieve pedestrian images using natural language queries. However, existing TBPS models, especially those based on CLIP, struggle with fine-grained understanding due to global representational bias and semantic sparsity inherited from training on short captions. This results in weak fine-grained alignment, exacerbated by the scarcity of region-level annotations. To address this, we propose ROGLE (Robust Global-Local Embedding), a unified framework that overcomes reliance on costly manual annotations through an automated Region-to-Sentence Matching (RSM) strategy. RSM automatically mines pseudo region-sentence pairs for scalable fine-grained supervision. Furthermore, ROGLE employs a multi-granular learning strategy that fuses global contrastive learning with region-level local alignment. We also introduce the P-VLG Benchmark, a large-scale dataset constructed by curating and enriching images from established public benchmarks. It features over 100,000 annotated regions and rich long-form captions, making it the first TBPS benchmark to support both global and local assessment protocols. Extensive experiments show that ROGLE significantly outperforms existing approaches, particularly on challenging long-form queries. Code and the P-VLG benchmark will be made publicly available.

Related papers

• Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO [43.173161414861966]
  Cycle Group Relative Policy Optimization (CycleGRPO) jointly optimize region understanding and localization for Multimodal Large Language Models (MLLMs)<n>A single MLLM first acts as the actor to generate region captions, then immediately transitions to a critic to ground its generated text back in the spatial domain.<n>A quality-aware token-level cycle-consistency reward is employed to assess the semantic discriminability of text captions.
  arXiv  Detail & Related papers  (2026-07-13T14:00:57Z)
• Zero-Shot Chinese Character Recognition via Global-Local Dual-Branch Alignment and Hierarchical Inference [15.854290215907179]
  Chinese character categories are extremely large, and unseen characters frequently arise in open-world scenarios.<n>Existing IDS-based retrieval methods usually encode a character image and its ideographic description sequence into a single global vector for matching.<n>We propose a Global-Local Hierarchical Perception Network (GL-HPN), which jointly learns global and local representations of character images and IDS sequences.
  arXiv  Detail & Related papers  (2026-05-09T09:05:59Z)
• Efficiency Follows Global-Local Decoupling [62.05489838893081]
  We introduce ConvNeur, a two-branch architecture in which a lightweight neural memory branch aggregates global context on a compact set of tokens.<n>A learned gate lets global cues modulate local features without entangling their objectives.<n>On standard classification, detection, and segmentation benchmarks, ConvNeur matches or surpasses comparable alternatives at similar or lower compute.
  arXiv  Detail & Related papers  (2026-03-20T02:20:16Z)
• Local-Global Prompt Learning via Sparse Optimal Transport [3.7098626170498643]
  Few-shot adaptation of vision-language models (VLMs) like CLIP typically relies on learning textual prompts matched to global image embeddings.<n>Recent works extend this paradigm by incorporating local image-text alignment to capture fine-grained visual cues.<n>We propose SOT-GLP, which introduces a shared sparse patch support and balanced optimal transport allocation.
  arXiv  Detail & Related papers  (2026-03-09T13:09:55Z)
• RegionReasoner: Region-Grounded Multi-Round Visual Reasoning [69.75509909581133]
  RegionReasoner is a reinforcement learning framework for visual reasoning.<n>It enforces grounded reasoning by requiring each reasoning trace to explicitly cite the corresponding reference bounding boxes.<n>RegionReasoner is optimized with structured rewards combining grounding fidelity and global-local semantic alignment.
  arXiv  Detail & Related papers  (2026-02-03T16:52:16Z)
• Large Language Model with Region-guided Referring and Grounding for CT Report Generation [4.804660464589285]
  Existing methods primarily only consider the global features of the entire volume.<n>We propose Reg2RG, the first region-guided referring and grounding framework for CT report generation.
  arXiv  Detail & Related papers  (2024-11-23T12:25:06Z)
• GlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection [5.530212768657544]
  We introduce glocal contrastive learning to improve the complementary learning of global and local prompts.<n>The generalization performance of GlocalCLIP in ZSAD was demonstrated on 15 real-world datasets.
  arXiv  Detail & Related papers  (2024-11-09T05:22:13Z)
• RegionGPT: Towards Region Understanding Vision Language Model [88.42271128373191]
  RegionGPT (short as RGPT) is a novel framework designed for complex region-level captioning and understanding. We develop an automated region caption data generation pipeline, enriching the training set with detailed region-level captions. We demonstrate that a universal RGPT model can be effectively applied and significantly enhancing performance across a range of region-level tasks.
  arXiv  Detail & Related papers  (2024-03-04T18:58:08Z)
• Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training [79.27663870280038]
  We introduce Contrastive Region Guidance (CRG), a training-free guidance method that enables open-source vision-language models to respond to visual prompts. When region annotations are provided, CRG increases absolute accuracy by up to 11.1% on ViP-Bench. We also show CRG's applicability to spatial reasoning, with 10% improvement on What'sUp.
  arXiv  Detail & Related papers  (2024-03-04T18:55:30Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Context-aware Biaffine Localizing Network for Temporal Sentence Grounding [61.18824806906945]
  This paper addresses the problem of temporal sentence grounding (TSG) TSG aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. We propose a novel localization framework that scores all pairs of start and end indices within the video simultaneously with a biaffine mechanism.
  arXiv  Detail & Related papers  (2021-03-22T03:13:05Z)
• Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification [82.6971648465279]
  We propose a novel Global-guided Reciprocal Learning framework for video-based person Re-ID. Our approach can achieve better performance than other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-03-07T12:27:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.