Region-R1: Reinforcing Query-Side Region Cropping for Multi-Modal Re-Ranking

• URL: http://arxiv.org/abs/2604.05268v2
• Date: Wed, 08 Apr 2026 04:25:11 GMT
• Title: Region-R1: Reinforcing Query-Side Region Cropping for Multi-Modal Re-Ranking
• Abstract summary: Multi-modal retrieval-augmented generation relies heavily on re-rankers to surface the most relevant evidence for image-question queries.<n>Standard re-rankers process the full query image as a global embedding, making them susceptible to visual distractors that skew similarity scores.<n>We propose Region-R1, a query-side region cropping framework that formulates region selection as a decision-making problem during re-ranking.
• Score: 15.826680638852224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-modal retrieval-augmented generation (MM-RAG) relies heavily on re-rankers to surface the most relevant evidence for image-question queries. However, standard re-rankers typically process the full query image as a global embedding, making them susceptible to visual distractors (e.g., background clutter) that skew similarity scores. We propose Region-R1, a query-side region cropping framework that formulates region selection as a decision-making problem during re-ranking, allowing the system to learn to retain the full image or focus only on a question-relevant region before scoring the retrieved candidates. Region-R1 learns a policy with a novel region-aware group relative policy optimization (r-GRPO) to dynamically crop a discriminative region. Across two challenging benchmarks, E-VQA and InfoSeek, Region-R1 delivers consistent gains, achieving state-of-the-art performances by increasing conditional Recall@1 by up to 20%. These results show the great promise of query-side adaptation as a simple but effective way to strengthen MM-RAG re-ranking.

Related papers

• RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models [42.63747293653087]
  Region-level retrieval plays a crucial role in realworld applications such as e-commerce product search and RAG.<n>We present RegRet, an LMM-based framework that enhances the regional representations without compromising global performance.<n>At its core, RegRet integrates a Region-Aware to capture detailed regional features while balancing them with the global background context.
  arXiv  Detail & Related papers  (2026-09-15T08:39:55Z)
• ROGLE: Robust Global-Local Alignment with Automated Region Supervision for Text-Based Person Search [12.859616063162491]
  Text-Based Person Search models struggle with fine-grained understanding due to global representational bias and semantic sparsity inherited from training on short captions.<n>We propose ROGLE, a unified framework that overcomes reliance on costly manual annotations through an automated Region-to-Sentence Matching strategy.<n>We also introduce the P-VLG Benchmark, a large-scale dataset constructed by curating and enriching images from established public benchmarks.
  arXiv  Detail & Related papers  (2026-06-01T07:41:44Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• AINet: Anchor Instances Learning for Regional Heterogeneity in Whole Slide Image [61.54860340942449]
  We introduce a novel concept of anchor instance (AI), a compact subset of instances that are representative within their regions (local) and discriminative at the bag (global) level.<n>These AIs act as semantic references to guide interactions across regions, correcting non-discriminative patterns while preserving regional diversity.<n>We develop a concise yet effective framework, AINet, which employs a simple predictor and surpasses state-of-the-art methods with substantially fewer FLOPs and parameters.
  arXiv  Detail & Related papers  (2026-02-21T09:36:27Z)
• Pixel-Grounded Retrieval for Knowledgeable Large Multimodal Models [58.46663983451155]
  PixSearch is an end-to-end Segmenting Large Multimodal Model (LMM) that unifies region-level perception and retrieval-augmented reasoning.<n>During encoding, PixSearch emits search> tokens to trigger retrieval, selects query modalities (text, image, or region), and generates pixel-level masks that directly serve as visual queries.<n>On egocentric and entity-centric VQA benchmarks, PixSearch substantially improves factual consistency and generalization.
  arXiv  Detail & Related papers  (2026-01-27T00:46:08Z)
• RegionRAG: Region-level Retrieval-Augumented Generation for Visually-Rich Documents [40.107303323097646]
  modelname is a novel framework that shifts the retrieval paradigm from the document level to the region level.<n> Experiments on six benchmarks demonstrate that RegionRAG achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-10-31T08:00:32Z)
• Creatively Upscaling Images with Global-Regional Priors [98.24171965992916]
  C-Upscale is a new recipe of tuning-free image upscaling.<n>It pivots on global-regional priors derived from given global prompt and estimated regional prompts.<n>It generates ultra-high-resolution images with higher visual fidelity and more creative regional details.
  arXiv  Detail & Related papers  (2025-05-22T17:51:50Z)
• Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition [26.27331683733832]
  Visual Place Recognition (VPR) is aimed at predicting the location of a query image by referencing a database of geotagged images.<n>Existing methods lack precisely modeling and full exploitation of discriminative regions.<n>We propose the Focus on Local (FoL) approach to stimulate the performance of image retrieval and re-ranking in VPR simultaneously.
  arXiv  Detail & Related papers  (2025-04-14T05:04:51Z)
• LocalSR: Image Super-Resolution in Local Region [42.85362760049813]
  We propose a novel task, called LocalSR, to restore only local regions of the low-resolution image.<n>Our approach, with its reduced low complexity, outperforms variants that focus exclusively on the region.
  arXiv  Detail & Related papers  (2024-12-05T16:30:54Z)
• Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training [79.27663870280038]
  We introduce Contrastive Region Guidance (CRG), a training-free guidance method that enables open-source vision-language models to respond to visual prompts. When region annotations are provided, CRG increases absolute accuracy by up to 11.1% on ViP-Bench. We also show CRG's applicability to spatial reasoning, with 10% improvement on What'sUp.
  arXiv  Detail & Related papers  (2024-03-04T18:55:30Z)
• R-MAE: Regions Meet Masked Autoencoders [113.73147144125385]
  We explore regions as a potential visual analogue of words for self-supervised image representation learning. Inspired by Masked Autoencoding (MAE), a generative pre-training baseline, we propose masked region autoencoding to learn from groups of pixels or regions.
  arXiv  Detail & Related papers  (2023-06-08T17:56:46Z)
• Region Rebalance for Long-Tailed Semantic Segmentation [89.84860341946283]
  We first investigate and identify the main challenges of addressing this issue through pixel rebalance. Then a simple and yet effective region rebalance scheme is derived based on our analysis. With the proposed region rebalance scheme, state-of-the-art BEiT receives +0.7% gain in terms of mIoU on the ADE20K val set.
  arXiv  Detail & Related papers  (2022-04-05T03:47:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.