Composed Object Retrieval: Object-level Retrieval via Composed Expressions

• URL: http://arxiv.org/abs/2508.04424v1
• Date: Wed, 06 Aug 2025 13:11:40 GMT
• Title: Composed Object Retrieval: Object-level Retrieval via Composed Expressions
• Authors: Tong Wang, Guanyu Yang, Nian Liu, Zongyan Han, Jinxing Zhou, Salman Khan, Fahad Shahbaz Khan,
• Abstract summary: Composed Object Retrieval (COR) is a brand-new task that goes beyond image-level retrieval to achieve object-level precision.<n>We construct COR127K, the first large-scale COR benchmark that contains 127,166 retrieval triplets with various semantic transformations in 408 categories.<n>We also present CORE, a unified end-to-end model that integrates reference region encoding, adaptive visual-textual interaction, and region-level contrastive learning.
• Score: 71.47650333199628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieving fine-grained visual content based on user intent remains a challenge in multi-modal systems. Although current Composed Image Retrieval (CIR) methods combine reference images with retrieval texts, they are constrained to image-level matching and cannot localize specific objects. To this end, we propose Composed Object Retrieval (COR), a brand-new task that goes beyond image-level retrieval to achieve object-level precision, allowing the retrieval and segmentation of target objects based on composed expressions combining reference objects and retrieval texts. COR presents significant challenges in retrieval flexibility, which requires systems to identify arbitrary objects satisfying composed expressions while avoiding semantically similar but irrelevant negative objects within the same scene. We construct COR127K, the first large-scale COR benchmark that contains 127,166 retrieval triplets with various semantic transformations in 408 categories. We also present CORE, a unified end-to-end model that integrates reference region encoding, adaptive visual-textual interaction, and region-level contrastive learning. Extensive experiments demonstrate that CORE significantly outperforms existing models in both base and novel categories, establishing a simple and effective baseline for this challenging task while opening new directions for fine-grained multi-modal retrieval research.

Related papers

• Find your Needle: Small Object Image Retrieval via Multi-Object Attention Optimization [5.2337753974570616]
  We address the challenge of Small Object Image Retrieval (SoIR), where the goal is to retrieve images containing a specific small object, in a cluttered scene.<n>Key challenge is constructing a single image descriptor, for scalable and efficient search, that effectively represents all objects in the image.<n>We introduce Multi-object Attention Optimization (MaO), a novel retrieval framework which incorporates a dedicated multi-object pre-training phase.
  arXiv  Detail & Related papers  (2025-03-10T08:27:02Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation [47.047267066525265]
  We introduce a novel approach that incorporates object-level contextual knowledge within images.<n>Our proposed approach achieves state-of-the-art performance with strong generalizability across diverse datasets.
  arXiv  Detail & Related papers  (2024-11-26T06:34:48Z)
• Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features [11.112981323262337]
  We present a simple yet effective approach to object-centric open-vocabulary image retrieval.<n>Our approach aggregates dense embeddings extracted from CLIP into a compact representation.<n>We show the effectiveness of our scheme to the task by achieving significantly better results than global feature approaches on three datasets.
  arXiv  Detail & Related papers  (2023-09-26T15:13:09Z)
• Learning Dynamic Query Combinations for Transformer-based Object Detection and Segmentation [37.24532930188581]
  Transformer-based detection and segmentation methods use a list of learned detection queries to retrieve information from the transformer network. We empirically find that random convex combinations of the learned queries are still good for the corresponding models. We propose to learn a convex combination with dynamic coefficients based on the high-level semantics of the image.
  arXiv  Detail & Related papers  (2023-07-23T06:26:27Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select [53.071352033539526]
  We study the problem of extracting N-ary relations from scientific articles. Our proposed method ReSel decomposes this task into a two-stage procedure. Our experiments on three scientific information extraction datasets show that ReSel outperforms state-of-the-art baselines significantly.
  arXiv  Detail & Related papers  (2022-10-26T02:28:02Z)
• Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects [70.49392581592089]
  We tackle the task of estimating the 3D orientation of previously-unseen objects from monocular images. We follow a retrieval-based strategy and prevent the network from learning object-specific features. Our experiments on the LineMOD, LineMOD-Occluded, and T-LESS datasets show that our method yields a significantly better generalization to unseen objects than previous works.
  arXiv  Detail & Related papers  (2022-03-16T08:53:00Z)
• Tasks Integrated Networks: Joint Detection and Retrieval for Image Search [99.49021025124405]
  In many real-world searching scenarios (e.g., video surveillance), the objects are seldom accurately detected or annotated. We first introduce an end-to-end Integrated Net (I-Net), which has three merits. We further propose an improved I-Net, called DC-I-Net, which makes two new contributions.
  arXiv  Detail & Related papers  (2020-09-03T03:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.