SAGE: Semantic Attribute Graphs for Multi-Entity Visual Retrieval

• URL: http://arxiv.org/abs/2609.04255v1
• Date: Tue, 01 Sep 2026 06:07:20 GMT
• Title: SAGE: Semantic Attribute Graphs for Multi-Entity Visual Retrieval
• Abstract summary: We propose SAGE, a training-free framework that parses semantic entities from dense document images.<n>SAGE substantially reduces semantic dilution and outperforms patch-level and OCR-based retrieval baselines.
• Score: 7.2391463583497995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense document images often contain many fine-grained visual and textual entities whose relevance depends on a user query. Standard vision-language retrievers encode cropped regions with a single vector, which can mix distinct entity signals and obscure the evidence needed for fine-grained retrieval. We call this failure mode Semantic Dilution and quantitatively show that it degrades entity-level retrieval as a function of entity density. To mitigate it, we propose SAGE, a training-free framework that parses semantic entities from dense document images, represents them as hierarchical graph nodes with multi-vector embeddings, and retrieves query-relevant evidence through iterative entity-level subgraph matching. We also introduce DEAR, a dataset of 1,055 query--image pairs sourced from product detail pages, where each query requires retrieving and comparing multiple fine-grained entities from visually dense inputs across four question types of increasing complexity. Experiments show that SAGE substantially reduces semantic dilution and outperforms patch-level and OCR-based retrieval baselines on DEAR, achieving a Recall@3 of 0.849 and a generation score of 2.746 on multi-entity visual comparison queries. Our code is available at https://github.com/All4Nothing/SAGE.

Related papers

• Query Symbolically or Retrieve Semantically? A Dataset and Method for Semi-Structured Question Answering [2.6264656632831667]
  DualGraph represents documents through two complementary views: a Textual Knowledge Graph for semantic retrieval and a Symbolic Knowledge Graph for symbolic querying over typed subject--predicate--object triples.<n>We also introduce SpecsQA, a benchmark from a commercial shopping website with semi-structured product documents and manually curated questions spanning open-ended and specification-oriented retrieval. Experiments show that DualGraph consistently outperforms state-of-the-art dense-retrieval, GraphRAG, symbolic, and table-oriented baselines across question types.
  arXiv  Detail & Related papers  (2026-05-26T15:22:42Z)
• BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment [5.285385905661152]
  Best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers.<n>We present textbfBRIDGE, a two-component system that resolves this mismatch without multimodal encoders.<n>textbfFORGE is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings.<n>textbfLENS is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces.
  arXiv  Detail & Related papers  (2026-04-08T15:28:21Z)
• Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders [41.08205377881149]
  This work explores text-to-image retrieval for queries that specify or describe a semantic category.<n>We transform the text query into a visual query using a generative diffusion model.<n>Then, we estimate image-to-image similarity with a vision model.
  arXiv  Detail & Related papers  (2025-08-29T18:24:38Z)
• Composed Object Retrieval: Object-level Retrieval via Composed Expressions [71.47650333199628]
  Composed Object Retrieval (COR) is a brand-new task that goes beyond image-level retrieval to achieve object-level precision.<n>We construct COR127K, the first large-scale COR benchmark that contains 127,166 retrieval triplets with various semantic transformations in 408 categories.<n>We also present CORE, a unified end-to-end model that integrates reference region encoding, adaptive visual-textual interaction, and region-level contrastive learning.
  arXiv  Detail & Related papers  (2025-08-06T13:11:40Z)
• Chain of Retrieval: Multi-Aspect Iterative Search Expansion and Post-Order Search Aggregation for Full Paper Retrieval [68.71038700559195]
  Chain of Retrieval(COR) is a novel iterative framework for full-paper retrieval.<n>We present SCIBENCH, a benchmark providing both complete and segmented contexts of full papers for queries and candidates.
  arXiv  Detail & Related papers  (2025-07-14T08:41:53Z)
• Entity Image and Mixed-Modal Image Retrieval Datasets [9.6977953463099]
  This paper introduces a novel benchmark to rigorously evaluate image retrieval that demands deep cross-modal contextual understanding.<n>We present two new datasets: the Entity Image dataset (EI), providing canonical images for Wikipedia entities, and the Mixed-Modal Image Retrieval dataset (MMIR), derived from the WIT dataset.<n>We empirically validate the benchmark's utility as both a training corpus and an evaluation set for mixed-modal retrieval.
  arXiv  Detail & Related papers  (2025-06-02T22:04:06Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• A Dual-way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking [17.847936914174543]
  Multimodal Entity Linking (MEL) aims at linking ambiguous mentions with multimodal information to entity in Knowledge Graph (KG) such as Wikipedia. We formulate multimodal entity linking as a neural text matching problem where each multimodal information (text and image) is treated as a query. This paper introduces a dual-way enhanced (DWE) framework for MEL.
  arXiv  Detail & Related papers  (2023-12-19T03:15:50Z)
• EDIS: Entity-Driven Image Search over Multimodal Web Content [95.40238328527931]
  We introduce textbfEntity-textbfDriven textbfImage textbfSearch (EDIS), a dataset for cross-modal image search in the news domain. EDIS consists of 1 million web images from actual search engine results and curated datasets, with each image paired with a textual description.
  arXiv  Detail & Related papers  (2023-05-23T02:59:19Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction [88.6585431949086]
  We propose a novel Hierarchical Visual Prefix fusion NeTwork (HVPNeT) for visual-enhanced entity and relation extraction. We regard visual representation as pluggable visual prefix to guide the textual representation for error insensitive forecasting decision. Experiments on three benchmark datasets demonstrate the effectiveness of our method, and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-07T02:10:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.