A Reference Architecture for Agentic Hybrid Retrieval in Dataset Search

• URL: http://arxiv.org/abs/2604.16394v1
• Date: Sat, 28 Mar 2026 22:56:57 GMT
• Title: A Reference Architecture for Agentic Hybrid Retrieval in Dataset Search
• Abstract summary: Ad hoc dataset search requires matching underspecified natural-language queries against sparse, heterogeneous metadata records.<n>We propose a bounded, auditable reference architecture for agentic hybrid retrieval that combines BM25 lexical search with dense-embedding retrieval.<n>An offline metadata augmentation step is introduced to reduce the vocabulary mismatch between user intent and provider-authored metadata.
• Score: 0.6299766708197881
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ad hoc dataset search requires matching underspecified natural-language queries against sparse, heterogeneous metadata records, a task where typical lexical or dense retrieval alone falls short. We reposition dataset search as a software-architecture problem and propose a bounded, auditable reference architecture for agentic hybrid retrieval that combines BM25 lexical search with dense-embedding retrieval via reciprocal rank fusion (RRF), orchestrated by a large language model (LLM) agent that repeatedly plans queries, evaluates the sufficiency of results, and reranks candidates. To reduce the vocabulary mismatch between user intent and provider-authored metadata, we introduce an offline metadata augmentation step in which an LLM generates pseudo-queries for each dataset record, augmenting both retrieval indexes before query time. Two architectural styles are examined: a single ReAct agent and a multi-agent horizontal architecture with Feedback Control. Their quality-attribute tradeoffs are analyzed with respect to modifiability, observability, performance, and governance. An evaluation framework comprising seven system variants is defined to isolate the contribution of each architectural decision. The architecture is presented as an extensible reference design for the software architecture community, incorporating explicit governance tactics to bound and audit nondeterministic LLM components.

Related papers

• ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation [3.6028767235872965]
  In this paper, we introduce a query-conditioned RAG framework that jointly adapts indexing and retrieval to the incoming query.<n>We evaluate our framework on large-scale, heterogeneous historical archives and show that conditioning both indexing and retrieval on the query consistently outperforms both naive baselines and strong state-of-the-art RAG systems.
  arXiv  Detail & Related papers  (2026-09-15T11:23:16Z)
• Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval [19.3757623473603]
  Real-world data spans tables, documents, and semi-structured files with implicit semantics.<n>We present a system that automatically discovers an executable schema from raw multi-source data.
  arXiv  Detail & Related papers  (2026-06-03T20:28:36Z)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
  We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
  arXiv  Detail & Related papers  (2026-05-03T19:13:11Z)
• AnnoRetrieve: Efficient Structured Retrieval for Unstructured Document Analysis [11.689256498133446]
  We propose AnnoRetrieve, a novel retrieval paradigm that shifts from embeddings to structured annotations.<n>Our system replaces expensive vector comparisons with lightweight structured queries over automatically induced schemas.<n>AnoRetrieve establishes a new paradigm for cost-effective, precise, and scalable document analysis through intelligent structuring.
  arXiv  Detail & Related papers  (2026-04-03T03:34:19Z)
• LLM-based Semantic Search for Conversational Queries in E-commerce [1.3645712130536118]
  We present an LLM-based semantic search framework that captures user intent from conversational queries.<n>Our framework achieves strong precision and recall across various settings compared to baseline approaches on a real-world dataset.
  arXiv  Detail & Related papers  (2026-01-23T06:35:28Z)
• Reasoning-enhanced Query Understanding through Decomposition and Interpretation [87.56450566014625]
  ReDI is a Reasoning-enhanced approach for query understanding through Decomposition and Interpretation.<n>We compiled a large-scale dataset of real-world complex queries from a major search engine.<n> Experiments on BRIGHT and BEIR demonstrate that ReDI consistently surpasses strong baselines in both sparse and dense retrieval paradigms.
  arXiv  Detail & Related papers  (2025-09-08T10:58:42Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision [0.0]
  "FemmIR" is a framework to retrieve results relevant to information needs expressed with multimodal queries by example without any similarity label.<n>We empirically evaluate FemmIR on a missing person use case with MuQNOL.
  arXiv  Detail & Related papers  (2025-06-25T00:25:08Z)
• Mixture-of-RAG: Integrating Text and Tables with Large Language Models [5.038576104344948]
  Heterogeneous Document RAG requires joint retrieval and reasoning across textual and hierarchical data.<n>We propose MixRAG, a novel three-stage framework that preserves hierarchical structure and heterogeneous relationships.<n>Experiments show that MixRAG boosts top-1 retrieval by 46% over strong text-only, table-only, and naive-mixture baselines.
  arXiv  Detail & Related papers  (2025-04-13T13:02:33Z)
• HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation [11.53083922927901]
  HM-RAG is a novel Hierarchical Multi-agent Multimodal RAG framework.<n>It pioneers collaborative intelligence for dynamic knowledge synthesis across structured, unstructured, and graph-based data.
  arXiv  Detail & Related papers  (2025-04-13T06:55:33Z)
• QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding [53.69841526266547]
  Fine-tuning a pre-trained Vision-Language Model with new datasets often falls short in optimizing the vision encoder.<n>We introduce QID, a novel, streamlined, architecture-preserving approach that integrates query embeddings into the vision encoder.
  arXiv  Detail & Related papers  (2025-04-03T18:47:16Z)
• MultiConIR: Towards multi-condition Information Retrieval [38.864056667809095]
  MultiConIR is a benchmark designed to evaluate retrieval and reranking models under nuanced multi-condition query scenarios.<n>Most retrievers and rerankers exhibit severe performance degradation as query complexity increases.<n>This work delves into the factors contributing to reranker performance deterioration and examines how condition positioning within queries affects similarity assessment.
  arXiv  Detail & Related papers  (2025-03-11T05:02:03Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases [93.96463520716759]
  We develop STARK, a large-scale Semi-structure retrieval benchmark on Textual and Knowledge Bases. Our benchmark covers three domains: product search, academic paper search, and queries in precision medicine. We design a novel pipeline to synthesize realistic user queries that integrate diverse relational information and complex textual properties.
  arXiv  Detail & Related papers  (2024-04-19T22:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.