Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems

• URL: http://arxiv.org/abs/2605.04018v1
• Date: Tue, 05 May 2026 17:42:50 GMT
• Title: Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems
• Abstract summary: Reasoning-intensive retrieval aims to surface evidence that supports downstream reasoning rather than merely matching topical similarity.<n>We introduce BRIGHT-Pro, an expert-annotated benchmark that expands each query with multi-aspect gold evidence.<n>We also construct RTriever- Synth, an aspect-decomposed synthetic corpus that generates complementary positives and positive-conditioned hard negatives.
• Score: 55.04168927059962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reasoning-intensive retrieval aims to surface evidence that supports downstream reasoning rather than merely matching topical similarity. This capability is increasingly important for agentic search systems, where retrievers must provide complementary evidence across iterative search and synthesis. However, existing work remains limited on both evaluation and training: benchmarks such as BRIGHT provide narrow gold sets and evaluate retrievers in isolation, while synthetic training corpora often optimize single-passage relevance rather than evidence portfolio construction. We introduce BRIGHT-Pro, an expert-annotated benchmark that expands each query with multi-aspect gold evidence and evaluates retrievers under both static and agentic search protocols. We further construct RTriever-Synth, an aspect-decomposed synthetic corpus that generates complementary positives and positive-conditioned hard negatives, and use it to LoRA fine-tune RTriever-4B from Qwen3-Embedding-4B. Experiments across lexical, general-purpose, and reasoning-intensive retrievers show that aspect-aware and agentic evaluation expose behaviors hidden by standard metrics, while RTriever-4B substantially improves over its base model.

Related papers

• Beyond One-Shot Expansion: Contrastive Evidence Exploration for Multi-Hop Retrieval [19.37201042661162]
  We propose a training-free multi-hop retrieval framework that integrates evidence-conditioned exploration, passage-specific contrastive refinement, and coverage-aware final ranking.<n> Experiments on MuSiQue, HotpotQA, and 2WikiMultihopQA demonstrate consistent improvements in retrieval quality and downstream QA performance over baselines.
  arXiv  Detail & Related papers  (2026-09-07T05:13:56Z)
• R$^2$-Searcher: Calibrating Retrieval and Reasoning Boundaries for Agentic Search [59.034878430183305]
  R$2$-Searcher explicitly explores and calibrates the retrieval and reasoning boundaries.<n>We show that R$2$-Searcher significantly outperforms state-of-the-art agentic search methods in answer accuracy and retrieval-reasoning quality.
  arXiv  Detail & Related papers  (2026-06-26T19:44:34Z)
• DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents [17.420077157633852]
  Reinforcement learning with rubric-based rewards improves deep research agents by optimizing them against checkable criteria that translate report quality into reward signals.<n>Most existing studies ask an LLM to generate rubrics for a given query, but when the model fails to infer the underlying information needs, the generated rubrics may be incomplete and reduce RL efficiency.<n>To obtain more reliable query--rubric supervision, we introduce DeepRubric, a data construction framework that reverses this process.
  arXiv  Detail & Related papers  (2026-06-15T17:52:27Z)
• Deep Research as Rubric for Reinforcement Learning [35.642522713903794]
  We propose a two-stage framework for constructing open-ended rubrics.<n>DR-rubric-8B supports bootstrap generation without frontier-model assistance.<n>Experiments show that DR-rubric achieves strong competitive performance with only 1K -- 3K training instances.
  arXiv  Detail & Related papers  (2026-05-31T08:25:04Z)
• RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards [76.17893114021757]
  Deep research systems that plan, search, evaluate evidence, and synthesize long-form reports lack ground-truth answers and span many tool-augmented decisions.<n>In this work, we argue that rubrics should serve not merely as final-answer evaluators, but as the shared interface that structures policy execution, judge feedback, and agent memory.<n>We introduceEM, a rubric-guided reinforcement learning framework that combines stagewise policy decomposition with reflection-based meta-policy evolution.
  arXiv  Detail & Related papers  (2026-05-11T17:40:38Z)
• CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search [51.911048955965136]
  CoSearch is a framework that jointly trains a multi-step reasoning agent and a generative document ranking model.<n>Our results show that joint training of the reasoning agent and retrieval system is both feasible and strongly performant.
  arXiv  Detail & Related papers  (2026-04-19T17:48:17Z)
• AgentIR: Reasoning-Aware Retrieval for Deep Research Agents [76.29382561831105]
  Deep Research agents generate explicit natural language reasoning before each search call.<n> Reasoning-Aware Retrieval embeds the agent's reasoning trace alongside its query.<n>DR- Synth generates Deep Research retriever training data from standard QA datasets.<n>AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch.
  arXiv  Detail & Related papers  (2026-03-04T18:47:26Z)
• Revisiting RAG Retrievers: An Information Theoretic Benchmark [10.168016086591452]
  MIGRASCOPE is a Mutual Information based RAG Retriever Analysis Scope.<n>We revisit state-of-the-art retrievers and introduce principled metrics grounded in information and statistical estimation theory.<n>We show that if chosen carefully, an ensemble of retrievers outperforms any single retriever.
  arXiv  Detail & Related papers  (2026-02-25T04:19:06Z)
• SIGHT: Reinforcement Learning with Self-Evidence and Information-Gain Diverse Branching for Search Agent [39.43590030917357]
  SIGHT is a framework that enhances search-based reasoning through Self-Evidence Support and Information-Gain Driven Diverse Branching.<n>SIGHT distills search results into high-fidelity evidence via SES and calculates an Information Gain score to pinpoint pivotal states.<n> Experiments on single-hop and multi-hop QA benchmarks demonstrate that SIGHT significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2026-02-12T04:16:55Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generation [0.0]
  We introduce FAIR-RAG, a novel agentic framework that transforms the standard RAG pipeline into a dynamic, evidence-driven reasoning process.<n>We conduct experiments on challenging multi-hop QA benchmarks, including HotpotQA, 2WikiMultiHopQA, and MusiQue.<n>Our work demonstrates that a structured, evidence-driven refinement process with explicit gap analysis is crucial for unlocking reliable and accurate reasoning in advanced RAG systems.
  arXiv  Detail & Related papers  (2025-10-25T15:59:33Z)
• DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrieval [36.38599923075882]
  DIVER is a retrieval pipeline designed for reasoning-intensive information retrieval.<n>It consists of four components: the document preprocessing stage, the query expansion stage, the retrieval stage and the reranking stage.<n>On the BRIGHT benchmark, DIVER achieves state-of-the-art nDCG@10 scores of 45.8 overall and 28.9 on original queries, consistently outperforming competitive reasoning-aware models.
  arXiv  Detail & Related papers  (2025-08-11T13:57:49Z)
• BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent [74.10138164281618]
  BrowseComp-Plus is a benchmark derived from BrowseComp, employing a fixed, carefully curated corpus.<n>This benchmark allows comprehensive evaluation and disentangled analysis of deep research agents and retrieval methods.
  arXiv  Detail & Related papers  (2025-08-08T17:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.