Overview of the TREC 2025 Retrieval Augmented Generation (RAG) Track

• URL: http://arxiv.org/abs/2603.09891v1
• Date: Tue, 10 Mar 2026 16:49:18 GMT
• Title: Overview of the TREC 2025 Retrieval Augmented Generation (RAG) Track
• Abstract summary: Second edition of the TREC Retrieval Augmented Generation (RAG) Track advances research on systems that integrate retrieval and generation.<n>Building on the foundation of the inaugural 2024 track, this year's challenge introduces long, multi-sentence narrative queries.<n>Participants are tasked with designing pipelines that combine retrieval and generation while ensuring transparency and factual grounding.
• Score: 52.136178637532886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The second edition of the TREC Retrieval Augmented Generation (RAG) Track advances research on systems that integrate retrieval and generation to address complex, real-world information needs. Building on the foundation of the inaugural 2024 track, this year's challenge introduces long, multi-sentence narrative queries to better reflect the deep search task with the growing demand for reasoning-driven responses. Participants are tasked with designing pipelines that combine retrieval and generation while ensuring transparency and factual grounding. The track leverages the MS MARCO V2.1 corpus and employs a multi-layered evaluation framework encompassing relevance assessment, response completeness, attribution verification, and agreement analysis. By emphasizing multi-faceted narratives and attribution-rich answers from over 150 submissions this year, the TREC 2025 RAG Track aims to foster innovation in creating trustworthy, context-aware systems for retrieval augmented generation.

Related papers

• DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning [58.09293442211248]
  Deep Research (DR) has emerged as a new agentic paradigm to tackle complex, open-ended research tasks.<n>This report presents DuMate-DeepResearch, a multi-agent DR framework built on the Qianfan Agent Foundry.
  arXiv  Detail & Related papers  (2026-06-05T14:10:48Z)
• From Questions to Trust Reports: A LLM-IR Framework for the TREC 2025 DRAGUN Track [4.485089827405665]
  The DRAGUN Track at TREC 2025 targets the growing need for effective support tools that help users evaluate the trustworthiness of online news.<n>We describe the UR_Trecking system submitted for both Task 1 (critical question generation) and Task 2 (retrieval-augmented trustworthiness reporting)<n>Our approach combines LLM-based question generation with semantic filtering, diversity enforcement using clustering, and several query expansion strategies.
  arXiv  Detail & Related papers  (2026-03-24T12:22:27Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage [85.77071158740544]
  Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation.<n>We investigate whether upstream retrieval metrics can serve as reliable early indicators of the final generated response's information coverage.<n>Our findings demonstrate strong correlations between coverage-based retrieval metrics and nugget coverage in generated responses at both topic and system levels.
  arXiv  Detail & Related papers  (2026-03-09T18:20:20Z)
• IterResearch: Rethinking Long-Horizon Agents via Markovian State Reconstruction [107.49922328855025]
  IterResearch is a novel iterative deep-research paradigm that reformulates long-horizon research as a Markov Decision Process.<n>It achieves substantial improvements over existing open-source agents with average +14.5pp across six benchmarks.<n>It serves as an effective prompting strategy, improving frontier models by up to 19.2pp over ReAct on long-horizon tasks.
  arXiv  Detail & Related papers  (2025-11-10T17:30:08Z)
• Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers [0.0]
  Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm to enhance large language models.<n>RAG introduces new challenges in retrieval quality, grounding fidelity, pipeline efficiency, and robustness against noisy or adversarial inputs.<n>This survey aims to consolidate current knowledge in RAG research and serve as a foundation for the next generation of retrieval-augmented language modeling systems.
  arXiv  Detail & Related papers  (2025-05-28T22:57:04Z)
• A Survey of Multimodal Retrieval-Augmented Generation [3.9616308910160445]
  Multimodal Retrieval-Augmented Generation (MRAG) enhances large language models (LLMs) by integrating multimodal data (text, images, videos) into retrieval and generation processes.<n>Recent studies show MRAG outperforms traditional Retrieval-Augmented Generation (RAG) in scenarios requiring both visual and textual understanding.
  arXiv  Detail & Related papers  (2025-03-26T02:43:09Z)
• Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets [8.734527090842139]
  We introduce a new resource for assessing the retrieval effectiveness and relevance of response generated by RAG systems.<n>Our dataset extends to the TREC iKAT 2024 collection, which includes 17 conversations and 20,575 relevance passage assessments.
  arXiv  Detail & Related papers  (2025-03-12T23:44:10Z)
• INQUIRE: A Natural World Text-to-Image Retrieval Benchmark [51.823709631153946]
  We introduce INQUIRE, a text-to-image retrieval benchmark designed to challenge multimodal vision-language models on expert-level queries. InQUIRE includes iNaturalist 2024 (iNat24), a new dataset of five million natural world images, along with 250 expert-level retrieval queries. Our benchmark evaluates two core retrieval tasks: (1) INQUIRE-Fullrank, a full dataset ranking task, and (2) INQUIRE-Rerank, a reranking task for refining top-100 retrievals.
  arXiv  Detail & Related papers  (2024-11-04T19:16:53Z)
• The 2nd FutureDial Challenge: Dialog Systems with Retrieval Augmented Generation (FutureDial-RAG) [23.849336345191556]
  The challenge builds upon the MobileCS2 dataset, a real-life customer service datasets with nearly 3000 high-quality dialogs. We define two tasks, track 1 for knowledge retrieval and track 2 for response generation, which are core research questions in dialog systems with RAG. We build baseline systems for the two tracks and design metrics to measure whether the systems can perform accurate retrieval and generate informative and coherent response.
  arXiv  Detail & Related papers  (2024-05-21T07:35:21Z)
• A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers [76.51245425667845]
  Relation extraction (RE) involves identifying the relations between entities from underlying content. Deep neural networks have dominated the field of RE and made noticeable progress. This survey is expected to facilitate researchers' collaborative efforts to address the challenges of real-world RE systems.
  arXiv  Detail & Related papers  (2023-06-03T08:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.