DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation

• URL: http://arxiv.org/abs/2508.16998v1
• Date: Sat, 23 Aug 2025 11:46:08 GMT
• Title: DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation
• Abstract summary: Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets.<n>We propose textbfDeeptextbfAgenttextbfRank (textbfDeAR), an open-source framework that decouples these tasks through a dual-stage approach.
• Score: 26.719288314007002
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets, yet single models often struggle to balance fine-grained relevance scoring with holistic cross-document analysis. We propose \textbf{De}ep\textbf{A}gent\textbf{R}ank (\textbf{\DeAR}), an open-source framework that decouples these tasks through a dual-stage approach, achieving superior accuracy and interpretability. In \emph{Stage 1}, we distill token-level relevance signals from a frozen 13B LLaMA teacher into a compact \{3, 8\}B student model using a hybrid of cross-entropy, RankNet, and KL divergence losses, ensuring robust pointwise scoring. In \emph{Stage 2}, we attach a second LoRA adapter and fine-tune on 20K GPT-4o-generated chain-of-thought permutations, enabling listwise reasoning with natural-language justifications. Evaluated on TREC-DL19/20, eight BEIR datasets, and NovelEval-2306, \DeAR surpasses open-source baselines by +5.1 nDCG@5 on DL20 and achieves 90.97 nDCG@10 on NovelEval, outperforming GPT-4 by +3.09. Without fine-tuning on Wikipedia, DeAR also excels in open-domain QA, achieving 54.29 Top-1 accuracy on Natural Questions, surpassing baselines like MonoT5, UPR, and RankGPT. Ablations confirm that dual-loss distillation ensures stable calibration, making \DeAR a highly effective and interpretable solution for modern reranking systems.\footnote{Dataset and code available at https://github.com/DataScienceUIBK/DeAR-Reranking.}.

Related papers

• NOWJ@COLIEE 2026: Adaptive Pipelines for Legal Retrieval and Reasoning [1.2884266282263097]
  This paper presents the methodologies and results of the NOWJ team's participation across all five tasks of the COLIEE 2026 competition.<n>For Task 1 (Legal Case Retrieval), we propose a four-stage candidate filtering, dense retrieval with complementary embedding models, cross-encoder reranking via fine-tuned generative rerankers.<n>For Task 2 (Legal Case Entailment), we combine BM25-based reranking, attention-based reranking, and consensus ensemble verification.<n>For Task 3 (Statute Law Retrieval and Entailment), we adopt a retrieval-aug
  arXiv  Detail & Related papers  (2026-07-18T02:46:06Z)
• AgentKGV: Agentic LLM-RAG Framework with Two-Stage Training for the Fact Verification of Knowledge Graphs [15.215103223762364]
  AgentKGV integrates dynamic routing and iterative query rewriting to verify knowledge graphs reliably at industrial scale.<n>Two-stage training strategy transfers reasoning ability from a large teacher model into a small model for stable query rewriting and reasoning.<n>Our framework improves macro-F1 over single-turn RAG by 5.5 %p, and two-stage training does it further by 9.4 %p.
  arXiv  Detail & Related papers  (2026-07-10T04:22:34Z)
• Querit-Reranker: Training Compact Multilingual Rerankers via Efficient Label-Free Distribution Adaptation [43.66118282567413]
  We present Querit-Reranker, a family of multilingual cross-encoder rerankers trained with a data-centric pipeline.<n>Our pipeline first learns general relevance modeling from large-scale ranking-oriented data, then adapts to target distributions.<n>Querit-Reranker-A0.4B improves average nDCG@10 from 54.11 to 59.28 on BEIR and from 59.87 to 67.70 on MIRACL.
  arXiv  Detail & Related papers  (2026-06-17T13:06:47Z)
• Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation [0.0]
  We study quality-latency-resource trade-offs in a retrieval-augmented generation (RAG) system that uses Low-Rank Adaptation (LoRA) of the generator.<n>We build a benchmark of 5,144 question-answer pairs over the official documentation and combine it with a fixed hybrid-retrieval pipeline (BGE-M3 dense, BGE-M3 native, Reciprocal Rank Fusion, cross-encoder reranking)
  arXiv  Detail & Related papers  (2026-05-27T09:37:55Z)
• Align Documents to Questions: Question-Oriented Document Rewriting for Retrieval-Augmented Generation [51.55755193937205]
  We propose QREAM, a style-controlled rewriter that aligns retrieved documents with a question-oriented style while preserving facts.<n>Our framework consists of two stages: (1) QREAM-ICL, which uses stylistic seeds to guide iterative rewriting exploration; and (2) QREAM-FT, a lightweight student model distilled from denoised ICL outputs.
  arXiv  Detail & Related papers  (2026-04-19T08:39:21Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking [33.25740773392183]
  ERank is a highly effective and efficient pointwise reranker built from a reasoning LLM that excels across diverse relevance scenarios.<n>We propose a novel two-stage training pipeline that begins with Supervised Fine-Tuning (SFT)<n>In this stage, we move beyond binary labels and train the model generatively to output fine grained integer scores, which significantly enhances relevance discrimination.<n>We evaluate the ERank reranker on the BRIGHT, FollowIR, TREC DL, and BEIR benchmarks, demonstrating superior effectiveness and robustness compared to existing approaches.
  arXiv  Detail & Related papers  (2025-08-30T14:56:53Z)
• Open-Source Agentic Hybrid RAG Framework for Scientific Literature Review [2.092154729589438]
  We present an agentic approach that encapsulates the hybrid RAG pipeline within an autonomous agent.<n>Our pipeline ingests bibliometric open-access data from PubMed, arXiv, and Google Scholar APIs.<n>A Llama-3.3-70B agent selects GraphRAG (translating queries to Cypher for KG) or VectorRAG (combining sparse and dense retrieval with re-ranking)
  arXiv  Detail & Related papers  (2025-07-30T18:54:15Z)
• Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights [75.83625828306839]
  textbfDrag-and-Drop LLMs (textitDnD) eliminates per-task training by mapping a handful of unlabeled task prompts directly to LoRA weight updates.<n>A lightweight text encoder distills each prompt batch into condition embeddings, which are then transformed by a cascaded hyper-convolutional decoder into the full set of LoRA matrices.
  arXiv  Detail & Related papers  (2025-06-19T15:38:21Z)
• Document Attribution: Examining Citation Relationships using Large Language Models [62.46146670035751]
  We propose a zero-shot approach that frames attribution as a straightforward textual entailment task.<n>We also explore the role of the attention mechanism in enhancing the attribution process.
  arXiv  Detail & Related papers  (2025-05-09T04:40:11Z)
• LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation [6.920352059545929]
  We present Less is More, the third-place winning approach in the LLMSR@XLLM25 structural reasoning task.<n>Our approach leverages a multi-agent framework with reverse-prompt induction, retrieval-augmented reasoning synthesis via GPT-4o, and dual-stage reward-guided filtering.<n>All modules are fine-tuned from Meta-Llama-3-8B-Instruct under a unified LoRA+ setup.
  arXiv  Detail & Related papers  (2025-04-23T04:19:52Z)
• Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning [76.50690734636477]
  We introduce Rank-R1, a novel LLM-based reranker that performs reasoning over both the user query and candidate documents before performing the ranking task.<n>Our experiments on the TREC DL and BRIGHT datasets show that Rank-R1 is highly effective, especially for complex queries.
  arXiv  Detail & Related papers  (2025-03-08T03:14:26Z)
• Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models [0.6827423171182154]
  Retrieval-Augmented Generation (RAG) systems face significant performance gaps when applied to technical domains.<n>We propose a framework combining granular evaluation metrics with synthetic data generation to optimize domain-specific RAG performance.<n>Our empirical analysis reveals critical insights: smaller chunks (less than 10 tokens) improve precision by 31-42%.
  arXiv  Detail & Related papers  (2025-02-21T06:38:57Z)
• RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs [60.38044044203333]
  Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG) We propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. For generation, we compare our model with many strong baselines, including GPT-4-0613, GPT-4-turbo-2024-0409, and ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks.
  arXiv  Detail & Related papers  (2024-07-02T17:59:17Z)
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting [65.00288634420812]
  Pairwise Ranking Prompting (PRP) is a technique to significantly reduce the burden on Large Language Models (LLMs) Our results are the first in the literature to achieve state-of-the-art ranking performance on standard benchmarks using moderate-sized open-sourced LLMs.
  arXiv  Detail & Related papers  (2023-06-30T11:32:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.