Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking

• URL: http://arxiv.org/abs/2601.11273v1
• Date: Fri, 16 Jan 2026 13:19:17 GMT
• Title: Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking
• Authors: Yongqi Fan, Yuxiang Chu, Zhentao Xia, Xiaoyang Chen, Jie Liu, Haijin Liang, Jin Ma, Ben He, Yingfei Sun, Dezhi Ye, Tong Ruan,
• Abstract summary: We propose textbfRank4Gen, a generator-aware ranker for RAG that targets the goal of emphRanking for Generators.<n>We construct textbfPRISM, a dataset built from multiple open-source corpora and diverse downstream generators. Experiments on five challenging and recent RAG benchmarks demonstrate that RRank4Gen achieves strong and competitive performance for complex evidence composition in RAG.
• Score: 32.2695287857621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the RAG paradigm, the information retrieval module provides context for generators by retrieving and ranking multiple documents to support the aggregation of evidence. However, existing ranking models are primarily optimized for query--document relevance, which often misaligns with generators' preferences for evidence selection and citation, limiting their impact on response quality. Moreover, most approaches do not account for preference differences across generators, resulting in unstable cross-generator performance. We propose \textbf{Rank4Gen}, a generator-aware ranker for RAG that targets the goal of \emph{Ranking for Generators}. Rank4Gen introduces two key preference modeling strategies: (1) \textbf{From Ranking Relevance to Response Quality}, which optimizes ranking with respect to downstream response quality rather than query--document relevance; and (2) \textbf{Generator-Specific Preference Modeling}, which conditions a single ranker on different generators to capture their distinct ranking preferences. To enable such modeling, we construct \textbf{PRISM}, a dataset built from multiple open-source corpora and diverse downstream generators. Experiments on five challenging and recent RAG benchmarks demonstrate that RRank4Gen achieves strong and competitive performance for complex evidence composition in RAG.

Related papers

• SimGR: Escaping the Pitfalls of Generative Decoding in LLM-based Recommendation [68.00727783181289]
  A core objective in recommender systems is to accurately model the distribution of user preferences over items to enable personalized recommendations.<n>We observe that existing methods inevitably introduce systematic bias when estimating item-level preference distributions.<n>We propose textbfSimply textbfGenerative textbfRecommendation (textbfSimGR), a framework that directly models item-level preference distributions in a shared latent space.
  arXiv  Detail & Related papers  (2026-02-08T07:26:52Z)
• Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection [70.73201284835498]
  We introduce Routing with Generated Data (RGD), a challenging setting in which routers are trained exclusively on generated queries and answers.<n>We evaluate query-answer routers (using both queries and labels) and query-only routers across four diverse benchmarks and 12 models.<n>We propose CASCAL, a novel query-only router that estimates model correctness through consensus voting and identifies model-specific skill niches via hierarchical clustering.
  arXiv  Detail & Related papers  (2026-01-14T18:43:32Z)
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning [50.27838512822097]
  We introduce GlobalQA, the first benchmark specifically designed to evaluate global RAG capabilities.<n>We propose GlobalRAG, a multi-tool collaborative framework that preserves structural coherence through chunk-level retrieval.<n>On the Qwen2.5-14B model, GlobalRAG achieves 6.63 F1 compared to the strongest baseline's 1.51 F1.
  arXiv  Detail & Related papers  (2025-10-30T07:29:14Z)
• Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation [72.34977512403643]
  Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) by retrieving relevant documents from an external corpus.<n>Existing RAG systems primarily focus on unimodal text documents, and often fall short in real-world scenarios where both queries and documents may contain mixed modalities (such as text and images)<n>We propose Nyx, a unified mixed-modal to mixed-modal retriever tailored for Universal Retrieval-Augmented Generation scenarios.
  arXiv  Detail & Related papers  (2025-10-20T09:56:43Z)
• GoalRank: Group-Relative Optimization for a Large Ranking Model [28.848650157261385]
  We argue that there always exists a generator-only model that achieves strictly smaller approximation error to the optimal ranking policy.<n>We propose GoalRank, a generator-only ranking framework.
  arXiv  Detail & Related papers  (2025-09-26T08:32:16Z)
• ImpRAG: Retrieval-Augmented Generation with Implicit Queries [34.72864597562907]
  ImpRAG is a query-free RAG system that integrates retrieval and generation into a unified model.<n>We show that ImpRAG achieves 3.6-11.5 improvements in exact match scores on unseen tasks with diverse formats.
  arXiv  Detail & Related papers  (2025-06-02T21:38:21Z)
• Aligning Web Query Generation with Ranking Objectives via Direct Preference Optimization [21.140086066964667]
  We propose a framework that integrates ranking signals into the query generation process.<n> Experiments show higher ranker-assessed relevance between query-document pairs after DPO.
  arXiv  Detail & Related papers  (2025-05-25T20:34:12Z)
• MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation [34.66546005629471]
  Large Language Models (LLMs) are essential tools for various natural language processing tasks but often suffer from generating outdated or incorrect information.<n>Retrieval-Augmented Generation (RAG) addresses this issue by incorporating external, real-time information retrieval to ground LLM responses.<n>To tackle this problem, we propose Multi-Agent Filtering Retrieval-Augmented Generation (MAIN-RAG)<n>MAIN-RAG is a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents.
  arXiv  Detail & Related papers  (2024-12-31T08:07:26Z)
• PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization [35.48003039415176]
  Retrieval-augmented generation (RAG) has alleviated the issues of outdated and hallucinatory content in large language models (LLMs)<n>RAG generators often suffer from inadequate response informativeness, response robustness, and citation quality.<n>We propose Multiple Perspective Preference Alignment for Retrieval-Augmented Generation (PA-RAG) to align with RAG requirements comprehensively.
  arXiv  Detail & Related papers  (2024-12-19T04:18:51Z)
• WeQA: A Benchmark for Retrieval Augmented Generation in Wind Energy Domain [2.8514947506989707]
  Large Language Models (LLM) and Retrieval Augmented Generation (RAG) approaches offer a transformative solution.<n>In this paper, we present a comprehensive framework to generate a domain relevant RAG benchmark.<n>We demonstrate the framework by introducing WeQA, a first-of-its-kind benchmark on the wind energy domain.
  arXiv  Detail & Related papers  (2024-08-21T17:43:11Z)
• RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs [60.38044044203333]
  Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG) We propose a novel instruction fine-tuning framework RankRAG, which instruction-tunes a single LLM for the dual purpose of context ranking and answer generation in RAG. For generation, we compare our model with many strong baselines, including GPT-4-0613, GPT-4-turbo-2024-0409, and ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks.
  arXiv  Detail & Related papers  (2024-07-02T17:59:17Z)
• Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers [0.0]
  Retrieval-Augmented Generation (RAG) is a prevalent approach to infuse a private knowledge base of documents with Large Language Models (LLM) to build Generative Q&A (Question-Answering) systems. We propose the 'Blended RAG' method of leveraging semantic search techniques, such as Vector indexes and Sparse indexes, blended with hybrid query strategies. Our study achieves better retrieval results and sets new benchmarks for IR (Information Retrieval) datasets like NQ and TREC-COVID datasets.
  arXiv  Detail & Related papers  (2024-03-22T17:13:46Z)
• RAGGED: Towards Informed Design of Scalable and Stable RAG Systems [51.171355532527365]
  Retrieval-augmented generation (RAG) enhances language models by integrating external knowledge.<n>RAGGED is a framework for systematically evaluating RAG systems.
  arXiv  Detail & Related papers  (2024-03-14T02:26:31Z)
• Joint Generator-Ranker Learning for Natural Language Generation [99.16268050116717]
  JGR is a novel joint training algorithm that integrates the generator and the ranker in a single framework. By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly.
  arXiv  Detail & Related papers  (2022-06-28T12:58:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.