Improving Zero-shot LLM Re-Ranker with Risk Minimization

• URL: http://arxiv.org/abs/2406.13331v2
• Date: Fri, 20 Dec 2024 04:59:10 GMT
• Title: Improving Zero-shot LLM Re-Ranker with Risk Minimization
• Abstract summary: In the Retrieval-Augmented Generation (RAG) system, advanced Large Language Models (LLMs) have emerged as effective Query Likelihood Models (QLMs) in an unsupervised way.<n>However, directly prompting LLMs to approximate QLMs inherently is biased, where the estimated distribution might diverge from the actual document-specific distribution.<n>We introduce a novel framework, $mathrmUR3$, which leverages Bayesian decision theory to both quantify and mitigate this estimation bias.
• Score: 20.32406191251512
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the Retrieval-Augmented Generation (RAG) system, advanced Large Language Models (LLMs) have emerged as effective Query Likelihood Models (QLMs) in an unsupervised way, which re-rank documents based on the probability of generating the query given the content of a document. However, directly prompting LLMs to approximate QLMs inherently is biased, where the estimated distribution might diverge from the actual document-specific distribution. In this study, we introduce a novel framework, $\mathrm{UR^3}$, which leverages Bayesian decision theory to both quantify and mitigate this estimation bias. Specifically, $\mathrm{UR^3}$ reformulates the problem as maximizing the probability of document generation, thereby harmonizing the optimization of query and document generation probabilities under a unified risk minimization objective. Our empirical results indicate that $\mathrm{UR^3}$ significantly enhances re-ranking, particularly in improving the Top-1 accuracy. It benefits the QA tasks by achieving higher accuracy with fewer input documents.

Related papers

• Conformal Reliability: A New Evaluation Metric for Conditional Generation [62.761166941396844]
  Conditional generative models have recently remarkable success in various applications.<n>We propose a novel evaluation metric called reliability based on prediction set at a pre-specified confidence level.<n>We introduce Conformal ReLiability (CReL), a framework that can (i) construct a prediction set and (ii) accurately optimize the reliability score within the constructed prediction set.
  arXiv  Detail & Related papers  (2026-05-29T03:52:44Z)
• Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking [0.20305676256390937]
  This paper decomposes uncertainty into its sources, distinguishes variance that shrinks with more data, and uses design-study projections to reduce total error.<n>Applying the approach to ideology annotation, safety classification, MMLU benchmarking, and a human-validated propaganda audit reveals different dominant variance sources by domain and scoring method.
  arXiv  Detail & Related papers  (2026-04-13T14:58:15Z)
• HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy [54.91468501159335]
  Cross-document relation extraction (RE) aims to identify relations between the head tail entities located in different documents.<n>We propose an inference strategy that improves reliability through multi-view verification at each level.
  arXiv  Detail & Related papers  (2026-04-09T07:55:27Z)
• LLM-as-Judge on a Budget [35.393598355979385]
  We present a principled variance-adaptive approach leveraging multi-armed bandit theory and concentration inequalities.<n>Our algorithm is shown to achieve a worst-case score-estimation error of $tildeOleft(sqrtfracsum_i=1K_i2Bright)$.<n>Experiments on emphSummarize-From-Feedback and emphHelpSteer2 demonstrate that our method significantly outperforms uniform allocation.
  arXiv  Detail & Related papers  (2026-02-17T10:35:41Z)
• SimGR: Escaping the Pitfalls of Generative Decoding in LLM-based Recommendation [68.00727783181289]
  A core objective in recommender systems is to accurately model the distribution of user preferences over items to enable personalized recommendations.<n>We observe that existing methods inevitably introduce systematic bias when estimating item-level preference distributions.<n>We propose textbfSimply textbfGenerative textbfRecommendation (textbfSimGR), a framework that directly models item-level preference distributions in a shared latent space.
  arXiv  Detail & Related papers  (2026-02-08T07:26:52Z)
• ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking [8.244386008877441]
  We introduce a novel two-stage training approach, ProRank, for SLM-based document reranking.<n>First, we propose a prompt warmup stage using reinforcement learning GRPO to steer SLMs to understand task prompts.<n>Then, we continuously fine-tune the SLMs with a fine-grained score learning stage without introducing additional layers to further improve the reranking quality.
  arXiv  Detail & Related papers  (2025-06-04T02:00:44Z)
• Optimal Policy Minimum Bayesian Risk [25.434911234706952]
  We present a novel method for incorporating reward and risk/similarity signals into Bayes risk decoding (MBRD)<n>Based on the concept of optimal policy in KL-controlled reinforcement learning, our framework provides a simple and well-defined mechanism for leveraging such signals.<n>It offers several advantages over traditional inference-time methods: higher robustness, improved accuracy, and well-understood behavior.
  arXiv  Detail & Related papers  (2025-05-22T19:43:37Z)
• Likelihood-Ratio Regularized Quantile Regression: Adapting Conformal Prediction to High-Dimensional Covariate Shifts [35.16750653336608]
  We introduce the likelihood ratio regularized quantile regression algorithm, which combines the pinball loss with a novel choice of regularization. We show that the LR-QR method has coverage at the desired level in the target domain, up to a small error term. Our experiments demonstrate that the LR-QR algorithm outperforms existing methods on high-dimensional prediction tasks.
  arXiv  Detail & Related papers  (2025-02-18T16:46:44Z)
• Inference Scaling for Bridging Retrieval and Augmented Generation [47.091086803980765]
  Retrieval-augmented generation (RAG) has emerged as a popular approach to steering the output of a large language model (LLM) We show such bias can be mitigated, from inference scaling, aggregating inference calls from the permuted order of retrieved contexts. We showcase the effectiveness of MOI on diverse RAG tasks, improving ROUGE-L on MS MARCO and EM on HotpotQA benchmarks by 7 points.
  arXiv  Detail & Related papers  (2024-12-14T05:06:43Z)
• Calibrating Verbalized Probabilities for Large Language Models [7.586709509332433]
  Calibrating verbalized probabilities presents a novel approach for reliably assessing and leveraging outputs from black-box Large Language Models. Recent methods have demonstrated improved calibration by applying techniques like Platt scaling or temperature scaling to the confidence scores generated by LLMs.
  arXiv  Detail & Related papers  (2024-10-09T09:20:24Z)
• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models [48.96686419141881]
  We introduce the first formal probabilistic evaluation framework for Large Language Models (LLMs) Namely, we propose novel metrics with high probability guarantees concerning the output distribution of a model. Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment.
  arXiv  Detail & Related papers  (2024-10-04T15:44:23Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models [52.46248487458641]
  Predictive models often need to work with incomplete information in real-world tasks. Current large language models (LLM) are insufficient for such accurate estimations. We propose BIRD, a novel probabilistic inference framework.
  arXiv  Detail & Related papers  (2024-04-18T20:17:23Z)
• Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shift [34.414261291690856]
  Key challenge of modern machine learning systems is to achieve Out-of-Distribution (OOD) generalization. We show that classical Maximum Likelihood Estimation (MLE) purely using source data achieves the minimax optimality. We illustrate the wide applicability of our framework by instantiating it to three concrete examples.
  arXiv  Detail & Related papers  (2023-11-27T16:06:48Z)
• Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking [36.90911173089409]
  Large language models (LLMs) have emerged as effective Query Likelihood Models (QLMs) This paper focuses on investigating the genuine zero-shot ranking effectiveness of recent LLMs. We introduce a novel state-of-the-art ranking system that integrates LLM-based QLMs with a hybrid zero-shot retriever.
  arXiv  Detail & Related papers  (2023-10-20T02:54:42Z)
• Minimum-Risk Recalibration of Classifiers [9.31067660373791]
  We introduce the concept of minimum-risk recalibration within the framework of mean-squared-error decomposition. We show that transferring a calibrated classifier requires significantly fewer target samples compared to recalibrating from scratch.
  arXiv  Detail & Related papers  (2023-05-18T11:27:02Z)
• Probable Domain Generalization via Quantile Risk Minimization [90.15831047587302]
  Domain generalization seeks predictors which perform well on unseen test distributions. We propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability.
  arXiv  Detail & Related papers  (2022-07-20T14:41:09Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• A Multi-Document Coverage Reward for RELAXed Multi-Document Summarization [11.02198476454955]
  We propose fine-tuning an MDS baseline with a reward that balances a reference-based metric with coverage of the input documents. Experimental results over the Multi-News and WCEP MDS datasets show significant improvements of up to +0.95 pp average ROUGE score and +3.17 pp METEOR score over the baseline.
  arXiv  Detail & Related papers  (2022-03-06T07:33:01Z)
• Navigating to the Best Policy in Markov Decision Processes [68.8204255655161]
  We investigate the active pure exploration problem in Markov Decision Processes. Agent sequentially selects actions and, from the resulting system trajectory, aims at the best as fast as possible.
  arXiv  Detail & Related papers  (2021-06-05T09:16:28Z)
• Distributionally Robust Bayesian Quadrature Optimization [60.383252534861136]
  We study BQO under distributional uncertainty in which the underlying probability distribution is unknown except for a limited set of its i.i.d. samples. A standard BQO approach maximizes the Monte Carlo estimate of the true expected objective given the fixed sample set. We propose a novel posterior sampling based algorithm, namely distributionally robust BQO (DRBQO) for this purpose.
  arXiv  Detail & Related papers  (2020-01-19T12:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.