RLPO: Residual Listwise Preference Optimization for Long-Context Review Ranking

• URL: http://arxiv.org/abs/2601.07449v1
• Date: Mon, 12 Jan 2026 11:45:19 GMT
• Title: RLPO: Residual Listwise Preference Optimization for Long-Context Review Ranking
• Authors: Hao Jiang, Zhi Yang, Annan Wang, Yichi Zhang, Weisi Lin,
• Abstract summary: Pointwise scoring is efficient but often fails to account for list-level interactions.<n>Listwise approaches can leverage global context, yet they are computationally expensive and become unstable as candidate lists grow.<n>We propose Residual Listwise Preference Optimization (RLPO), which formulates ranking as listwise representation-level residual correction.
• Score: 50.709454968853954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Review ranking is pivotal in e-commerce for prioritizing diagnostic and authentic feedback from the deluge of user-generated content. While large language models have improved semantic assessment, existing ranking paradigms face a persistent trade-off in long-context settings. Pointwise scoring is efficient but often fails to account for list-level interactions, leading to miscalibrated top-$k$ rankings. Listwise approaches can leverage global context, yet they are computationally expensive and become unstable as candidate lists grow. To address this, we propose Residual Listwise Preference Optimization (RLPO), which formulates ranking as listwise representation-level residual correction over a strong pointwise LLM scorer. RLPO first produces calibrated pointwise scores and item representations, then applies a lightweight encoder over the representations to predict listwise score residuals, avoiding full token-level listwise processing. We also introduce a large-scale benchmark for long-context review ranking with human verification. Experiments show RLPO improves NDCG@k over strong pointwise and listwise baselines and remains robust as list length increases.

Related papers

• K-order Ranking Preference Optimization for Large Language Models [43.249451587412544]
  We argue that optimizing top-K ranking consistency could be more appropriate for real-world applications.<n>We propose K-order Ranking Preference Optimization (KPO) by extending the DPO's Plackett-Luce model to accommodate top-K rankings.
  arXiv  Detail & Related papers  (2025-05-31T07:46:42Z)
• AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking [35.685682379377134]
  Listwise reranking with large language models (LLMs) enhances top-ranked results in retrieval-based applications.<n>We propose AcuRank, an adaptive reranking framework that dynamically adjusts both the amount and target of computation based on uncertainty estimates over document relevance.<n>Results on the TREC-DL and BEIR benchmarks show that our method consistently achieves a superior accuracy-efficiency trade-off and scales better with compute than fixed-computation baselines.
  arXiv  Detail & Related papers  (2025-05-24T05:15:49Z)
• In-context Ranking Preference Optimization [65.5489745857577]
  We propose an In-context Ranking Preference Optimization (IRPO) framework to optimize large language models (LLMs) based on ranking lists constructed during inference.<n>We show IRPO outperforms standard DPO approaches in ranking performance, highlighting its effectiveness in aligning LLMs with direct in-context ranking preferences.
  arXiv  Detail & Related papers  (2025-04-21T23:06:12Z)
• Self-Calibrated Listwise Reranking with Large Language Models [137.6557607279876]
  Large language models (LLMs) have been employed in reranking tasks through a sequence-to-sequence approach. This reranking paradigm requires a sliding window strategy to iteratively handle larger candidate sets. We propose a novel self-calibrated listwise reranking method, which aims to leverage LLMs to produce global relevance scores for ranking.
  arXiv  Detail & Related papers  (2024-11-07T10:31:31Z)
• FIRST: Faster Improved Listwise Reranking with Single Token Decoding [56.727761901751194]
  First, we introduce FIRST, a novel listwise LLM reranking approach leveraging the output logits of the first generated identifier to directly obtain a ranked ordering of the candidates. Empirical results demonstrate that FIRST accelerates inference by 50% while maintaining a robust ranking performance with gains across the BEIR benchmark. Our results show that LLM rerankers can provide a stronger distillation signal compared to cross-encoders, yielding substantial improvements in retriever recall after relevance feedback.
  arXiv  Detail & Related papers  (2024-06-21T21:27:50Z)
• LiPO: Listwise Preference Optimization through Learning-to-Rank [62.02782819559389]
  Policy can learn more effectively from a ranked list of plausible responses given the prompt.<n>We show that LiPO-$lambda$ can outperform DPO variants and SLiC by a clear margin on several preference alignment tasks.
  arXiv  Detail & Related papers  (2024-02-02T20:08:10Z)
• Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models [63.714662435555674]
  Large language models (LLMs) exhibit positional bias in how they use context. We propose permutation self-consistency, a form of self-consistency over ranking list outputs of black-box LLMs. Our approach improves scores from conventional inference by up to 7-18% for GPT-3.5 and 8-16% for LLaMA v2 (70B)
  arXiv  Detail & Related papers  (2023-10-11T17:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.