LLM-based Listwise Reranking under the Effect of Positional Bias

• URL: http://arxiv.org/abs/2604.03642v1
• Date: Sat, 04 Apr 2026 08:40:57 GMT
• Title: LLM-based Listwise Reranking under the Effect of Positional Bias
• Abstract summary: LLM-based listwise passage reranking has attracted attention for its effectiveness in ranking candidate passages.<n>These models suffer from positional bias, where passages positioned towards the end of the input are less likely to be moved to top positions in the ranking.<n>We propose DebiasFirst, a method that integrates positional calibration and position-aware data augmentation during fine-tuning.
• Score: 69.07550988151473
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-based listwise passage reranking has attracted attention for its effectiveness in ranking candidate passages. However, these models suffer from positional bias, where passages positioned towards the end of the input are less likely to be moved to top positions in the ranking. We hypothesize that there are two primary sources of positional bias: (1) architectural bias inherent in LLMs and (2) the imbalanced positioning of relevant documents. To address this, we propose DebiasFirst, a method that integrates positional calibration and position-aware data augmentation during fine-tuning. Positional calibration uses inverse propensity scoring to adjust for positional bias by re-weighting the contributions of different positions in the loss function when training. Position-aware augmentation augments training data to ensure that each passage appears equally across varied positions in the input list. This approach markedly enhances both effectiveness and robustness to the original ranking across diverse first-stage retrievers, reducing the dependence of NDCG@10 performance on the position of relevant documents. DebiasFirst also complements the inference-stage debiasing methods, offering a practical solution for mitigating positional bias in reranking.

Related papers

• Is Position Bias in Dense Retrievers Built In-or Learned from Data? [3.331218753253791]
  We study how the positional distribution of evidence in training data affects retrieval-level bias direction.<n>Position-balanced training reduces positional sensitivity by 57--87% on position-aware benchmarks.<n>These results identify training-position distribution as a major controllable factor in retrieval-level position bias.
  arXiv  Detail & Related papers  (2026-05-26T05:50:16Z)
• Correcting for Position Bias in Learning to Rank: A Control Function Approach [9.986244291715762]
  We propose a novel control function-based method that accounts for position bias in a two-stage process.<n>Unlike previous position bias correction methods, our method does not require knowledge of the click or propensity model.<n> Experimental results demonstrate that our method outperforms state-of-the-art approaches in correcting for position bias.
  arXiv  Detail & Related papers  (2025-06-08T04:10:14Z)
• In-context Ranking Preference Optimization [65.5489745857577]
  We propose an In-context Ranking Preference Optimization (IRPO) framework to optimize large language models (LLMs) based on ranking lists constructed during inference.<n>We show IRPO outperforms standard DPO approaches in ranking performance, highlighting its effectiveness in aligning LLMs with direct in-context ranking preferences.
  arXiv  Detail & Related papers  (2025-04-21T23:06:12Z)
• Eliminating Position Bias of Language Models: A Mechanistic Approach [119.34143323054143]
  Position bias has proven to be a prevalent issue of modern language models (LMs)<n>Our mechanistic analysis attributes the position bias to two components employed in nearly all state-of-the-art LMs: causal attention and relative positional encodings.<n>By eliminating position bias, models achieve better performance and reliability in downstream tasks, including LM-as-a-judge, retrieval-augmented QA, molecule generation, and math reasoning.
  arXiv  Detail & Related papers  (2024-07-01T09:06:57Z)
• Mitigate Position Bias with Coupled Ranking Bias on CTR Prediction [40.9668016608627]
  Most existing methods ignore the widely coupled ranking bias, which is also related to the placing position of the item. We show how this widely coexisted ranking bias deteriorates the performance of the existing position bias estimation methods. To mitigate the position bias with the presence of the ranking bias, we propose a novel position bias estimation method, namely gradient, which fuses two estimation methods using a fusing weight.
  arXiv  Detail & Related papers  (2024-05-29T10:31:53Z)
• Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs [18.832135309689736]
  Recent advances in large language models (LLMs) have enhanced their ability to process long input contexts. Recent studies show a positional bias in LLMs, demonstrating varying performance depending on the location of useful information. We develop a Position-Aware PAPEFT approach which is composed of a data augmentation technique and an efficient parameter adapter.
  arXiv  Detail & Related papers  (2024-04-01T19:04:17Z)
• Click-Conversion Multi-Task Model with Position Bias Mitigation for Sponsored Search in eCommerce [51.211924408864355]
  We propose two position-bias-free prediction models: Position-Aware Click-Conversion (PACC) and PACC via Position Embedding (PACC-PE) Experiments on the E-commerce sponsored product search dataset show that our proposed models have better ranking effectiveness and can greatly alleviate position bias in both CTR and CVR prediction.
  arXiv  Detail & Related papers  (2023-07-29T19:41:16Z)
• Learning List-Level Domain-Invariant Representations for Ranking [59.3544317373004]
  We propose list-level alignment -- learning domain-invariant representations at the higher level of lists. The benefits are twofold: it leads to the first domain adaptation generalization bound for ranking, in turn providing theoretical support for the proposed method.
  arXiv  Detail & Related papers  (2022-12-21T04:49:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.