Addressing Personalized Bias for Unbiased Learning to Rank

• URL: http://arxiv.org/abs/2508.20798v1
• Date: Thu, 28 Aug 2025 14:01:31 GMT
• Title: Addressing Personalized Bias for Unbiased Learning to Rank
• Authors: Zechun Niu, Lang Mei, Liu Yang, Ziyuan Zhao, Qiang Yan, Jiaxin Mao, Ji-Rong Wen,
• Abstract summary: Unbiased learning to rank (ULTR) aims to learn unbiased ranking models from biased user behavior logs.<n>We propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives.
• Score: 56.663619153713434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unbiased learning to rank (ULTR), which aims to learn unbiased ranking models from biased user behavior logs, plays an important role in Web search. Previous research on ULTR has studied a variety of biases in users' clicks, such as position bias, presentation bias, and outlier bias. However, existing work often assumes that the behavior logs are collected from an ``average'' user, neglecting the differences between different users in their search and browsing behaviors. In this paper, we introduce personalized factors into the ULTR framework, which we term the user-aware ULTR problem. Through a formal causal analysis of this problem, we demonstrate that existing user-oblivious methods are biased when different users have different preferences over queries and personalized propensities of examining documents. To address such a personalized bias, we propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives. Specifically, our approach models the distribution of user browsing behaviors for each query and aggregates user-weighted examination probabilities to determine propensities. We theoretically prove that the user-aware estimator is unbiased under some mild assumptions and shows lower variance compared to the straightforward way of calculating a user-dependent propensity for each impression. Finally, we empirically verify the effectiveness of our user-aware estimator by conducting extensive experiments on two semi-synthetic datasets and a real-world dataset.

Related papers

• A Causal Information-Flow Framework for Unbiased Learning-to-Rank [52.54102347581931]
  In web search and recommendation systems, user clicks are widely used to train ranking models.<n>We introduce a novel causal learning-based ranking framework that extends Unbiased Learning-to-Rank.<n>Our method consistently reduces measured bias leakage and improves ranking performance.
  arXiv  Detail & Related papers  (2026-01-09T07:19:35Z)
• User-Assistant Bias in LLMs [11.825607435336776]
  Large language models (LLMs) can bias towards relying on their own or the user's information in chat history, leading to overly stubborn or agreeable behaviors in multi-turn conversations.<n>We introduce an 8k multi-turn conversation dataset $textbfUserAssist$ to benchmark, understand and manipulate the user-assistant bias in frontier LLMs.
  arXiv  Detail & Related papers  (2025-08-16T20:33:09Z)
• Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance [74.43264459255121]
  In real-world scenarios, users often click only one or two results after examining multiple relevant options.<n>We propose a query-level click propensity model to capture the probability that users will click on different result lists.<n>Our method introduces a Dual Inverse Propensity Weighting mechanism to address both relevance saturation and position bias.
  arXiv  Detail & Related papers  (2025-02-17T03:55:51Z)
• Whole Page Unbiased Learning to Rank [59.52040055543542]
  Unbiased Learning to Rank(ULTR) algorithms are proposed to learn an unbiased ranking model with biased click data. We propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model. Experimental results on a real-world dataset verify the effectiveness of the BAL.
  arXiv  Detail & Related papers  (2022-10-19T16:53:08Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Cross Pairwise Ranking for Unbiased Item Recommendation [57.71258289870123]
  We develop a new learning paradigm named Cross Pairwise Ranking (CPR) CPR achieves unbiased recommendation without knowing the exposure mechanism. We prove in theory that this way offsets the influence of user/item propensity on the learning.
  arXiv  Detail & Related papers  (2022-04-26T09:20:27Z)
• Handling Position Bias for Unbiased Learning to Rank in Hotels Search [0.951828574518325]
  We will investigate the importance of properly handling the position bias in an online test environment in Tripadvisor Hotels search. We propose an empirically effective method of handling the position bias that fully leverages the user action data. The online A/B test results show that this method leads to an improved search ranking model.
  arXiv  Detail & Related papers  (2020-02-28T03:48:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.