A Causal Information-Flow Framework for Unbiased Learning-to-Rank

• URL: http://arxiv.org/abs/2601.05590v1
• Date: Fri, 09 Jan 2026 07:19:35 GMT
• Title: A Causal Information-Flow Framework for Unbiased Learning-to-Rank
• Authors: Haoming Gong, Qingyao Ai, Zhihao Tao, Yongfeng Zhang,
• Abstract summary: In web search and recommendation systems, user clicks are widely used to train ranking models.<n>We introduce a novel causal learning-based ranking framework that extends Unbiased Learning-to-Rank.<n>Our method consistently reduces measured bias leakage and improves ranking performance.
• Score: 52.54102347581931
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In web search and recommendation systems, user clicks are widely used to train ranking models. However, click data is heavily biased, i.e., users tend to click higher-ranked items (position bias), choose only what was shown to them (selection bias), and trust top results more (trust bias). Without explicitly modeling these biases, the true relevance of ranked items cannot be correctly learned from clicks. Existing Unbiased Learning-to-Rank (ULTR) methods mainly correct position bias and rely on propensity estimation, but they cannot measure remaining bias, provide risk guarantees, or jointly handle multiple bias sources. To overcome these challenges, this paper introduces a novel causal learning-based ranking framework that extends ULTR by combining Structural Causal Models (SCMs) with information-theoretic tools. SCMs specify how clicks are generated and help identify the true relevance signal from click data, while conditional mutual information, measures how much bias leaks into the learned relevance estimates. We use this leakage measure to define a rigorous notion of disentanglement and include it as a regularizer during model training to reduce bias. In addition, we incorporate a causal inference estimator, i.e., doubly robust estimator, to ensure more reliable risk estimation. Experiments on standard Learning-to-Rank benchmarks show that our method consistently reduces measured bias leakage and improves ranking performance, especially in realistic scenarios where multiple biases-such as position and trust bias-interact strongly.

Related papers

• LLMs for estimating positional bias in logged interaction data [44.839172857330674]
  We propose a novel method for estimating position bias using Large Language Models (LLMs)<n>Our experiments show that propensities estimated with our LLM-as-a-judge approach are stable across score buckets.<n>An IPS-weighted reranker trained with these propensities matches the production model on standard NDCG@10 while improving weighted NDCG@10 by roughly 2%.
  arXiv  Detail & Related papers  (2025-09-03T20:26:06Z)
• Addressing Personalized Bias for Unbiased Learning to Rank [56.663619153713434]
  Unbiased learning to rank (ULTR) aims to learn unbiased ranking models from biased user behavior logs.<n>We propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives.
  arXiv  Detail & Related papers  (2025-08-28T14:01:31Z)
• Correcting for Position Bias in Learning to Rank: A Control Function Approach [9.986244291715762]
  We propose a novel control function-based method that accounts for position bias in a two-stage process.<n>Unlike previous position bias correction methods, our method does not require knowledge of the click or propensity model.<n> Experimental results demonstrate that our method outperforms state-of-the-art approaches in correcting for position bias.
  arXiv  Detail & Related papers  (2025-06-08T04:10:14Z)
• Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance [74.43264459255121]
  In real-world scenarios, users often click only one or two results after examining multiple relevant options.<n>We propose a query-level click propensity model to capture the probability that users will click on different result lists.<n>Our method introduces a Dual Inverse Propensity Weighting mechanism to address both relevance saturation and position bias.
  arXiv  Detail & Related papers  (2025-02-17T03:55:51Z)
• An Offline Metric for the Debiasedness of Click Models [52.25681483524383]
  Click models are a common method for extracting information from user clicks.<n>Recent work shows that the current evaluation practices in the community fail to guarantee that a well-performing click model generalizes well to downstream tasks.<n>We introduce the concept of debiasedness in click modeling and derive a metric for measuring it.
  arXiv  Detail & Related papers  (2023-04-19T10:59:34Z)
• Whole Page Unbiased Learning to Rank [59.52040055543542]
  Unbiased Learning to Rank(ULTR) algorithms are proposed to learn an unbiased ranking model with biased click data. We propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model. Experimental results on a real-world dataset verify the effectiveness of the BAL.
  arXiv  Detail & Related papers  (2022-10-19T16:53:08Z)
• Cross Pairwise Ranking for Unbiased Item Recommendation [57.71258289870123]
  We develop a new learning paradigm named Cross Pairwise Ranking (CPR) CPR achieves unbiased recommendation without knowing the exposure mechanism. We prove in theory that this way offsets the influence of user/item propensity on the learning.
  arXiv  Detail & Related papers  (2022-04-26T09:20:27Z)
• AutoDebias: Learning to Debias for Recommendation [43.84313723394282]
  We propose textitAotoDebias that leverages another (small) set of uniform data to optimize the debiasing parameters. We derive the generalization bound for AutoDebias and prove its ability to acquire the appropriate debiasing strategy.
  arXiv  Detail & Related papers  (2021-05-10T08:03:48Z)
• Handling Position Bias for Unbiased Learning to Rank in Hotels Search [0.951828574518325]
  We will investigate the importance of properly handling the position bias in an online test environment in Tripadvisor Hotels search. We propose an empirically effective method of handling the position bias that fully leverages the user action data. The online A/B test results show that this method leads to an improved search ranking model.
  arXiv  Detail & Related papers  (2020-02-28T03:48:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.