Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems

• URL: http://arxiv.org/abs/2405.20626v1
• Date: Fri, 31 May 2024 05:31:00 GMT
• Title: Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems
• Authors: Shengyu Zhang, Ziqi Jiang, Jiangchao Yao, Fuli Feng, Kun Kuang, Zhou Zhao, Shuo Li, Hongxia Yang, Tat-Seng Chua, Fei Wu,
• Abstract summary: We show the uneven distribution of historical interactions and the biased training of recommender models. The key to debiased training lies in eliminating the effect of confounders that influence both the user's historical behaviors and the next behavior. We propose a causal multi-teacher distillation framework (CausalD) to address unobserved confounders.
• Score: 142.3424649008479
• License:
• Abstract: Recommendation performance usually exhibits a long-tail distribution over users -- a small portion of head users enjoy much more accurate recommendation services than the others. We reveal two sources of this performance heterogeneity problem: the uneven distribution of historical interactions (a natural source); and the biased training of recommender models (a model source). As addressing this problem cannot sacrifice the overall performance, a wise choice is to eliminate the model bias while maintaining the natural heterogeneity. The key to debiased training lies in eliminating the effect of confounders that influence both the user's historical behaviors and the next behavior. The emerging causal recommendation methods achieve this by modeling the causal effect between user behaviors, however potentially neglect unobserved confounders (\eg, friend suggestions) that are hard to measure in practice. To address unobserved confounders, we resort to the front-door adjustment (FDA) in causal theory and propose a causal multi-teacher distillation framework (CausalD). FDA requires proper mediators in order to estimate the causal effects of historical behaviors on the next behavior. To achieve this, we equip CausalD with multiple heterogeneous recommendation models to model the mediator distribution. Then, the causal effect estimated by FDA is the expectation of recommendation prediction over the mediator distribution and the prior distribution of historical behaviors, which is technically achieved by multi-teacher ensemble. To pursue efficient inference, CausalD further distills multiple teachers into one student model to directly infer the causal effect for making recommendations.

Related papers

• How Fair is Your Diffusion Recommender Model? [17.78188684065516]
  Diffusion-based recommender systems have recently proven to outperform traditional generative recommendation approaches. Machine learning literature has raised several concerns regarding the possibility that diffusion models may inadvertently carry information bias and lead to unfair outcomes. We conduct one of the first fairness investigations in the literature on DiffRec, a pioneer approach in diffusion-based recommendation.
  arXiv  Detail & Related papers  (2024-09-06T15:17:40Z)
• Be Aware of the Neighborhood Effect: Modeling Selection Bias under Interference [50.95521705711802]
  Previous studies have focused on addressing selection bias to achieve unbiased learning of the prediction model. This paper formally formulates the neighborhood effect as an interference problem from the perspective of causal inference. We propose a novel ideal loss that can be used to deal with selection bias in the presence of neighborhood effect.
  arXiv  Detail & Related papers  (2024-04-30T15:20:41Z)
• Improving Bias Mitigation through Bias Experts in Natural Language Understanding [10.363406065066538]
  We propose a new debiasing framework that introduces binary classifiers between the auxiliary model and the main model. Our proposed strategy improves the bias identification ability of the auxiliary model.
  arXiv  Detail & Related papers  (2023-12-06T16:15:00Z)
• Guide the Learner: Controlling Product of Experts Debiasing Method Based on Token Attribution Similarities [17.082695183953486]
  A popular workaround is to train a robust model by re-weighting training examples based on a secondary biased model. Here, the underlying assumption is that the biased model resorts to shortcut features. We introduce a fine-tuning strategy that incorporates the similarity between the main and biased model attribution scores in a Product of Experts loss function.
  arXiv  Detail & Related papers  (2023-02-06T15:21:41Z)
• Rethinking Missing Data: Aleatoric Uncertainty-Aware Recommendation [59.500347564280204]
  We propose a new Aleatoric Uncertainty-aware Recommendation (AUR) framework. AUR consists of a new uncertainty estimator along with a normal recommender model. As the chance of mislabeling reflects the potential of a pair, AUR makes recommendations according to the uncertainty.
  arXiv  Detail & Related papers  (2022-09-22T04:32:51Z)
• Cross Pairwise Ranking for Unbiased Item Recommendation [57.71258289870123]
  We develop a new learning paradigm named Cross Pairwise Ranking (CPR) CPR achieves unbiased recommendation without knowing the exposure mechanism. We prove in theory that this way offsets the influence of user/item propensity on the learning.
  arXiv  Detail & Related papers  (2022-04-26T09:20:27Z)
• Deep Causal Reasoning for Recommendations [47.83224399498504]
  A new trend in recommender system research is to negate the influence of confounders from a causal perspective. We model the recommendation as a multi-cause multi-outcome (MCMO) inference problem. We show that MCMO modeling may lead to high variance due to scarce observations associated with the high-dimensional causal space.
  arXiv  Detail & Related papers  (2022-01-06T15:00:01Z)
• Probabilistic and Variational Recommendation Denoising [56.879165033014026]
  Learning from implicit feedback is one of the most common cases in the application of recommender systems. We propose probabilistic and variational recommendation denoising for implicit feedback. We employ the proposed DPI and DVAE on four state-of-the-art recommendation models and conduct experiments on three datasets.
  arXiv  Detail & Related papers  (2021-05-20T08:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.