User-centered Evaluation of Popularity Bias in Recommender Systems

• URL: http://arxiv.org/abs/2103.06364v1
• Date: Wed, 10 Mar 2021 22:12:51 GMT
• Title: User-centered Evaluation of Popularity Bias in Recommender Systems
• Authors: Himan Abdollahpouri, Masoud Mansoury, Robin Burke, Bamshad Mobasher, Edward Malthouse
• Abstract summary: Recommendation and ranking systems suffer from popularity bias; the tendency of the algorithm to favor a few popular items while under-representing the majority of other items. In this paper, we show the limitations of the existing metrics to evaluate popularity bias mitigation when we want to assess these algorithms from the users' perspective. We present an effective approach that mitigates popularity bias from the user-centered point of view.
• Score: 4.30484058393522
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recommendation and ranking systems are known to suffer from popularity bias; the tendency of the algorithm to favor a few popular items while under-representing the majority of other items. Prior research has examined various approaches for mitigating popularity bias and enhancing the recommendation of long-tail, less popular, items. The effectiveness of these approaches is often assessed using different metrics to evaluate the extent to which over-concentration on popular items is reduced. However, not much attention has been given to the user-centered evaluation of this bias; how different users with different levels of interest towards popular items are affected by such algorithms. In this paper, we show the limitations of the existing metrics to evaluate popularity bias mitigation when we want to assess these algorithms from the users' perspective and we propose a new metric that can address these limitations. In addition, we present an effective approach that mitigates popularity bias from the user-centered point of view. Finally, we investigate several state-of-the-art approaches proposed in recent years to mitigate popularity bias and evaluate their performances using the existing metrics and also from the users' perspective. Our experimental results using two publicly-available datasets show that existing popularity bias mitigation techniques ignore the users' tolerance towards popular items. Our proposed user-centered method can tackle popularity bias effectively for different users while also improving the existing metrics.

Related papers

• Large Language Models as Recommender Systems: A Study of Popularity Bias [46.17953988777199]
  Popular items are disproportionately recommended, overshadowing less popular but potentially relevant items. Recent advancements have seen the integration of general-purpose Large Language Models into recommender systems. Our study explores whether LLMs contribute to or can alleviate popularity bias in recommender systems.
  arXiv  Detail & Related papers  (2024-06-03T12:53:37Z)
• Metrics for popularity bias in dynamic recommender systems [0.0]
  Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed.
  arXiv  Detail & Related papers  (2023-10-12T16:15:30Z)
• Test Time Embedding Normalization for Popularity Bias Mitigation [6.145760252113906]
  Popularity bias is a widespread problem in the field of recommender systems. We propose 'Test Time Embedding Normalization' as a simple yet effective strategy for mitigating popularity bias.
  arXiv  Detail & Related papers  (2023-08-22T08:57:44Z)
• A Survey on Popularity Bias in Recommender Systems [5.952279576277445]
  We discuss the potential reasons for popularity bias and review existing approaches to detect, mitigate and quantify popularity bias in recommender systems. We critically discuss todays literature, where we observe that the research is almost entirely based on computational experiments and on certain assumptions regarding the practical effects of including long-tail items in the recommendations.
  arXiv  Detail & Related papers  (2023-08-02T12:58:11Z)
• Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling [66.02953670238647]
  Tail users suffer from significantly lower-quality recommendation than the head users after joint training. A model trained on tail users separately still achieve inferior results due to limited data. We propose a novel approach that significantly improves the recommendation performance of the tail users.
  arXiv  Detail & Related papers  (2022-08-19T02:50:19Z)
• Reconciling the Quality vs Popularity Dichotomy in Online Cultural Markets [62.146882023375746]
  We propose a model of an idealized online cultural market in which $N$ items, endowed with a hidden quality metric, are recommended to users by a ranking algorithm possibly biased by the current items' popularity. Our goal is to better understand the underlying mechanisms of the well-known fact that popularity bias can prevent higher-quality items from becoming more popular than lower-quality items, producing an undesirable misalignment between quality and popularity rankings.
  arXiv  Detail & Related papers  (2022-04-28T14:36:11Z)
• The Unfairness of Popularity Bias in Book Recommendation [0.0]
  Popularity bias refers to the problem that popular items are recommended frequently while less popular items are recommended rarely or not at all. We analyze the well-known Book-Crossing dataset and define three user groups based on their tendency towards popular items. Our results indicate that most state-of-the-art recommendation algorithms suffer from popularity bias in the book domain.
  arXiv  Detail & Related papers  (2022-02-27T20:21:46Z)
• An Adaptive Boosting Technique to Mitigate Popularity Bias in Recommender System [1.5800354337004194]
  A typical accuracy measure is biased towards popular items, i.e., it promotes better accuracy for popular items compared to non-popular items. This paper considers a metric that measures the popularity bias as the difference in error on popular items and non-popular items. Motivated by the fair boosting algorithm on classification, we propose an algorithm that reduces the popularity bias present in the data.
  arXiv  Detail & Related papers  (2021-09-13T03:04:55Z)
• Estimation of Fair Ranking Metrics with Incomplete Judgments [70.37717864975387]
  We propose a sampling strategy and estimation technique for four fair ranking metrics. We formulate a robust and unbiased estimator which can operate even with very limited number of labeled items.
  arXiv  Detail & Related papers  (2021-08-11T10:57:00Z)
• PURS: Personalized Unexpected Recommender System for Improving User Satisfaction [76.98616102965023]
  We describe a novel Personalized Unexpected Recommender System (PURS) model that incorporates unexpectedness into the recommendation process. Extensive offline experiments on three real-world datasets illustrate that the proposed PURS model significantly outperforms the state-of-the-art baseline approaches.
  arXiv  Detail & Related papers  (2021-06-05T01:33:21Z)
• Partial Bandit and Semi-Bandit: Making the Most Out of Scarce Users' Feedback [62.997667081978825]
  We present a novel approach for considering user feedback and evaluate it using three distinct strategies. Despite a limited number of feedbacks returned by users (as low as 20% of the total), our approach obtains similar results to those of state of the art approaches.
  arXiv  Detail & Related papers  (2020-09-16T07:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.