Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness

• URL: http://arxiv.org/abs/2110.08353v1
• Date: Fri, 15 Oct 2021 20:30:51 GMT
• Title: Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
• Authors: Nicola Neophytou, Bhaskar Mitra and Catherine Stinson
• Abstract summary: We investigate how recommender performance varies according to popularity and demographics. We find statistically significant differences in recommender performance by both age and gender. We observe that recommendation utility steadily degrades for older users, and is lower for women than men.
• Score: 6.210698627561645
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recommendation algorithms are susceptible to popularity bias: a tendency to recommend popular items even when they fail to meet user needs. A related issue is that the recommendation quality can vary by demographic groups. Marginalized groups or groups that are under-represented in the training data may receive less relevant recommendations from these algorithms compared to others. In a recent study, Ekstrand et al. investigate how recommender performance varies according to popularity and demographics, and find statistically significant differences in recommendation utility between binary genders on two datasets, and significant effects based on age on one dataset. Here we reproduce those results and extend them with additional analyses. We find statistically significant differences in recommender performance by both age and gender. We observe that recommendation utility steadily degrades for older users, and is lower for women than men. We also find that the utility is higher for users from countries with more representation in the dataset. In addition, we find that total usage and the popularity of consumed content are strong predictors of recommender performance and also vary significantly across demographic groups.

Related papers

• ComPO: Community Preferences for Language Model Personalization [122.54846260663922]
  ComPO is a method to personalize preference optimization in language models. We collect and release ComPRed, a question answering dataset with community-level preferences from Reddit.
  arXiv  Detail & Related papers  (2024-10-21T14:02:40Z)
• A Study on Accuracy, Miscalibration, and Popularity Bias in Recommendations [6.694971161661218]
  We study how different genres affect the inconsistency of recommendation performance. We find that users with little interest in popular content receive the worst recommendation accuracy. Our experiments show that particular genres contribute to a different extent to the inconsistency of recommendation performance.
  arXiv  Detail & Related papers  (2023-03-01T10:39:58Z)
• Improving Recommendation Fairness via Data Augmentation [66.4071365614835]
  Collaborative filtering based recommendation learns users' preferences from all users' historical behavior data, and has been popular to facilitate decision making. A recommender system is considered unfair when it does not perform equally well for different user groups according to users' sensitive attributes. In this paper, we study how to improve recommendation fairness from the data augmentation perspective.
  arXiv  Detail & Related papers  (2023-02-13T13:11:46Z)
• Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness [15.210232622716129]
  Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. We show that some of the examples in the augmented dataset can be not important or even harmful for fairness.
  arXiv  Detail & Related papers  (2022-11-20T22:42:30Z)
• Equal Experience in Recommender Systems [21.298427869586686]
  We introduce a novel fairness notion (that we call equal experience) to regulate unfairness in the presence of biased data. We propose an optimization framework that incorporates the fairness notion as a regularization term, as well as introduce computationally-efficient algorithms that solve the optimization.
  arXiv  Detail & Related papers  (2022-10-12T05:53:05Z)
• Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling [66.02953670238647]
  Tail users suffer from significantly lower-quality recommendation than the head users after joint training. A model trained on tail users separately still achieve inferior results due to limited data. We propose a novel approach that significantly improves the recommendation performance of the tail users.
  arXiv  Detail & Related papers  (2022-08-19T02:50:19Z)
• Debiased Recommendation with Neural Stratification [19.841871819722016]
  We propose to cluster the users for computing more accurate IPS via increasing the exposure densities. We conduct extensive experiments based on real-world datasets to demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-08-15T15:45:35Z)
• Cross Pairwise Ranking for Unbiased Item Recommendation [57.71258289870123]
  We develop a new learning paradigm named Cross Pairwise Ranking (CPR) CPR achieves unbiased recommendation without knowing the exposure mechanism. We prove in theory that this way offsets the influence of user/item propensity on the learning.
  arXiv  Detail & Related papers  (2022-04-26T09:20:27Z)
• Predicting user demographics based on interest analysis [1.7403133838762448]
  This paper proposes a framework to predict users' demographic based on ratings registered by users in a system. Using all ratings registered by users improves the prediction accuracy by at least 16% compared with previously studied models.
  arXiv  Detail & Related papers  (2021-08-02T16:25:09Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)
• Enhancing Facial Data Diversity with Style-based Face Aging [59.984134070735934]
  In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. We propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns. We show that the proposed method outperforms state-of-the-art algorithms for age transfer.
  arXiv  Detail & Related papers  (2020-06-06T21:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.