Auditing Yelp's Business Ranking and Review Recommendation Through the Lens of Fairness

• URL: http://arxiv.org/abs/2308.02129v2
• Date: Tue, 28 Jan 2025 23:31:59 GMT
• Title: Auditing Yelp's Business Ranking and Review Recommendation Through the Lens of Fairness
• Authors: Mohit Singhal, Javier Pacheco, Seyyed Mohammad Sadegh Moosavi Khorzooghi, Tanushree Debi, Abolfazl Asudeh, Gautam Das, Shirin Nilizadeh,
• Abstract summary: This study investigates the bias of Yelp's business ranking and review recommendation system. We find that reviews of less-established users are disproportionately categorized as not-recommended. We also find a positive association between restaurants' location in hotspot regions and their average exposure.
• Score: 10.550608462255056
• License:
• Abstract: Auditing is critical to ensuring the fairness and reliability of decision-making systems. However, auditing a black-box system for bias can be challenging due to the lack of transparency in the model's internal workings. In many web applications, such as Yelp, it is challenging, if not impossible, to manipulate their inputs systematically to identify bias in the output. Yelp connects users and businesses, where users identify new businesses and simultaneously express their experiences through reviews. Yelp recommendation software moderates user-provided content by categorizing it into recommended and not-recommended sections. The recommended reviews, among other attributes, are used by Yelp's ranking algorithm to rank businesses in a neighborhood. Due to Yelp's substantial popularity and its high impact on local businesses' success, understanding the bias of its algorithms is crucial. This data-driven study, for the first time, investigates the bias of Yelp's business ranking and review recommendation system. We examine three hypotheses to assess if Yelp's recommendation software shows bias against reviews of less established users with fewer friends and reviews and if Yelp's business ranking algorithm shows bias against restaurants located in specific neighborhoods, particularly in hotspot regions, with specific demographic compositions. Our findings show that reviews of less-established users are disproportionately categorized as not-recommended. We also find a positive association between restaurants' location in hotspot regions and their average exposure. Furthermore, we observed some cases of severe disparity bias in cities where the hotspots are in neighborhoods with less demographic diversity or higher affluence and education levels.

Related papers

• Reducing Popularity Influence by Addressing Position Bias [0.0]
  We show that position debiasing can effectively reduce a skew in the popularity of items induced by the position bias through a feedback loop. We show that position debiasing can significantly improve assortment utilization, without any degradation in user engagement or financial metrics.
  arXiv  Detail & Related papers  (2024-12-11T21:16:37Z)
• Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems [74.47680026838128]
  Two typical forms of bias in user interaction data with recommender systems (RSs) are popularity bias and positivity bias. We consider multifactorial selection bias affected by both item and rating value factors. We propose smoothing and alternating gradient descent techniques to reduce variance and improve the robustness of its optimization.
  arXiv  Detail & Related papers  (2024-04-29T12:18:21Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Whole Page Unbiased Learning to Rank [59.52040055543542]
  Unbiased Learning to Rank(ULTR) algorithms are proposed to learn an unbiased ranking model with biased click data. We propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model. Experimental results on a real-world dataset verify the effectiveness of the BAL.
  arXiv  Detail & Related papers  (2022-10-19T16:53:08Z)
• The Unfairness of Active Users and Popularity Bias in Point-of-Interest Recommendation [4.578469978594752]
  This paper studies the interplay between (i) the unfairness of active users, (ii) the unfairness of popular items, and (iii) the accuracy of recommendation as three angles of our study triangle. For item fairness, we divide items into short-head, mid-tail, and long-tail groups and study the exposure of these item groups into the top-k recommendation list of users. Our study shows that most recommendation models cannot satisfy both consumer and producer fairness, indicating a trade-off between these variables possibly due to natural biases in data.
  arXiv  Detail & Related papers  (2022-02-27T08:02:19Z)
• Reviews in motion: a large scale, longitudinal study of review recommendations on Yelp [24.34131115451651]
  We focus on "reclassification," wherein a platform changes its filtering decision for a review. We compile over 12.5M reviews--more than 2M unique--across over 10k businesses. Our data suggests demographic disparities in reclassifications, with more changes in lower density and low-middle income areas.
  arXiv  Detail & Related papers  (2022-02-18T03:27:53Z)
• "You eat with your eyes first": Optimizing Yelp Image Advertising [0.8594140167290099]
  We use Yelp's image dataset and star-based review system as a measurement of an image's effectiveness in promoting a business. We achieve 90-98% accuracy in classifying simplified star ratings for various image categories and observe that images containing blue skies, open surroundings, and many windows are correlated with higher Yelp reviews.
  arXiv  Detail & Related papers  (2020-11-03T02:49:40Z)
• DeepFair: Deep Learning for Improving Fairness in Recommender Systems [63.732639864601914]
  The lack of bias management in Recommender Systems leads to minority groups receiving unfair recommendations. We propose a Deep Learning based Collaborative Filtering algorithm that provides recommendations with an optimum balance between fairness and accuracy without knowing demographic information about the users.
  arXiv  Detail & Related papers  (2020-06-09T13:39:38Z)
• Fairness-Aware Explainable Recommendation over Knowledge Graphs [73.81994676695346]
  We analyze different groups of users according to their level of activity, and find that bias exists in recommendation performance between different groups. We show that inactive users may be more susceptible to receiving unsatisfactory recommendations, due to insufficient training data for the inactive users. We propose a fairness constrained approach via re-ranking to mitigate this problem in the context of explainable recommendation over knowledge graphs.
  arXiv  Detail & Related papers  (2020-06-03T05:04:38Z)
• Quarantine Deceiving Yelp's Users by Detecting Unreliable Rating Reviews [1.3999481573773074]
  We focus on quarantining Yelp's users that employ both review spike detection (RSD) algorithm and spam detection technique in bridging review networks (BRN) We found that more than 80% of Yelp's accounts are unreliable, and more than 80% of highly-rated businesses are subject to spamming.
  arXiv  Detail & Related papers  (2020-04-21T02:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.