Reviews in motion: a large scale, longitudinal study of review recommendations on Yelp

• URL: http://arxiv.org/abs/2202.09005v1
• Date: Fri, 18 Feb 2022 03:27:53 GMT
• Title: Reviews in motion: a large scale, longitudinal study of review recommendations on Yelp
• Authors: Ryan Amos, Roland Maio, Prateek Mittal
• Abstract summary: We focus on "reclassification," wherein a platform changes its filtering decision for a review. We compile over 12.5M reviews--more than 2M unique--across over 10k businesses. Our data suggests demographic disparities in reclassifications, with more changes in lower density and low-middle income areas.
• Score: 24.34131115451651
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The United Nations Consumer Protection Guidelines lists "access ... to adequate information ... to make informed choices" as a core consumer protection right. However, problematic online reviews and imperfections in algorithms that detect those reviews pose obstacles to the fulfillment of this right. Research on reviews and review platforms often derives insights from a single web crawl, but the decisions those crawls observe may not be static. A platform may feature a review one day and filter it from view the next day. An appreciation for these dynamics is necessary to understand how a platform chooses which reviews consumers encounter and which reviews may be unhelpful or suspicious. We introduce a novel longitudinal angle to the study of reviews. We focus on "reclassification," wherein a platform changes its filtering decision for a review. To that end, we perform repeated web crawls of Yelp to create three longitudinal datasets. These datasets highlight the platform's dynamic treatment of reviews. We compile over 12.5M reviews--more than 2M unique--across over 10k businesses. Our datasets are available for researchers to use. Our longitudinal approach gives us a unique perspective on Yelp's classifier and allows us to explore reclassification. We find that reviews routinely move between Yelp's two main classifier classes ("Recommended" and "Not Recommended")--up to 8% over eight years--raising concerns about prior works' use of Yelp's classes as ground truth. These changes have impacts on small scales; for example, a business going from a 3.5 to 4.5 star rating despite no new reviews. Some reviews move multiple times: we observed up to five reclassifications in eleven months. Our data suggests demographic disparities in reclassifications, with more changes in lower density and low-middle income areas.

Related papers

• Auditing Yelp's Business Ranking and Review Recommendation Through the Lens of Fairness [10.957942355264093]
  This study investigates Yelp's business ranking and review recommendation system through the lens of fairness. We find that reviews of female and less-established users are disproportionately categorized as recommended. We also find a positive association between restaurants being located in hotspot regions and their average exposure.
  arXiv  Detail & Related papers  (2023-08-04T04:12:33Z)
• Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media [77.34726150561087]
  We propose to leverage the high-quality elite Yelp reviews to generate fake reviews from the OpenAI GPT review creator. We apply the model to predict non-elite reviews and identify the patterns across several dimensions. We show that social media platforms are continuously challenged by machine-generated fake reviews.
  arXiv  Detail & Related papers  (2023-02-10T19:40:10Z)
• Integrating Rankings into Quantized Scores in Peer Review [61.27794774537103]
  In peer review, reviewers are usually asked to provide scores for the papers. To mitigate this issue, conferences have started to ask reviewers to additionally provide a ranking of the papers they have reviewed. There are no standard procedure for using this ranking information and Area Chairs may use it in different ways. We take a principled approach to integrate the ranking information into the scores.
  arXiv  Detail & Related papers  (2022-04-05T19:39:13Z)
• Large Scale Analysis of Open MOOC Reviews to Support Learners' Course Selection [17.376856503445826]
  We analyze 2.4 million reviews (which is the largest MOOC reviews dataset used until now) from five different platforms. Results show that numeric ratings are clearly biased (63% of them are 5-star ratings) We expect our study to shed some light on the area and promote a more transparent approach in online education reviews.
  arXiv  Detail & Related papers  (2022-01-11T10:24:49Z)
• Attention-based Bidirectional LSTM for Deceptive Opinion Spam Classification [0.0]
  Fraudulent reviews are deliberately posted on various online review platforms to trick customers to buy, visit or distract against a product or a restaurant. The work aims at detecting and classifying the reviews as deceptive or truthful. It involves use of various deep learning techniques for classifying the reviews and an overview of proposed approach.
  arXiv  Detail & Related papers  (2021-12-29T19:02:04Z)
• Can We Automate Scientific Reviewing? [89.50052670307434]
  We discuss the possibility of using state-of-the-art natural language processing (NLP) models to generate first-pass peer reviews for scientific papers. We collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews.
  arXiv  Detail & Related papers  (2021-01-30T07:16:53Z)
• Abstractive Opinion Tagging [65.47649273721679]
  In e-commerce, opinion tags refer to a ranked list of tags provided by the e-commerce platform that reflect characteristics of reviews of an item. Current mechanisms for generating opinion tags rely on either manual or labelling methods, which is time-consuming and ineffective. We propose an abstractive opinion tagging framework, named AOT-Net, to generate a ranked list of opinion tags given a large number of reviews.
  arXiv  Detail & Related papers  (2021-01-18T05:08:15Z)
• ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis [62.76038841302741]
  We build a novel ReviewRobot to automatically assign a review score and write comments for multiple categories such as novelty and meaningful comparison. Experimental results show that our review score predictor reaches 71.4%-100% accuracy. Human assessment by domain experts shows that 41.7%-70.5% of the comments generated by ReviewRobot are valid and constructive, and better than human-written ones for 20% of the time.
  arXiv  Detail & Related papers  (2020-10-13T02:17:58Z)
• Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing [66.57888248681303]
  We propose a novel emerging issue detection approach named MERIT. Based on the AOBST model, we infer the topics negatively reflected in user reviews for one app version. Experiments on popular apps from Google Play and Apple's App Store demonstrate the effectiveness of MERIT.
  arXiv  Detail & Related papers  (2020-08-23T06:34:05Z)
• How do online consumers review negatively? [1.52292571922932]
  Using 1, 450, 000 negative reviews from JD.com, the largest B2C platform in China, the behavioral patterns from temporal, perceptional and emotional perspectives are explored. Consumers from lower levels express more intensive negative feelings, especially on product pricing and seller attitudes. The value of negative reviews from higher-level consumers is unexpectedly highlighted because of less emotionalization and less biased narration.
  arXiv  Detail & Related papers  (2020-04-28T12:54:30Z)
• Context-aware Helpfulness Prediction for Online Product Reviews [34.47368084659301]
  We propose a neural deep learning model that predicts the helpfulness score of a review. This model is based on convolutional neural network (CNN) and a context-aware encoding mechanism. We validated our model on human annotated dataset and the result shows that our model significantly outperforms existing models for helpfulness prediction.
  arXiv  Detail & Related papers  (2020-04-27T18:19:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.