Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings

• URL: http://arxiv.org/abs/2606.04286v1
• Date: Tue, 02 Jun 2026 23:26:44 GMT
• Title: Using Text-Based Causal Inference to Disentangle Factors Influencing Online Review Ratings
• Abstract summary: This paper introduces a methodology based on recent advances in text-based causal analysis, specifically CausalBERT, to disentangle the effect of each factor on overall review ratings.<n>We treat the textual mentions in reviews as proxies for real-world attributes.<n>We validate our approach on real and semi-synthetic data from over 600K reviews of U.S. K-12 schools.
• Score: 7.424784821580992
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online reviews provide valuable insights into the perceived quality of facets of a product or service. While aspect-based sentiment analysis has focused on extracting these facets from reviews, there is less work understanding the impact of each aspect on overall perception. This is particularly challenging given correlations among aspects, making it difficult to isolate the effects of each. This paper introduces a methodology based on recent advances in text-based causal analysis, specifically CausalBERT, to disentangle the effect of each factor on overall review ratings. We enhance CausalBERT with three key improvements: temperature scaling for better calibrated treatment assignment estimates; hyperparameter optimization to reduce confound overadjustment; and interpretability methods to characterize discovered confounds. In this work, we treat the textual mentions in reviews as proxies for real-world attributes. We validate our approach on real and semi-synthetic data from over 600K reviews of U.S. K-12 schools. We find that the proposed enhancements result in more reliable estimates, and that perception of school administration and performance on benchmarks are significant drivers of overall school ratings.

Related papers

• Which Review Aspect Has a Greater Impact on the Duration of Open Peer Review in Multiple Rounds? -- Evidence from Nature Communications [4.536998653366957]
  This study examines the relationship between sentiment toward specific review aspects and peer review duration.<n>It also investigates how this relationship varies across disciplines and review rounds.
  arXiv  Detail & Related papers  (2026-06-20T06:41:26Z)
• Insights from the ICLR Peer Review and Rebuttal Process [11.96840383745149]
  We present a large-scale analysis of the ICLR 2024 and 2025 peer review processes.<n>We examine review scores, author-reviewer engagement, temporal patterns in review submissions, and co-reviewer influence effects.<n>Our findings show that initial scores and the ratings of co-reviewers are the strongest predictors of score changes during the rebuttal.
  arXiv  Detail & Related papers  (2025-11-19T14:21:52Z)
• Data Fusion for Partial Identification of Causal Effects [62.56890808004615]
  We propose a novel partial identification framework that enables researchers to answer key questions.<n>Is the causal effect positive or negative? and How severe must assumption violations be to overturn this conclusion?<n>We apply our framework to the Project STAR study, which investigates the effect of classroom size on students' third-grade standardized test performance.
  arXiv  Detail & Related papers  (2025-05-30T07:13:01Z)
• Identifying Aspects in Peer Reviews [59.02879434536289]
  We develop a data-driven schema for deriving aspects from a corpus of peer reviews.<n>We introduce a dataset of peer reviews augmented with aspects and show how it can be used for community-level review analysis.
  arXiv  Detail & Related papers  (2025-04-09T14:14:42Z)
• Analytical and Empirical Study of Herding Effects in Recommendation Systems [72.6693986712978]
  We study how to manage product ratings via rating aggregation rules and shortlisted representative reviews. We show that proper recency aware rating aggregation rules can improve the speed of convergence in Amazon and TripAdvisor.
  arXiv  Detail & Related papers  (2024-08-20T14:29:23Z)
• AgentReview: Exploring Peer Review Dynamics with LLM Agents [13.826819101545926]
  We introduce AgentReview, the first large language model (LLM) based peer review simulation framework. Our study reveals significant insights, including a notable 37.1% variation in paper decisions due to reviewers' biases.
  arXiv  Detail & Related papers  (2024-06-18T15:22:12Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis [56.84237932819403]
  This paper aims to estimate and mitigate the bad effect of textual modality for strong OOD generalization. Inspired by this, we devise a model-agnostic counterfactual framework for multimodal sentiment analysis.
  arXiv  Detail & Related papers  (2022-07-24T03:57:40Z)
• Effects of Multi-Aspect Online Reviews with Unobserved Confounders: Estimation and Implication [19.74820808192969]
  We study the effects of online reviews on business revenue and direct effects with the numerical cause -- ratings -- being the mediator. We draw on recent advances in machine learning and causal inference to together estimate the hidden confounders and causal effects. We present empirical evaluations using real-world examples to discuss the importance and implications of differentiating the multi-aspect effects in strategizing business operations.
  arXiv  Detail & Related papers  (2021-10-04T23:38:21Z)
• Ranking Scientific Papers Using Preference Learning [48.78161994501516]
  We cast it as a paper ranking problem based on peer review texts and reviewer scores. We introduce a novel, multi-faceted generic evaluation framework for making final decisions based on peer reviews.
  arXiv  Detail & Related papers  (2021-09-02T19:41:47Z)
• Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment [4.588028371034406]
  Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. In this work, we detail the process by which we create our domain-dependent lexicon and aspect-informed review form. We end by analyzing validity and discussing conclusions from our corpus of over 6800 peer reviews from nine courses.
  arXiv  Detail & Related papers  (2021-08-02T15:45:11Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.