Causal Analysis of Author Demographics in Academic Peer Review
- URL: http://arxiv.org/abs/2603.06641v1
- Date: Wed, 25 Feb 2026 21:43:13 GMT
- Title: Causal Analysis of Author Demographics in Academic Peer Review
- Abstract summary: This paper quantitatively assesses the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings.<n>Our research indicates statistically substantial causal disadvantages for authors from minority racial groups.<n>The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes.
- Score: 1.6932009464531739
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Academic meritocracy is jeopardized by systematic imbalances; for example, whereas Black and Hispanic individuals constitute over 30% of the U.S. population, they represent fewer than 10% of tenured academics in science and engineering. Peer review serves as a crucial gatekeeper in this process, however it encounters ongoing issues over biases that may hinder scientific advancement. The issue is now exacerbated by the growing influence of artificial intelligence (AI) in academic assessment. This paper transcends correlation to quantitatively assess the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings. We utilize a causal inference methodology on a dataset of 530 papers, simulating the academic selection process by employing the prestige of the publication venue as a surrogate for review rank. Our research indicates statistically substantial causal disadvantages for authors from minority racial groups (average treatment effects [ATE]: -0.42 points in ranking), female authors (ATE: -0.25), and those associated with institutions in the Global South (ATE: -0.57). The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes, indicating that such measures are essential for establishing a more equitable and credible scientific environment.
Related papers
- Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework [55.078301794183496]
We focus on a core reviewing skill that underpins high-quality peer review: detecting faulty research logic.<n>This involves evaluating the internal consistency between a paper's results, interpretations, and claims.<n>We present a fully automated counterfactual evaluation framework that isolates and tests this skill under controlled conditions.
arXiv Detail & Related papers (2025-08-29T08:48:00Z) - Beyond authorship: Analyzing contributions in PLOS ONE and the challenges of appropriate attribution [0.0]
The study analyzes 81,823 publications from the journal PLOS ONE.<n> 9.14% of articles feature at least one author with inappropriate authorship, affecting over 14,000 individuals.<n>Inappropriate authorship is more concentrated in Asia, Africa, and specific European countries like Italy.
arXiv Detail & Related papers (2025-04-08T06:47:52Z) - Biases in gendered citation practices: an exploratory study and some reflections on the Matthew and Matilda effects [2.0277446818410994]
This paper aims at analyzing gendered citation practices in the software engineering literature.<n>Our results show that some efforts still need to be done to achieve fairness in citation practices in the SE field.
arXiv Detail & Related papers (2024-09-18T19:53:41Z) - The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review [49.43514488610211]
Author-provided rankings could be leveraged to improve peer review processes at machine learning conferences.<n>We focus on the Isotonic Mechanism, which calibrates raw review scores using the author-provided rankings.<n>We propose several cautious, low-risk applications of the Isotonic Mechanism and author-provided rankings in peer review.
arXiv Detail & Related papers (2024-08-24T01:51:23Z) - A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
arXiv Detail & Related papers (2024-02-20T11:28:50Z) - The Howard-Harvard effect: Institutional reproduction of intersectional
inequalities [0.36868085124383626]
The U.S. higher education system concentrates the production of science and scientists within a few institutions.
This has implications for minoritized scholars and the topics with which they are disproportionately associated.
We observe a Howard-Harvard effect, in which the topical profile of minoritized scholars are amplified in mission-driven institutions.
arXiv Detail & Related papers (2024-02-06T20:46:54Z) - Position: AI/ML Influencers Have a Place in the Academic Process [82.2069685579588]
We investigate the role of social media influencers in enhancing the visibility of machine learning research.
We have compiled a comprehensive dataset of over 8,000 papers, spanning tweets from December 2018 to October 2023.
Our statistical and causal inference analysis reveals a significant increase in citations for papers endorsed by these influencers.
arXiv Detail & Related papers (2024-01-24T20:05:49Z) - Uncertain research country rankings. Should we continue producing uncertain rankings? [0.0]
Citation-based assessments of countries' research capabilities often misrepresent their ability to achieve breakthrough advancements.<n>The study evaluates the effectiveness of top-percentile citation metrics as indicators of breakthrough research.
arXiv Detail & Related papers (2023-12-29T11:07:37Z) - How do Authors' Perceptions of their Papers Compare with Co-authors'
Perceptions and Peer-review Decisions? [87.00095008723181]
Authors have roughly a three-fold overestimate of the acceptance probability of their papers.
Female authors exhibit a marginally higher (statistically significant) miscalibration than male authors.
At least 30% of respondents of both accepted and rejected papers said that their perception of their own paper improved after the review process.
arXiv Detail & Related papers (2022-11-22T15:59:30Z) - Investigating Fairness Disparities in Peer Review: A Language Model
Enhanced Approach [77.61131357420201]
We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs)
We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date.
We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
arXiv Detail & Related papers (2022-11-07T16:19:42Z) - Toward Understanding Bias Correlations for Mitigation in NLP [34.956581421295]
This work aims to provide a first systematic study toward understanding bias correlations in mitigation.
We examine bias mitigation in two common NLP tasks -- toxicity detection and word embeddings.
Our findings suggest that biases are correlated and present scenarios in which independent debiasing approaches may be insufficient.
arXiv Detail & Related papers (2022-05-24T22:48:47Z) - Ranking Scientific Papers Using Preference Learning [48.78161994501516]
We cast it as a paper ranking problem based on peer review texts and reviewer scores.
We introduce a novel, multi-faceted generic evaluation framework for making final decisions based on peer reviews.
arXiv Detail & Related papers (2021-09-02T19:41:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.