Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations

• URL: http://arxiv.org/abs/2511.05613v1
• Date: Thu, 06 Nov 2025 14:25:32 GMT
• Title: Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
• Authors: Anka Reuel, Avijit Ghosh, Jenny Chim, Andrew Tran, Yanan Long, Jennifer Mickel, Usman Gohar, Srishti Yadav, Pawan Sasanka Ammanamanchi, Mowafak Allaham, Hossein A. Rahmani, Mubashara Akhtar, Felix Friedrich, Robert Scholz, Michael Alexander Riegler, Jan Batzner, Eliya Habba, Arushi Saxena, Anastassia Kornilova, Kevin Wei, Prajna Soni, Yohan Mathew, Kevin Klyman, Jeba Sania, Subramanyam Sahoo, Olivia Beyer Bruvik, Pouya Sadeghi, Sujata Goswami, Angelina Wang, Yacine Jernite, Zeerak Talat, Stella Biderman, Mykel Kochenderfer, Sanmi Koyejo, Irene Solaiman,
• Abstract summary: First comprehensive analysis of both first-party and third-party social impact evaluation reporting across a wide range of model developers.<n>First-party reporting is sparse, often superficial, and has declined over time in key areas such as environmental impact and bias.<n>Third-party evaluators provide broader and more rigorous coverage of bias, harmful content, and performance disparities.
• Score: 37.727313946463276
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models are increasingly central to high-stakes AI systems, and governance frameworks now depend on evaluations to assess their risks and capabilities. Although general capability evaluations are widespread, social impact assessments covering bias, fairness, privacy, environmental costs, and labor practices remain uneven across the AI ecosystem. To characterize this landscape, we conduct the first comprehensive analysis of both first-party and third-party social impact evaluation reporting across a wide range of model developers. Our study examines 186 first-party release reports and 183 post-release evaluation sources, and complements this quantitative analysis with interviews of model developers. We find a clear division of evaluation labor: first-party reporting is sparse, often superficial, and has declined over time in key areas such as environmental impact and bias, while third-party evaluators including academic researchers, nonprofits, and independent organizations provide broader and more rigorous coverage of bias, harmful content, and performance disparities. However, this complementarity has limits. Only model developers can authoritatively report on data provenance, content moderation labor, financial costs, and training infrastructure, yet interviews reveal that these disclosures are often deprioritized unless tied to product adoption or regulatory compliance. Our findings indicate that current evaluation practices leave major gaps in assessing AI's societal impacts, highlighting the urgent need for policies that promote developer transparency, strengthen independent evaluation ecosystems, and create shared infrastructure to aggregate and compare third-party evaluations in a consistent and accessible way.

Related papers

• Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback [81.0031690510116]
  We present a structured approach for automated novelty evaluation that models expert reviewer behavior through three stages.<n>Our method is informed by a large scale analysis of human written novelty reviews.<n> Evaluated on 182 ICLR 2025 submissions, the approach achieves 86.5% alignment with human reasoning and 75.3% agreement on novelty conclusions.
  arXiv  Detail & Related papers  (2025-08-14T16:18:37Z)
• OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models [55.21589313404023]
  OpenReview is a continually evolving repository of research papers, peer reviews, author rebuttals, meta-reviews, and decision outcomes.<n>We highlight three promising areas in which OpenReview can uniquely contribute: enhancing the quality, scalability, and accountability of peer review processes; enabling meaningful, open-ended benchmarks rooted in genuine expert deliberation; and supporting alignment research through real-world interactions reflecting expert assessment, intentions, and scientific values.<n>We suggest the community collaboratively explore standardized benchmarks and usage guidelines around OpenReview, inviting broader dialogue on responsible data use, ethical considerations, and collective stewardship.
  arXiv  Detail & Related papers  (2025-05-24T09:07:13Z)
• Towards Automated Situation Awareness: A RAG-Based Framework for Peacebuilding Reports [2.230742111425553]
  This paper introduces a dynamic Retrieval-Augmented Generation (RAG) system that autonomously generates situation awareness reports.<n>Our system constructs query-specific knowledge bases on demand, ensuring timely, relevant, and accurate insights.<n>The system is tested across multiple real-world scenarios, demonstrating its effectiveness in producing coherent, insightful, and actionable reports.
  arXiv  Detail & Related papers  (2025-05-14T16:36:30Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators [11.819135112719623]
  We argue that despite potential advantages in addressing contamination issues, private evaluations introduce inadvertent financial and evaluation risks.<n>Key concerns include the potential conflict of interest arising from private data curators' business relationships with their clients.<n>We highlight that the subjective preferences of private expert annotators will lead to inherent evaluation bias towards the models trained with the private curators' data.
  arXiv  Detail & Related papers  (2025-02-09T23:57:33Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research [75.84463664853125]
  We provide a first attempt to quantify concerns regarding three topics, namely, environmental impact, equity, and impact on peer reviewing. We capture existing (dis)parities between different and within groups with respect to seniority, academia, and industry. We devise recommendations to mitigate found disparities, some of which already successfully implemented.
  arXiv  Detail & Related papers  (2023-06-29T12:44:53Z)
• Evaluating the Social Impact of Generative AI Systems in Systems and Society [43.32010533676472]
  Generative AI systems across modalities, ranging from text (including code), image, audio, and video, have broad social impacts. There is no official standard for means of evaluating those impacts or for which impacts should be evaluated. We present a guide that moves toward a standard approach in evaluating a base generative AI system for any modality.
  arXiv  Detail & Related papers  (2023-06-09T15:05:13Z)
• Heterogeneous Ensemble for ESG Ratings Prediction [1.9659095632676094]
  Investors rely on specialized rating agencies that issue ratings along the environmental, social and governance dimensions. Rating agencies base their analysis on subjective assessment of sustainability reports, not provided by every company. We propose a heterogeneous ensemble model to predict ESG ratings using fundamental data.
  arXiv  Detail & Related papers  (2021-09-21T10:42:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.