Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects

• URL: http://arxiv.org/abs/2505.18893v4
• Date: Fri, 30 May 2025 14:09:51 GMT
• Title: Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects
• Authors: Reva Schwartz, Rumman Chowdhury, Akash Kundu, Heather Frase, Marzieh Fadaee, Tom David, Gabriella Waters, Afaf Taik, Morgan Briggs, Patrick Hall, Shomik Jain, Kyra Yee, Spencer Thomas, Sundeep Bhandari, Paul Duncan, Andrew Thompson, Maya Carlyle, Qinghua Lu, Matthew Holmes, Theodora Skeadas,
• Abstract summary: The paper argues that measuring the indirect and secondary effects of AI will require expansion beyond static, single-turn approaches conducted in silico.<n>We describe the need for data and methods that can facilitate contextual awareness and enable downstream interpretation and decision making about AI's secondary effects.
• Score: 3.1402583853710433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors. AI capability evaluations can capture detail about first-order effects, such as whether immediate system outputs are accurate, or contain toxic, biased or stereotypical content, but AI's second-order effects, i.e. any long-term outcomes and consequences that may result from AI use in the real world, have become a significant area of interest as the technology becomes embedded in our daily lives. These secondary effects can include shifts in user behavior, societal, cultural and economic ramifications, workforce transformations, and long-term downstream impacts that may result from a broad and growing set of risks. This position paper argues that measuring the indirect and secondary effects of AI will require expansion beyond static, single-turn approaches conducted in silico to include testing paradigms that can capture what actually materializes when people use AI technology in context. Specifically, we describe the need for data and methods that can facilitate contextual awareness and enable downstream interpretation and decision making about AI's secondary effects, and recommend requirements for a new ecosystem.

Related papers

• Aspirational Affordances of AI [0.0]
  There are growing concerns about how artificial intelligence systems may confine individuals and groups to static or restricted narratives about who or what they could be.<n>We introduce the concept of aspirational affordance to describe how culturally shared interpretive resources can shape individual cognition.<n>We show how this concept can ground productive evaluations of the risks of AI-enabled representations and narratives.
  arXiv  Detail & Related papers  (2025-04-21T22:37:49Z)
• From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate [69.05573887799203]
  Much of this debate has concentrated on direct impact without addressing the significant indirect effects.<n>This paper examines how the problem of Jevons' Paradox applies to AI, whereby efficiency gains may paradoxically spur increased consumption.<n>We argue that understanding these second-order impacts requires an interdisciplinary approach, combining lifecycle assessments with socio-economic analyses.
  arXiv  Detail & Related papers  (2025-01-27T22:45:06Z)
• Considerations Influencing Offense-Defense Dynamics From Artificial Intelligence [0.0]
  AI can enhance defensive capabilities but also presents avenues for malicious exploitation and large-scale societal harm.<n>This paper proposes a taxonomy to map and examine the key factors that influence whether AI systems predominantly pose threats or offer protective benefits to society.
  arXiv  Detail & Related papers  (2024-12-05T10:05:53Z)
• How Performance Pressure Influences AI-Assisted Decision Making [57.53469908423318]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• The Potential Impact of AI Innovations on U.S. Occupations [3.0829845709781725]
  We employ Deep Learning Natural Language Processing to identify AI patents that may impact various occupational tasks at scale. Our methodology relies on a comprehensive dataset of 17,879 task descriptions and quantifies AI's potential impact. Our results reveal that some occupations will potentially be impacted, and that impact is intricately linked to specific skills.
  arXiv  Detail & Related papers  (2023-12-07T21:44:07Z)
• Predictable Artificial Intelligence [77.1127726638209]
  This paper introduces the ideas and challenges of Predictable AI.<n>It explores the ways in which we can anticipate key validity indicators of present and future AI ecosystems.<n>We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems.
  arXiv  Detail & Related papers  (2023-10-09T21:36:21Z)
• FATE in AI: Towards Algorithmic Inclusivity and Accessibility [0.0]
  To prevent algorithmic disparities, fairness, accountability, transparency, and ethics (FATE) in AI are being implemented. This study examines FATE-related desiderata, particularly transparency and ethics, in areas of the global South that are underserved by AI. To promote inclusivity, a community-led strategy is proposed to collect and curate representative data for responsible AI design.
  arXiv  Detail & Related papers  (2023-01-03T15:08:10Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Adaptive cognitive fit: Artificial intelligence augmented management of information facets and representations [62.997667081978825]
  Explosive growth in big data technologies and artificial intelligence [AI] applications have led to increasing pervasiveness of information facets. Information facets, such as equivocality and veracity, can dominate and significantly influence human perceptions of information. We suggest that artificially intelligent technologies that can adapt information representations to overcome cognitive limitations are necessary.
  arXiv  Detail & Related papers  (2022-04-25T02:47:25Z)
• AI Assurance using Causal Inference: Application to Public Policy [0.0]
  Most AI approaches can only be represented as "black boxes" and suffer from the lack of transparency. It is crucial not only to develop effective and robust AI systems, but to make sure their internal processes are explainable and fair.
  arXiv  Detail & Related papers  (2021-12-01T16:03:06Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)
• LioNets: A Neural-Specific Local Interpretation Technique Exploiting Penultimate Layer Information [6.570220157893279]
  Interpretable machine learning (IML) is an urgent topic of research. This paper focuses on a local-based, neural-specific interpretation process applied to textual and time-series data.
  arXiv  Detail & Related papers  (2021-04-13T09:39:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.