The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms

• URL: http://arxiv.org/abs/2508.16672v1
• Date: Thu, 21 Aug 2025 07:07:41 GMT
• Title: The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms
• Abstract summary: We analyzed nearly 460,000 AI model cards from Hugging Face to examine how developers report risks.<n>We compared these with risks identified by researchers in the MIT Risk Repository and with real-world incidents from the AI Incident Database.<n>Our findings show the need for clearer, structured risk reporting that helps developers think about human-interaction and systemic risks early in the design process.
• Score: 7.37865739449202
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We analyzed nearly 460,000 AI model cards from Hugging Face to examine how developers report risks. From these, we extracted around 3,000 unique risk mentions and built the \emph{AI Model Risk Catalog}. We compared these with risks identified by researchers in the MIT Risk Repository and with real-world incidents from the AI Incident Database. Developers focused on technical issues like bias and safety, while researchers emphasized broader social impacts. Both groups paid little attention to fraud and manipulation, which are common harms arising from how people interact with AI. Our findings show the need for clearer, structured risk reporting that helps developers think about human-interaction and systemic risks early in the design process. The catalog and paper appendix are available at: https://social-dynamics.net/ai-risks/catalog.

Related papers

• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5 [61.787178868669265]
  This technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication.<n>This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
  arXiv  Detail & Related papers  (2026-02-16T04:30:06Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• "We are not Future-ready": Understanding AI Privacy Risks and Existing Mitigation Strategies from the Perspective of AI Developers in Europe [56.1653658714305]
  We interviewed 25 AI developers based in Europe to understand which privacy threats they believe pose the greatest risk to users, developers, and businesses.<n>We find that there is little consensus among AI developers on the relative ranking of privacy risks.<n>While AI developers are aware of proposed mitigation strategies for addressing these risks, they reported minimal real-world adoption.
  arXiv  Detail & Related papers  (2025-10-01T13:51:33Z)
• Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path? [37.13209023718946]
  Unchecked AI agency poses significant risks to public safety and security.<n>We discuss how these risks arise from current AI training methods.<n>We propose a core building block for further advances the development of a non-agentic AI system.
  arXiv  Detail & Related papers  (2025-02-21T18:28:36Z)
• Fully Autonomous AI Agents Should Not be Developed [58.88624302082713]
  This paper argues that fully autonomous AI agents should not be developed.<n>In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels.<n>Our analysis reveals that risks to people increase with the autonomy of a system.
  arXiv  Detail & Related papers  (2025-02-04T19:00:06Z)
• Risk Alignment in Agentic AI Systems [0.0]
  Agentic AIs capable of undertaking complex actions with little supervision raise new questions about how to safely create and align such systems with users, developers, and society. Risk alignment will matter for user satisfaction and trust, but it will also have important ramifications for society more broadly. We present three papers that bear on key normative and technical aspects of these questions.
  arXiv  Detail & Related papers  (2024-10-02T18:21:08Z)
• Mapping Technical Safety Research at AI Companies: A literature review and incentives analysis [0.0]
  Report analyzes the technical research into safe AI development being conducted by three leading AI companies. Anthropic, Google DeepMind, and OpenAI. We defined safe AI development as developing AI systems that are unlikely to pose large-scale misuse or accident risks.
  arXiv  Detail & Related papers  (2024-09-12T09:34:55Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• The AI Risk Repository: A Comprehensive Meta-Review, Database, and Taxonomy of Risks From Artificial Intelligence [35.77247656798871]
  The risks posed by Artificial Intelligence (AI) are of considerable concern to academics, auditors, policymakers, AI companies, and the public.<n>A lack of shared understanding of AI risks can impede our ability to comprehensively discuss, research, and react to them.<n>This paper addresses this gap by creating an AI Risk Repository to serve as a common frame of reference.
  arXiv  Detail & Related papers  (2024-08-14T10:32:06Z)
• Near to Mid-term Risks and Opportunities of Open-Source Generative AI [94.06233419171016]
  Applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation. This regulation is likely to put at risk the budding field of open-source Generative AI.
  arXiv  Detail & Related papers  (2024-04-25T21:14:24Z)
• Control Risk for Potential Misuse of Artificial Intelligence in Science [85.91232985405554]
  We aim to raise awareness of the dangers of AI misuse in science. We highlight real-world examples of misuse in chemical science. We propose a system called SciGuard to control misuse risks for AI models in science.
  arXiv  Detail & Related papers  (2023-12-11T18:50:57Z)
• An Overview of Catastrophic AI Risks [38.84933208563934]
  This paper provides an overview of the main sources of catastrophic AI risks, which we organize into four categories. Malicious use, in which individuals or groups intentionally use AIs to cause harm; AI race, in which competitive environments compel actors to deploy unsafe AIs or cede control to AIs. organizational risks, highlighting how human factors and complex systems can increase the chances of catastrophic accidents. rogue AIs, describing the inherent difficulty in controlling agents far more intelligent than humans.
  arXiv  Detail & Related papers  (2023-06-21T03:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.