From Incidents to Insights: Patterns of Responsibility following AI Harms

• URL: http://arxiv.org/abs/2505.04291v1
• Date: Wed, 07 May 2025 09:59:36 GMT
• Title: From Incidents to Insights: Patterns of Responsibility following AI Harms
• Authors: Isabel Richards, Claire Benn, Miri Zilka,
• Abstract summary: The AI Incident Database was inspired by aviation safety databases, which enable collective learning from failures to prevent future incidents.<n>The database documents hundreds of AI failures, collected from the news and media.<n>We argue that by looking beyond technically-focused learning, the dataset can provide new, highly valuable insights.
• Score: 1.9389881806157316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The AI Incident Database was inspired by aviation safety databases, which enable collective learning from failures to prevent future incidents. The database documents hundreds of AI failures, collected from the news and media. However, criticism highlights that the AIID's reliance on media reporting limits its utility for learning about implementation failures. In this paper, we accept that the AIID falls short in its original mission, but argue that by looking beyond technically-focused learning, the dataset can provide new, highly valuable insights: specifically, opportunities to learn about patterns between developers, deployers, victims, wider society, and law-makers that emerge after AI failures. Through a three-tier mixed-methods analysis of 962 incidents and 4,743 related reports from the AIID, we examine patterns across incidents, focusing on cases with public responses tagged in the database. We identify 'typical' incidents found in the AIID, from Tesla crashes to deepfake scams. Focusing on this interplay between relevant parties, we uncover patterns in accountability and social expectations of responsibility. We find that the presence of identifiable responsible parties does not necessarily lead to increased accountability. The likelihood of a response and what it amounts to depends highly on context, including who built the technology, who was harmed, and to what extent. Controversy-rich incidents provide valuable data about societal reactions, including insights into social expectations. Equally informative are cases where controversy is notably absent. This work shows that the AIID's value lies not just in preventing technical failures, but in documenting patterns of harms and of institutional response and social learning around AI incidents. These patterns offer crucial insights for understanding how society adapts to and governs emerging AI technologies.

Related papers

• Information Retrieval in the Age of Generative AI: The RGB Model [77.96475639967431]
  This paper presents a novel quantitative approach to shed light on the complex information dynamics arising from the growing use of generative AI tools.<n>We propose a model to characterize the generation, indexing, and dissemination of information in response to new topics.<n>Our findings suggest that the rapid pace of generative AI adoption, combined with increasing user reliance, can outpace human verification, escalating the risk of inaccurate information proliferation.
  arXiv  Detail & Related papers  (2025-04-29T10:21:40Z)
• Who is Responsible When AI Fails? Mapping Causes, Entities, and Consequences of AI Privacy and Ethical Incidents [29.070947259551478]
  We analyzed 202 real-world AI privacy and ethical incidents.<n>This produced a taxonomy that classifies incident types across AI lifecycle stages.<n>It accounts for contextual factors such as causes, responsible entities, disclosure sources, and impacts.
  arXiv  Detail & Related papers  (2025-03-28T21:57:38Z)
• Position: A taxonomy for reporting and describing AI security incidents [57.98317583163334]
  We argue that specific are required to describe and report security incidents of AI systems.<n>Existing frameworks for either non-AI security or generic AI safety incident reporting are insufficient to capture the specific properties of AI security.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Lessons for Editors of AI Incidents from the AI Incident Database [2.5165775267615205]
  The AI Incident Database (AIID) is a project that catalogs AI incidents and supports further research by providing a platform to classify incidents. This study reviews the AIID's dataset of 750+ AI incidents and two independent ambiguities applied to these incidents to identify common challenges to indexing and analyzing AI incidents. We report mitigations to make incident processes more robust to uncertainty related to cause, extent of harm, severity, or technical details of implicated systems.
  arXiv  Detail & Related papers  (2024-09-24T19:46:58Z)
• Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
  General purpose AI seems to have lowered the barriers for the public to use AI and harness its power. We introduce PARTICIP-AI, a framework for laypeople to speculate and assess AI use cases and their impacts.
  arXiv  Detail & Related papers  (2024-03-21T19:12:37Z)
• What's my role? Modelling responsibility for AI-based safety-critical systems [1.0549609328807565]
  It is difficult for developers and manufacturers to be held responsible for harmful behaviour of an AI-SCS. A human operator can become a "liability sink" absorbing blame for the consequences of AI-SCS outputs they weren't responsible for creating. This paper considers different senses of responsibility (role, moral, legal and causal), and how they apply in the context of AI-SCS safety.
  arXiv  Detail & Related papers  (2023-12-30T13:45:36Z)
• VerifAI: Verified Generative AI [22.14231506649365]
  Generative AI has made significant strides, yet concerns about its accuracy and reliability continue to grow. We propose that verifying the outputs of generative AI from a data management perspective is an emerging issue for generative AI. Our vision is to promote the development of verifiable generative AI and contribute to a more trustworthy and responsible use of AI.
  arXiv  Detail & Related papers  (2023-07-06T06:11:51Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Indexing AI Risks with Incidents, Issues, and Variants [5.8010446129208155]
  backlog of "issues" that do not meet database's incident ingestion criteria have accumulated. Similar to databases in aviation and computer security, the AIID proposes to adopt a two-tiered system for indexing AI incidents.
  arXiv  Detail & Related papers  (2022-11-18T17:32:19Z)
• Attacking Open-domain Question Answering by Injecting Misinformation [116.25434773461465]
  We study the risk of misinformation to Question Answering (QA) models by investigating the sensitivity of open-domain QA models to misinformation documents. Experiments show that QA models are vulnerable to even small amounts of evidence contamination brought by misinformation. We discuss the necessity of building a misinformation-aware QA system that integrates question-answering and misinformation detection.
  arXiv  Detail & Related papers  (2021-10-15T01:55:18Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.