What's my role? Modelling responsibility for AI-based safety-critical
systems
- URL: http://arxiv.org/abs/2401.09459v1
- Date: Sat, 30 Dec 2023 13:45:36 GMT
- Title: What's my role? Modelling responsibility for AI-based safety-critical
systems
- Authors: Philippa Ryan, Zoe Porter, Joanna Al-Qaddoumi, John McDermid, Ibrahim
Habli
- Abstract summary: It is difficult for developers and manufacturers to be held responsible for harmful behaviour of an AI-SCS.
A human operator can become a "liability sink" absorbing blame for the consequences of AI-SCS outputs they weren't responsible for creating.
This paper considers different senses of responsibility (role, moral, legal and causal), and how they apply in the context of AI-SCS safety.
- Score: 1.0549609328807565
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: AI-Based Safety-Critical Systems (AI-SCS) are being increasingly deployed in
the real world. These can pose a risk of harm to people and the environment.
Reducing that risk is an overarching priority during development and operation.
As more AI-SCS become autonomous, a layer of risk management via human
intervention has been removed. Following an accident it will be important to
identify causal contributions and the different responsible actors behind those
to learn from mistakes and prevent similar future events. Many authors have
commented on the "responsibility gap" where it is difficult for developers and
manufacturers to be held responsible for harmful behaviour of an AI-SCS. This
is due to the complex development cycle for AI, uncertainty in AI performance,
and dynamic operating environment. A human operator can become a "liability
sink" absorbing blame for the consequences of AI-SCS outputs they weren't
responsible for creating, and may not have understanding of.
This cross-disciplinary paper considers different senses of responsibility
(role, moral, legal and causal), and how they apply in the context of AI-SCS
safety. We use a core concept (Actor(A) is responsible for Occurrence(O)) to
create role responsibility models, producing a practical method to capture
responsibility relationships and provide clarity on the previously identified
responsibility issues. Our paper demonstrates the approach with two examples: a
retrospective analysis of the Tempe Arizona fatal collision involving an
autonomous vehicle, and a safety focused predictive role-responsibility
analysis for an AI-based diabetes co-morbidity predictor. In both examples our
primary focus is on safety, aiming to reduce unfair or disproportionate blame
being placed on operators or developers. We present a discussion and avenues
for future research.
Related papers
- Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs.
We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
arXiv Detail & Related papers (2024-05-14T09:42:21Z) - A risk-based approach to assessing liability risk for AI-driven harms
considering EU liability directive [0.0]
Historical instances of harm caused by AI have led to European Union establishing an AI Liability Directive.
The future ability of provider to contest a product liability claim will depend on good practices adopted in designing, developing, and maintaining AI systems.
This paper provides a risk-based approach to examining liability for AI-driven injuries.
arXiv Detail & Related papers (2023-12-18T15:52:43Z) - Control Risk for Potential Misuse of Artificial Intelligence in Science [85.91232985405554]
We aim to raise awareness of the dangers of AI misuse in science.
We highlight real-world examples of misuse in chemical science.
We propose a system called SciGuard to control misuse risks for AI models in science.
arXiv Detail & Related papers (2023-12-11T18:50:57Z) - Managing extreme AI risks amid rapid progress [171.05448842016125]
We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems.
There is a lack of consensus about how exactly such risks arise, and how to manage them.
Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
arXiv Detail & Related papers (2023-10-26T17:59:06Z) - Unravelling Responsibility for AI [0.8836921728313208]
It is widely acknowledged that we need to establish where responsibility lies for the outputs and impacts of AI-enabled systems.
This paper draws upon central distinctions in philosophy and law to clarify the concept of responsibility for AI.
arXiv Detail & Related papers (2023-08-04T13:12:17Z) - Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time.
We discuss how biased models can lead to more negative real-world outcomes for certain groups.
If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
arXiv Detail & Related papers (2023-04-16T11:22:59Z) - AI Maintenance: A Robustness Perspective [91.28724422822003]
We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance.
We propose an AI model inspection framework to detect and mitigate robustness risks.
Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
arXiv Detail & Related papers (2023-01-08T15:02:38Z) - Operationalising Responsible AI Using a Pattern-Oriented Approach: A
Case Study on Chatbots in Financial Services [11.33499498841489]
Responsible AI is the practice of developing and using AI systems in a way that benefits the humans, society, and environment.
Various responsible AI principles have been released recently, but those principles are very abstract and not practical enough.
To bridge the gap, we adopt a pattern-oriented approach and build a responsible AI pattern catalogue.
arXiv Detail & Related papers (2023-01-03T23:11:03Z) - Liability regimes in the age of AI: a use-case driven analysis of the
burden of proof [1.7510020208193926]
New emerging technologies powered by Artificial Intelligence (AI) have the potential to disruptively transform our societies for the better.
But there is growing concerns about certain intrinsic characteristics of these methodologies that carry potential risks to both safety and fundamental rights.
This paper presents three case studies, as well as the methodology to reach them, that illustrate these difficulties.
arXiv Detail & Related papers (2022-11-03T13:55:36Z) - Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations.
It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
arXiv Detail & Related papers (2022-01-26T18:53:09Z) - Human Perceptions on Moral Responsibility of AI: A Case Study in
AI-Assisted Bail Decision-Making [8.688778020322758]
We measure people's perceptions of eight different notions of moral responsibility concerning AI and human agents.
We show that AI agents are held causally responsible and blamed similarly to human agents for an identical task.
We find that people expect both AI and human decision-makers and advisors to justify their decisions regardless of their nature.
arXiv Detail & Related papers (2021-02-01T04:07:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.