The case for delegated AI autonomy for Human AI teaming in healthcare

• URL: http://arxiv.org/abs/2503.18778v1
• Date: Mon, 24 Mar 2025 15:26:54 GMT
• Title: The case for delegated AI autonomy for Human AI teaming in healthcare
• Authors: Yan Jia, Harriet Evans, Zoe Porter, Simon Graham, John McDermid, Tom Lawton, David Snead, Ibrahim Habli,
• Abstract summary: We propose an advanced approach to integrating artificial intelligence (AI) into healthcare: autonomous decision support.<n>This approach allows the AI algorithm to act autonomously for a subset of patient cases whilst serving a supportive role in other subsets of patient cases based on defined delegation criteria.<n>It ensures safe handling of patient cases and potentially reduces clinician review time, whilst being mindful of AI tool limitations.
• Score: 3.441725960809854
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper we propose an advanced approach to integrating artificial intelligence (AI) into healthcare: autonomous decision support. This approach allows the AI algorithm to act autonomously for a subset of patient cases whilst serving a supportive role in other subsets of patient cases based on defined delegation criteria. By leveraging the complementary strengths of both humans and AI, it aims to deliver greater overall performance than existing human-AI teaming models. It ensures safe handling of patient cases and potentially reduces clinician review time, whilst being mindful of AI tool limitations. After setting the approach within the context of current human-AI teaming models, we outline the delegation criteria and apply them to a specific AI-based tool used in histopathology. The potential impact of the approach and the regulatory requirements for its successful implementation are then discussed.

Related papers

• Over-Relying on Reliance: Towards Realistic Evaluations of AI-Based Clinical Decision Support [12.247046469627554]
  We advocate for moving beyond evaluation metrics like Trust, Reliance, Acceptance, and Performance on the AI's task. We call on the community to prioritize ecologically valid, domain-appropriate study setups that measure the emergent forms of value that AI can bring to healthcare professionals.
  arXiv  Detail & Related papers  (2025-04-10T03:28:56Z)
• Explanation-Driven Interventions for Artificial Intelligence Model Customization: Empowering End-Users to Tailor Black-Box AI in Rhinocytology [3.4705962607086973]
  This article presents a novel End-User Development (EUD) approach for black-box AI models. The proposed method advances Human-Centered AI (HCAI), promoting a symbiotic relationship between humans and adaptive, user-tailored AI systems.
  arXiv  Detail & Related papers  (2025-04-07T08:44:48Z)
• Envisioning an AI-Enhanced Mental Health Ecosystem [1.534667887016089]
  We explore various AI applications in peer support, self-help interventions, proactive monitoring, and data-driven insights.<n>We propose a hybrid ecosystem where AI assists but does not replace human providers, emphasising responsible deployment and evaluation.
  arXiv  Detail & Related papers  (2025-03-19T04:21:38Z)
• How Performance Pressure Influences AI-Assisted Decision Making [57.53469908423318]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Interactive Example-based Explanations to Improve Health Professionals' Onboarding with AI for Human-AI Collaborative Decision Making [2.964175945467257]
  A growing research explores the usage of AI explanations on user's decision phases for human-AI collaborative decision-making. Previous studies found the issues of overreliance on wrong' AI outputs. We propose interactive example-based explanations to improve health professionals' offboarding with AI.
  arXiv  Detail & Related papers  (2024-09-24T07:20:09Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Making [48.179458030691286]
  We examine three AI roles: Recommender, Analyzer, and Devil's Advocate. Our results show each role's distinct strengths and limitations in task performance, reliance appropriateness, and user experience. These insights offer valuable implications for designing AI assistants with adaptive functional roles according to different situations.
  arXiv  Detail & Related papers  (2024-03-04T07:32:28Z)
• Applying HCAI in developing effective human-AI teaming: A perspective from human-AI joint cognitive systems [10.746728034149989]
  Research and application have used human-AI teaming (HAT) as a new paradigm to develop AI systems. We elaborate on our proposed conceptual framework of human-AI joint cognitive systems (HAIJCS) We propose a conceptual framework of human-AI joint cognitive systems (HAIJCS) to represent and implement HAT.
  arXiv  Detail & Related papers  (2023-07-08T06:26:38Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging [24.911186503082465]
  This study explores the effects of providing AI assistance at the start of a diagnostic session in radiology versus after the radiologist has made a provisional decision. We found that participants who are asked to register provisional responses in advance of reviewing AI inferences are less likely to agree with the AI regardless of whether the advice is accurate and, in instances of disagreement with the AI, are less likely to seek the second opinion of a colleague.
  arXiv  Detail & Related papers  (2022-05-19T16:59:25Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.