Safety challenges of AI in medicine

• URL: http://arxiv.org/abs/2409.18968v1
• Date: Wed, 11 Sep 2024 13:47:47 GMT
• Title: Safety challenges of AI in medicine
• Authors: Xiaoye Wang, Nicole Xi Zhang, Hongyu He, Trang Nguyen, Kun-Hsing Yu, Hao Deng, Cynthia Brandt, Danielle S. Bitterman, Ling Pan, Ching-Yu Cheng, James Zou, Dianbo Liu,
• Abstract summary: Review examines potential risks in AI practices that may compromise safety in medicine. Examines reduced performance across diverse populations, inconsistent operational stability, the need for high-quality data for effective model tuning, and the risk of data breaches during model development and deployment. Second part of this article explores safety issues specific to large language models (LLMs) in medical contexts.
• Score: 23.817939398729955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in artificial intelligence (AI), particularly in deep learning and large language models (LLMs), have accelerated their integration into medicine. However, these developments have also raised public concerns about the safe application of AI. In healthcare, these concerns are especially pertinent, as the ethical and secure deployment of AI is crucial for protecting patient health and privacy. This review examines potential risks in AI practices that may compromise safety in medicine, including reduced performance across diverse populations, inconsistent operational stability, the need for high-quality data for effective model tuning, and the risk of data breaches during model development and deployment. For medical practitioners, patients, and researchers, LLMs provide a convenient way to interact with AI and data through language. However, their emergence has also amplified safety concerns, particularly due to issues like hallucination. Second part of this article explores safety issues specific to LLMs in medical contexts, including limitations in processing complex logic, challenges in aligning AI objectives with human values, the illusion of understanding, and concerns about diversity. Thoughtful development of safe AI could accelerate its adoption in real-world medical settings.

Related papers

• Standardization Trends on Safety and Trustworthiness Technology for Advanced AI [0.0]
  Recent AI technologies based on large language models and foundation models are approaching or surpassing artificial general intelligence. These advancements have raised concerns regarding the safety and trustworthiness of advanced AI. Efforts are being expended to develop internationally agreed-upon standards to ensure the safety and reliability of AI.
  arXiv  Detail & Related papers  (2024-10-29T15:50:24Z)
• Enhancing Guardrails for Safe and Secure Healthcare AI [0.0]
  I propose enhancements to existing guardrails frameworks, such as Nvidia NeMo Guardrails, to better suit healthcare-specific needs. I aim to ensure the secure, reliable, and accurate use of AI in healthcare, mitigating misinformation risks and improving patient safety.
  arXiv  Detail & Related papers  (2024-09-25T06:30:06Z)
• HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactions [76.42274173122328]
  We present HAICOSYSTEM, a framework examining AI agent safety within diverse and complex social interactions. We run 1840 simulations based on 92 scenarios across seven domains (e.g., healthcare, finance, education) Our experiments show that state-of-the-art LLMs, both proprietary and open-sourced, exhibit safety risks in over 50% cases.
  arXiv  Detail & Related papers  (2024-09-24T19:47:21Z)
• SoK: Security and Privacy Risks of Medical AI [14.592921477833848]
  The integration of technology and healthcare has ushered in a new era where software systems, powered by artificial intelligence and machine learning, have become essential components of medical products and services. This paper explores the security and privacy threats posed by AI/ML applications in healthcare.
  arXiv  Detail & Related papers  (2024-09-11T16:59:58Z)
• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? [59.96471873997733]
  We propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context. We aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
  arXiv  Detail & Related papers  (2024-07-31T17:59:24Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• RAISE -- Radiology AI Safety, an End-to-end lifecycle approach [5.829180249228172]
  The integration of AI into radiology introduces opportunities for improved clinical care provision and efficiency. The focus should be on ensuring models meet the highest standards of safety, effectiveness and efficacy. The roadmap presented herein aims to expedite the achievement of deployable, reliable, and safe AI in radiology.
  arXiv  Detail & Related papers  (2023-11-24T15:59:14Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond [0.0]
  This comprehensive research article rigorously investigates the ethical dimensions intricately linked to the rapid evolution of AI technologies. Central to this article is the proposition of a conscientious AI framework, meticulously crafted to accentuate values of transparency, equity, answerability, and a human-centric orientation. The article unequivocally accentuates the pressing need for globally standardized AI ethics principles and frameworks.
  arXiv  Detail & Related papers  (2023-08-31T18:12:12Z)
• FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare [73.78776682247187]
  Concerns have been raised about the technical, clinical, ethical and legal risks associated with medical AI. This work describes the FUTURE-AI guideline as the first international consensus framework for guiding the development and deployment of trustworthy AI tools in healthcare.
  arXiv  Detail & Related papers  (2023-08-11T10:49:05Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.