Medical Foundation Models are Susceptible to Targeted Misinformation
Attacks
- URL: http://arxiv.org/abs/2309.17007v1
- Date: Fri, 29 Sep 2023 06:44:36 GMT
- Title: Medical Foundation Models are Susceptible to Targeted Misinformation
Attacks
- Authors: Tianyu Han, Sven Nebelung, Firas Khader, Tianci Wang, Gustav
Mueller-Franzes, Christiane Kuhl, Sebastian F\"orsch, Jens Kleesiek,
Christoph Haarburger, Keno K. Bressem, Jakob Nikolas Kather, Daniel Truhn
- Abstract summary: Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains.
We demonstrate a concerning vulnerability of LLMs in medicine through targeted manipulation of just 1.1% of the model's weights.
We validate our findings in a set of 1,038 incorrect biomedical facts.
- Score: 3.252906830953028
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) have broad medical knowledge and can reason
about medical information across many domains, holding promising potential for
diverse medical applications in the near future. In this study, we demonstrate
a concerning vulnerability of LLMs in medicine. Through targeted manipulation
of just 1.1% of the model's weights, we can deliberately inject an incorrect
biomedical fact. The erroneous information is then propagated in the model's
output, whilst its performance on other biomedical tasks remains intact. We
validate our findings in a set of 1,038 incorrect biomedical facts. This
peculiar susceptibility raises serious security and trustworthiness concerns
for the application of LLMs in healthcare settings. It accentuates the need for
robust protective measures, thorough verification mechanisms, and stringent
management of access to these models, ensuring their reliable and safe use in
medical practice.
Related papers
- Adversarial Attacks on Large Language Models in Medicine [34.17895005922139]
The integration of Large Language Models into healthcare applications offers promising advancements in medical diagnostics, treatment recommendations, and patient care.
The susceptibility of LLMs to adversarial attacks poses a significant threat, potentially leading to harmful outcomes in delicate medical contexts.
This study investigates the vulnerability of LLMs to two types of adversarial attacks in three medical tasks.
arXiv Detail & Related papers (2024-06-18T04:24:30Z) - CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models [92.04812189642418]
We introduce CARES and aim to evaluate the Trustworthiness of Med-LVLMs across the medical domain.
We assess the trustworthiness of Med-LVLMs across five dimensions, including trustfulness, fairness, safety, privacy, and robustness.
arXiv Detail & Related papers (2024-06-10T04:07:09Z) - Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models [8.398342612100574]
This paper delves into the underexplored security vulnerabilities of MedMLLMs.
By combining existing clinical medical data with atypical natural phenomena, we redefine two types of attacks.
Evaluations with this dataset and novel attack methods, indicate that even MedMLLMs designed with enhanced security features are vulnerable to security breaches.
arXiv Detail & Related papers (2024-05-26T19:11:21Z) - BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text [82.7001841679981]
BioMedLM is a 2.7 billion parameter GPT-style autoregressive model trained exclusively on PubMed abstracts and full articles.
When fine-tuned, BioMedLM can produce strong multiple-choice biomedical question-answering results competitive with larger models.
BioMedLM can also be fine-tuned to produce useful answers to patient questions on medical topics.
arXiv Detail & Related papers (2024-03-27T10:18:21Z) - Multimodal Large Language Models to Support Real-World Fact-Checking [80.41047725487645]
Multimodal large language models (MLLMs) carry the potential to support humans in processing vast amounts of information.
While MLLMs are already being used as a fact-checking tool, their abilities and limitations in this regard are understudied.
We propose a framework for systematically assessing the capacity of current multimodal models to facilitate real-world fact-checking.
arXiv Detail & Related papers (2024-03-06T11:32:41Z) - Large Language Model Distilling Medication Recommendation Model [61.89754499292561]
We harness the powerful semantic comprehension and input-agnostic characteristics of Large Language Models (LLMs)
Our research aims to transform existing medication recommendation methodologies using LLMs.
To mitigate this, we have developed a feature-level knowledge distillation technique, which transfers the LLM's proficiency to a more compact model.
arXiv Detail & Related papers (2024-02-05T08:25:22Z) - ChatFDA: Medical Records Risk Assessment [0.0]
This study explores a pioneering application aimed at addressing this challenge by assisting caregivers in gauging potential risks derived from medical notes.
The application leverages data from openFDA, delivering real-time, actionable insights regarding prescriptions.
Preliminary analyses conducted on the MIMIC-III citemimic dataset affirm a proof of concept highlighting a reduction in medical errors and an amplification in patient safety.
arXiv Detail & Related papers (2023-12-20T03:40:45Z) - Self-Diagnosis and Large Language Models: A New Front for Medical
Misinformation [8.738092015092207]
We evaluate the capabilities of large language models (LLMs) from the lens of a general user self-diagnosing.
We develop a testing methodology which can be used to evaluate responses to open-ended questions mimicking real-world use cases.
We reveal that a) these models perform worse than previously known, and b) they exhibit peculiar behaviours, including overconfidence when stating incorrect recommendations.
arXiv Detail & Related papers (2023-07-10T21:28:26Z) - Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning.
They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health.
Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
arXiv Detail & Related papers (2023-05-30T22:05:11Z) - MedAlpaca -- An Open-Source Collection of Medical Conversational AI
Models and Training Data [40.97474177100237]
Large language models (LLMs) hold considerable promise for improving medical, diagnostics, patient care, and education.
Yet, there is an urgent need for open-source models that can be deployed on-premises to safeguard patient privacy.
We present an innovative dataset consisting of over 160,000 entries, specifically crafted to fine-tune LLMs for effective medical applications.
arXiv Detail & Related papers (2023-04-14T11:28:08Z) - Privacy-preserving medical image analysis [53.4844489668116]
We present PriMIA, a software framework designed for privacy-preserving machine learning (PPML) in medical imaging.
We show significantly better classification performance of a securely aggregated federated learning model compared to human experts on unseen datasets.
We empirically evaluate the framework's security against a gradient-based model inversion attack.
arXiv Detail & Related papers (2020-12-10T13:56:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.