Med-MMHL: A Multi-Modal Dataset for Detecting Human- and LLM-Generated Misinformation in the Medical Domain

• URL: http://arxiv.org/abs/2306.08871v1
• Date: Thu, 15 Jun 2023 05:59:11 GMT
• Title: Med-MMHL: A Multi-Modal Dataset for Detecting Human- and LLM-Generated Misinformation in the Medical Domain
• Authors: Yanshen Sun, Jianfeng He, Shuo Lei, Limeng Cui, Chang-Tien Lu
• Abstract summary: Med-MMHL is a novel multi-modal misinformation detection dataset in a general medical domain encompassing multiple diseases. Our dataset aims to facilitate comprehensive research and development of methodologies for detecting misinformation across diverse diseases and various scenarios.
• Score: 14.837495995122598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pervasive influence of misinformation has far-reaching and detrimental effects on both individuals and society. The COVID-19 pandemic has witnessed an alarming surge in the dissemination of medical misinformation. However, existing datasets pertaining to misinformation predominantly focus on textual information, neglecting the inclusion of visual elements, and tend to center solely on COVID-19-related misinformation, overlooking misinformation surrounding other diseases. Furthermore, the potential of Large Language Models (LLMs), such as the ChatGPT developed in late 2022, in generating misinformation has been overlooked in previous works. To overcome these limitations, we present Med-MMHL, a novel multi-modal misinformation detection dataset in a general medical domain encompassing multiple diseases. Med-MMHL not only incorporates human-generated misinformation but also includes misinformation generated by LLMs like ChatGPT. Our dataset aims to facilitate comprehensive research and development of methodologies for detecting misinformation across diverse diseases and various scenarios, including human and LLM-generated misinformation detection at the sentence, document, and multi-modal levels. To access our dataset and code, visit our GitHub repository: \url{https://github.com/styxsys0927/Med-MMHL}.

Related papers

• MedINST: Meta Dataset of Biomedical Instructions [47.7146913542186]
  MedINST comprises 133 biomedical NLP tasks and over 7 million training samples. We fine-tune several LLMs on MedINST and evaluate on MedINST32, showcasing enhanced cross-task generalization.
  arXiv  Detail & Related papers  (2024-10-17T11:38:54Z)
• FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection [83.54960238236548]
  FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models. FEDMEKI allows medical foundation models to learn from a broader spectrum of medical knowledge without direct data exposure.
  arXiv  Detail & Related papers  (2024-08-17T15:18:56Z)
• Assessing and Enhancing Large Language Models in Rare Disease Question-answering [64.32570472692187]
  We introduce a rare disease question-answering (ReDis-QA) dataset to evaluate the performance of Large Language Models (LLMs) in diagnosing rare diseases. We collected 1360 high-quality question-answer pairs within the ReDis-QA dataset, covering 205 rare diseases. We then benchmarked several open-source LLMs, revealing that diagnosing rare diseases remains a significant challenge for these models. Experiment results demonstrate that ReCOP can effectively improve the accuracy of LLMs on the ReDis-QA dataset by an average of 8%.
  arXiv  Detail & Related papers  (2024-08-15T21:09:09Z)
• AMIR: Automated MisInformation Rebuttal -- A COVID-19 Vaccination Datasets based Recommendation System [0.05461938536945722]
  This work explored how existing information obtained from social media can be harnessed to facilitate automated rebuttal of misinformation at scale. It leverages two publicly available datasets, FaCov (fact-checked articles) and misleading (social media Twitter) data on COVID-19 Vaccination.
  arXiv  Detail & Related papers  (2023-10-29T13:07:33Z)
• Med-HALT: Medical Domain Hallucination Test for Large Language Models [0.0]
  This research paper focuses on the challenges posed by hallucinations in large language models (LLMs) We propose a new benchmark and dataset, Med-HALT (Medical Domain Hallucination Test), designed specifically to evaluate and reduce hallucinations.
  arXiv  Detail & Related papers  (2023-07-28T06:43:04Z)
• Med-Flamingo: a Multimodal Medical Few-shot Learner [58.85676013818811]
  We propose Med-Flamingo, a multimodal few-shot learner adapted to the medical domain. Based on OpenFlamingo-9B, we continue pre-training on paired and interleaved medical image-text data from publications and textbooks. We conduct the first human evaluation for generative medical VQA where physicians review the problems and blinded generations in an interactive app.
  arXiv  Detail & Related papers  (2023-07-27T20:36:02Z)
• Cross-Modal Causal Intervention for Medical Report Generation [109.83549148448469]
  Medical report generation (MRG) is essential for computer-aided diagnosis and medication guidance. Due to the spurious correlations within image-text data induced by visual and linguistic biases, it is challenging to generate accurate reports reliably describing lesion areas. We propose a novel Visual-Linguistic Causal Intervention (VLCI) framework for MRG, which consists of a visual deconfounding module (VDM) and a linguistic deconfounding module (LDM)
  arXiv  Detail & Related papers  (2023-03-16T07:23:55Z)
• Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection [6.1204874238049705]
  A drastic rise in potentially life-threatening misinformation has been a by-product of the COVID-19 pandemic. We evaluate fifteen Transformer-based models on five COVID-19 misinformation datasets. We show tokenizers and models tailored to COVID-19 data do not provide a significant advantage over general-purpose ones.
  arXiv  Detail & Related papers  (2021-11-15T15:01:55Z)
• Cross-lingual COVID-19 Fake News Detection [54.125563009333995]
  We make the first attempt to detect COVID-19 misinformation in a low-resource language (Chinese) only using the fact-checked news in a high-resource language (English) We propose a deep learning framework named CrossFake to jointly encode the cross-lingual news body texts and capture the news content. Empirical results on our dataset demonstrate the effectiveness of CrossFake under the cross-lingual setting.
  arXiv  Detail & Related papers  (2021-10-13T04:44:02Z)
• Case Study on Detecting COVID-19 Health-Related Misinformation in Social Media [7.194177427819438]
  This paper presents a mechanism to detect COVID-19 health-related misinformation in social media. We defined misinformation themes and associated keywords incorporated into the misinformation detection mechanism using applied machine learning techniques. Our method shows promising results with at most 78% accuracy in classifying health-related misinformation versus true information.
  arXiv  Detail & Related papers  (2021-06-12T16:26:04Z)
• Drink Bleach or Do What Now? Covid-HeRA: A Study of Risk-Informed Health Decision Making in the Presence of COVID-19 Misinformation [23.449057978351945]
  We frame health misinformation as a risk assessment task. We study the severity of each misinformation story and how readers perceive this severity. We evaluate several traditional and state-of-the-art models and show there is a significant gap in performance.
  arXiv  Detail & Related papers  (2020-10-17T08:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.