HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking

• URL: http://arxiv.org/abs/2309.08503v2
• Date: Mon, 25 Mar 2024 08:33:37 GMT
• Title: HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking
• Authors: Juraj Vladika, Phillip Schneider, Florian Matthes,
• Abstract summary: HealthFC is a dataset of 750 health-related claims in German and English labeled for veracity by medical experts. We provide an analysis of the dataset, highlighting its characteristics and challenges. We show that the dataset is a challenging test bed with a high potential for future use.
• Score: 5.065947993017158
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In the digital age, seeking health advice on the Internet has become a common practice. At the same time, determining the trustworthiness of online medical content is increasingly challenging. Fact-checking has emerged as an approach to assess the veracity of factual claims using evidence from credible knowledge sources. To help advance automated Natural Language Processing (NLP) solutions for this task, in this paper we introduce a novel dataset HealthFC. It consists of 750 health-related claims in German and English, labeled for veracity by medical experts and backed with evidence from systematic reviews and clinical trials. We provide an analysis of the dataset, highlighting its characteristics and challenges. The dataset can be used for NLP tasks related to automated fact-checking, such as evidence retrieval, claim verification, or explanation generation. For testing purposes, we provide baseline systems based on different approaches, examine their performance, and discuss the findings. We show that the dataset is a challenging test bed with a high potential for future use.

Related papers

• Diagnosing Medical Datasets with Training Dynamics [0.0]
  This study explores the potential of using training dynamics as an automated alternative to human annotation. The framework used is Data Maps, which classifies data points into categories such as easy-to-learn, hard-to-learn, and ambiguous. A comprehensive evaluation was conducted to assess the feasibility and transferability of the Data Maps framework to the medical domain.
  arXiv  Detail & Related papers  (2024-11-03T18:37:35Z)
• FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection [83.54960238236548]
  FEDMEKI not only preserves data privacy but also enhances the capability of medical foundation models. FEDMEKI allows medical foundation models to learn from a broader spectrum of medical knowledge without direct data exposure.
  arXiv  Detail & Related papers  (2024-08-17T15:18:56Z)
• Identifying and Aligning Medical Claims Made on Social Media with Medical Evidence [0.12277343096128711]
  We study three core tasks: identifying medical claims, extracting medical vocabulary from these claims, and retrieving evidence relevant to those identified medical claims. We propose a novel system that can generate synthetic medical claims to aid each of these core tasks.
  arXiv  Detail & Related papers  (2024-05-18T07:50:43Z)
• Leveraging text data for causal inference using electronic health records [1.4182510510164876]
  This paper presents a unified framework for leveraging text data to support causal inference with electronic health data. We show how incorporating text data in a traditional matching analysis can help strengthen the validity of an estimated treatment effect. We believe these methods have the potential to expand the scope of secondary analysis of clinical data to domains where structured EHR data is limited.
  arXiv  Detail & Related papers  (2023-06-09T16:06:02Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes [50.8044927215346]
  We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state. We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
  arXiv  Detail & Related papers  (2023-02-11T18:07:11Z)
• Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine [68.7814360102644]
  We propose the Re$3$Writer method with retrieval-augmented generation and knowledge-grounded reasoning. We demonstrate the effectiveness of our method in generating patient discharge instructions.
  arXiv  Detail & Related papers  (2022-10-23T16:34:39Z)
• CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking [55.75590135151682]
  CHEF is the first CHinese Evidence-based Fact-checking dataset of 10K real-world claims. The dataset covers multiple domains, ranging from politics to public health, and provides annotated evidence retrieved from the Internet.
  arXiv  Detail & Related papers  (2022-06-06T09:11:03Z)
• Healthsheet: Development of a Transparency Artifact for Health Datasets [13.57051456780329]
  We introduce Healthsheet, a contextualized adaptation of the original questionnaire citegebru 2018datasheets for health-specific applications. We work with three publicly-available healthcare datasets as our case studies.
  arXiv  Detail & Related papers  (2022-02-26T01:05:55Z)
• Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data [0.0]
  We present a series of tests including a large variety of contemporary uncertainty estimation techniques. In contrast to previous work, we design tests on realistic and clinically relevant OOD groups, and run experiments on real-world medical data.
  arXiv  Detail & Related papers  (2020-11-06T10:41:39Z)
• Assessing the Severity of Health States based on Social Media Posts [62.52087340582502]
  We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
  arXiv  Detail & Related papers  (2020-09-21T03:45:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.