HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking
- URL: http://arxiv.org/abs/2309.08503v2
- Date: Mon, 25 Mar 2024 08:33:37 GMT
- Title: HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking
- Authors: Juraj Vladika, Phillip Schneider, Florian Matthes,
- Abstract summary: HealthFC is a dataset of 750 health-related claims in German and English labeled for veracity by medical experts.
We provide an analysis of the dataset, highlighting its characteristics and challenges.
We show that the dataset is a challenging test bed with a high potential for future use.
- Score: 5.065947993017158
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: In the digital age, seeking health advice on the Internet has become a common practice. At the same time, determining the trustworthiness of online medical content is increasingly challenging. Fact-checking has emerged as an approach to assess the veracity of factual claims using evidence from credible knowledge sources. To help advance automated Natural Language Processing (NLP) solutions for this task, in this paper we introduce a novel dataset HealthFC. It consists of 750 health-related claims in German and English, labeled for veracity by medical experts and backed with evidence from systematic reviews and clinical trials. We provide an analysis of the dataset, highlighting its characteristics and challenges. The dataset can be used for NLP tasks related to automated fact-checking, such as evidence retrieval, claim verification, or explanation generation. For testing purposes, we provide baseline systems based on different approaches, examine their performance, and discuss the findings. We show that the dataset is a challenging test bed with a high potential for future use.
Related papers
- Transforming Wearable Data into Health Insights using Large Language Model Agents [25.92023580781527]
We introduce the Personal Health Insights Agent (PHIA), an agent system to analyze and interpret behavioral health data from wearables.
Based on 650 hours of human and expert evaluation, PHIA can accurately address over 84% of factual numerical questions and more than 83% of crowd-sourced open-ended questions.
arXiv Detail & Related papers (2024-06-10T17:00:54Z) - Identifying and Aligning Medical Claims Made on Social Media with Medical Evidence [0.12277343096128711]
We study three core tasks: identifying medical claims, extracting medical vocabulary from these claims, and retrieving evidence relevant to those identified medical claims.
We propose a novel system that can generate synthetic medical claims to aid each of these core tasks.
arXiv Detail & Related papers (2024-05-18T07:50:43Z) - Leveraging text data for causal inference using electronic health records [1.4182510510164876]
This paper presents a unified framework for leveraging text data to support causal inference with electronic health data.
We show how incorporating text data in a traditional matching analysis can help strengthen the validity of an estimated treatment effect.
We believe these methods have the potential to expand the scope of secondary analysis of clinical data to domains where structured EHR data is limited.
arXiv Detail & Related papers (2023-06-09T16:06:02Z) - SPeC: A Soft Prompt-Based Calibration on Performance Variability of
Large Language Model in Clinical Notes Summarization [50.01382938451978]
We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization.
Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
arXiv Detail & Related papers (2023-03-23T04:47:46Z) - Informing clinical assessment by contextualizing post-hoc explanations
of risk prediction models in type-2 diabetes [50.8044927215346]
We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state.
We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability.
Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
arXiv Detail & Related papers (2023-02-11T18:07:11Z) - Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine [68.7814360102644]
We propose the Re$3$Writer method with retrieval-augmented generation and knowledge-grounded reasoning.
We demonstrate the effectiveness of our method in generating patient discharge instructions.
arXiv Detail & Related papers (2022-10-23T16:34:39Z) - CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking [55.75590135151682]
CHEF is the first CHinese Evidence-based Fact-checking dataset of 10K real-world claims.
The dataset covers multiple domains, ranging from politics to public health, and provides annotated evidence retrieved from the Internet.
arXiv Detail & Related papers (2022-06-06T09:11:03Z) - Healthsheet: Development of a Transparency Artifact for Health Datasets [13.57051456780329]
We introduce Healthsheet, a contextualized adaptation of the original questionnaire citegebru 2018datasheets for health-specific applications.
We work with three publicly-available healthcare datasets as our case studies.
arXiv Detail & Related papers (2022-02-26T01:05:55Z) - CREATe: Clinical Report Extraction and Annotation Technology [53.731999072534876]
Clinical case reports are written descriptions of the unique aspects of a particular clinical case.
There has been no attempt to develop an end-to-end system to annotate, index, or otherwise curate these reports.
We propose a novel computational resource platform, CREATe, for extracting, indexing, and querying the contents of clinical case reports.
arXiv Detail & Related papers (2021-02-28T16:50:14Z) - Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD
Detection On Medical Tabular Data [0.0]
We present a series of tests including a large variety of contemporary uncertainty estimation techniques.
In contrast to previous work, we design tests on realistic and clinically relevant OOD groups, and run experiments on real-world medical data.
arXiv Detail & Related papers (2020-11-06T10:41:39Z) - Assessing the Severity of Health States based on Social Media Posts [62.52087340582502]
We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state.
The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
arXiv Detail & Related papers (2020-09-21T03:45:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.