HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability

• URL: http://arxiv.org/abs/2407.00668v2
• Date: Wed, 3 Jul 2024 15:18:40 GMT
• Title: HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability
• Authors: Yanfang Chen, Ding Chen, Shichao Song, Simin Niu, Hanyu Wang, Zeyun Tang, Feiyu Xiong, Zhiyu Li,
• Abstract summary: This paper builds a dataset containing 1.12 million health-related rumors (HealthRCN) through web scraping of common health-related questions. We propose retrieval-augmented large language models for Chinese health rumor detection and explainability (HRDE)
• Score: 6.800433977880405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As people increasingly prioritize their health, the speed and breadth of health information dissemination on the internet have also grown. At the same time, the presence of false health information (health rumors) intermingled with genuine content poses a significant potential threat to public health. However, current research on Chinese health rumors still lacks a large-scale, public, and open-source dataset of health rumor information, as well as effective and reliable rumor detection methods. This paper addresses this gap by constructing a dataset containing 1.12 million health-related rumors (HealthRCN) through web scraping of common health-related questions and a series of data processing steps. HealthRCN is the largest known dataset of Chinese health information rumors to date. Based on this dataset, we propose retrieval-augmented large language models for Chinese health rumor detection and explainability (HRDE). This model leverages retrieved relevant information to accurately determine whether the input health information is a rumor and provides explanatory responses, effectively aiding users in verifying the authenticity of health information. In evaluation experiments, we compared multiple models and found that HRDE outperformed them all, including GPT-4-1106-Preview, in rumor detection accuracy and answer quality. HRDE achieved an average accuracy of 91.04% and an F1 score of 91.58%.

Related papers

• Do LLMs Provide Consistent Answers to Health-Related Questions across Languages? [14.87110905165928]
  We examine the consistency of responses provided by Large Language Models (LLMs) to health-related questions across English, German, Turkish, and Chinese. We reveal significant inconsistencies in responses that could spread healthcare misinformation. Our findings emphasize the need for improved cross-lingual alignment to ensure accurate and equitable healthcare information.
  arXiv  Detail & Related papers  (2025-01-24T18:51:26Z)
• CHBench: A Chinese Dataset for Evaluating Health in Large Language Models [19.209493319541693]
  We present CHBench, the first comprehensive Chinese Health-related Benchmark. CHBench includes 6,493 entries related to mental health and 2,999 entries focused on physical health. This dataset serves as a foundation for evaluating Chinese LLMs' capacity to comprehend and generate accurate health-related information.
  arXiv  Detail & Related papers  (2024-09-24T05:44:46Z)
• Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data [43.48422400822597]
  Large language models (LLMs) are capable of many natural language tasks, yet they are far from perfect. This paper investigates the capacity of LLMs to make inferences about health based on contextual information. We present a comprehensive evaluation of 12 state-of-the-art LLMs with prompting and fine-tuning techniques on four public health datasets.
  arXiv  Detail & Related papers  (2024-01-12T19:40:11Z)
• RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging [67.02500668641831]
  Deep learning models trained on noisy datasets are sensitive to the noise type and lead to less generalization on unseen samples. We propose a Robust Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlapping subsets of training data.
  arXiv  Detail & Related papers  (2022-10-15T22:32:20Z)
• COLD: A Benchmark for Chinese Offensive Language Detection [54.60909500459201]
  We use COLDataset, a Chinese offensive language dataset with 37k annotated sentences. We also propose textscCOLDetector to study output offensiveness of popular Chinese language models. Our resources and analyses are intended to help detoxify the Chinese online communities and evaluate the safety performance of generative language models.
  arXiv  Detail & Related papers  (2022-01-16T11:47:23Z)
• Cross-lingual COVID-19 Fake News Detection [54.125563009333995]
  We make the first attempt to detect COVID-19 misinformation in a low-resource language (Chinese) only using the fact-checked news in a high-resource language (English) We propose a deep learning framework named CrossFake to jointly encode the cross-lingual news body texts and capture the news content. Empirical results on our dataset demonstrate the effectiveness of CrossFake under the cross-lingual setting.
  arXiv  Detail & Related papers  (2021-10-13T04:44:02Z)
• Know it to Defeat it: Exploring Health Rumor Characteristics and Debunking Efforts on Chinese Social Media during COVID-19 Crisis [65.74516068984232]
  We conduct a comprehensive analysis of four months of rumor-related online discussion during COVID-19 on Weibo, a Chinese microblogging site. Results suggest that the dread (cause fear) type of health rumors provoked significantly more discussions and lasted longer than the wish (raise hope) type. We show the efficacy of debunking in suppressing rumor discussions, which is time-sensitive and varies across rumor types and debunkers.
  arXiv  Detail & Related papers  (2021-09-25T14:02:29Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Drink Bleach or Do What Now? Covid-HeRA: A Study of Risk-Informed Health Decision Making in the Presence of COVID-19 Misinformation [23.449057978351945]
  We frame health misinformation as a risk assessment task. We study the severity of each misinformation story and how readers perceive this severity. We evaluate several traditional and state-of-the-art models and show there is a significant gap in performance.
  arXiv  Detail & Related papers  (2020-10-17T08:34:57Z)
• Assessing the Severity of Health States based on Social Media Posts [62.52087340582502]
  We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
  arXiv  Detail & Related papers  (2020-09-21T03:45:14Z)
• Ginger Cannot Cure Cancer: Battling Fake Health News with a Comprehensive Data Repository [40.76937321931461]
  Massive fake health news which is spreading over the Internet, has become a severe threat to public health. We construct a comprehensive repository, FakeHealth, which includes news contents with rich features, news reviews with detailed explanations, social engagements and a user-user social network.
  arXiv  Detail & Related papers  (2020-01-27T17:27:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.