Combat COVID-19 Infodemic Using Explainable Natural Language Processing Models

• URL: http://arxiv.org/abs/2103.00747v1
• Date: Mon, 1 Mar 2021 04:28:39 GMT
• Title: Combat COVID-19 Infodemic Using Explainable Natural Language Processing Models
• Authors: Jackie Ayoub, X. Jessie Yang, Feng Zhou
• Abstract summary: We propose an explainable natural language processing model based on DistilBERT and SHAP to combat misinformation about COVID-19. Our results provided good implications in detecting misinformation about COVID-19 and improving public trust.
• Score: 15.782463163357976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Misinformation of COVID-19 is prevalent on social media as the pandemic unfolds, and the associated risks are extremely high. Thus, it is critical to detect and combat such misinformation. Recently, deep learning models using natural language processing techniques, such as BERT (Bidirectional Encoder Representations from Transformers), have achieved great successes in detecting misinformation. In this paper, we proposed an explainable natural language processing model based on DistilBERT and SHAP (Shapley Additive exPlanations) to combat misinformation about COVID-19 due to their efficiency and effectiveness. First, we collected a dataset of 984 claims about COVID-19 with fact checking. By augmenting the data using back-translation, we doubled the sample size of the dataset and the DistilBERT model was able to obtain good performance (accuracy: 0.972; areas under the curve: 0.993) in detecting misinformation about COVID-19. Our model was also tested on a larger dataset for AAAI2021 - COVID-19 Fake News Detection Shared Task and obtained good performance (accuracy: 0.938; areas under the curve: 0.985). The performance on both datasets was better than traditional machine learning models. Second, in order to boost public trust in model prediction, we employed SHAP to improve model explainability, which was further evaluated using a between-subjects experiment with three conditions, i.e., text (T), text+SHAP explanation (TSE), and text+SHAP explanation+source and evidence (TSESE). The participants were significantly more likely to trust and share information related to COVID-19 in the TSE and TSESE conditions than in the T condition. Our results provided good implications in detecting misinformation about COVID-19 and improving public trust.

Related papers

• A Comparative Study of Hybrid Models in Health Misinformation Text Classification [0.43695508295565777]
  This study evaluates the effectiveness of machine learning (ML) and deep learning (DL) models in detecting COVID-19-related misinformation on online social networks (OSNs) Our study concludes that DL and hybrid DL models are more effective than conventional ML algorithms for detecting COVID-19 misinformation on OSNs.
  arXiv  Detail & Related papers  (2024-10-08T19:43:37Z)
• Developing a Multi-variate Prediction Model For COVID-19 From Crowd-sourced Respiratory Voice Data [0.0]
  We develop a deep learning model to identify COVID-19 from voice recording data. We use the Cambridge COVID-19 Sound database which contains 893 speech samples, crowd-sourced from 4352 participants via a COVID-19 Sounds app. Based on the voice data, we develop deep learning classification models to detect COVID-19 cases.
  arXiv  Detail & Related papers  (2024-02-12T12:52:47Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [50.62245481416744]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding [51.74479522965712]
  We propose a meta-learner called the B-Learner, which can efficiently learn sharp bounds on the CATE function under limits on hidden confounding. We prove its estimates are valid, sharp, efficient, and have a quasi-oracle property with respect to the constituent estimators under more general conditions than existing methods.
  arXiv  Detail & Related papers  (2023-04-20T18:07:19Z)
• Two-Stage Classifier for COVID-19 Misinformation Detection Using BERT: a Study on Indonesian Tweets [0.15229257192293202]
  Research on COVID-19 misinformation detection in Indonesia is still scarce. In this study, we propose the two-stage classifier model using IndoBERT pre-trained language model for the Tweet misinformation detection task. The experimental results show that the combination of the BERT sequence classifier for relevance prediction and Bi-LSTM for misinformation detection outperformed other machine learning models with an accuracy of 87.02%.
  arXiv  Detail & Related papers  (2022-06-30T15:33:20Z)
• Checkovid: A COVID-19 misinformation detection system on Twitter using network and content mining perspectives [9.69596041242667]
  During the COVID-19 pandemic, social media platforms were ideal for communicating due to social isolation and quarantine. To tackle this problem, we present two COVID-19 related misinformation datasets on Twitter. We propose a misinformation detection system comprising network-based and content-based processes based on machine learning algorithms and NLP techniques.
  arXiv  Detail & Related papers  (2021-07-20T20:58:23Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)
• Deep learning-based COVID-19 pneumonia classification using chest CT images: model generalizability [54.86482395312936]
  Deep learning (DL) classification models were trained to identify COVID-19-positive patients on 3D computed tomography (CT) datasets from different countries. We trained nine identical DL-based classification models by using combinations of the datasets with a 72% train, 8% validation, and 20% test data split. The models trained on multiple datasets and evaluated on a test set from one of the datasets used for training performed better.
  arXiv  Detail & Related papers  (2021-02-18T21:14:52Z)
• Model Generalization on COVID-19 Fake News Detection [41.03093888315081]
  We aim to achieve a robust model for the COVID-19 fake-news detection task proposed at CONSTRAINT 2021 (FakeNews-19) We evaluate our models on two COVID-19 fake-news test sets.
  arXiv  Detail & Related papers  (2021-01-11T12:23:41Z)
• Understanding and Improving Lexical Choice in Non-Autoregressive Translation [98.11249019844281]
  We propose to expose the raw data to NAT models to restore the useful information of low-frequency words. Our approach pushes the SOTA NAT performance on the WMT14 English-German and WMT16 Romanian-English datasets up to 27.8 and 33.8 BLEU points, respectively.
  arXiv  Detail & Related papers  (2020-12-29T03:18:50Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.