Model Generalization on COVID-19 Fake News Detection

• URL: http://arxiv.org/abs/2101.03841v1
• Date: Mon, 11 Jan 2021 12:23:41 GMT
• Title: Model Generalization on COVID-19 Fake News Detection
• Authors: Yejin Bang, Etsuko Ishii, Samuel Cahyawijaya, Ziwei Ji, Pascale Fung
• Abstract summary: We aim to achieve a robust model for the COVID-19 fake-news detection task proposed at CONSTRAINT 2021 (FakeNews-19) We evaluate our models on two COVID-19 fake-news test sets.
• Score: 41.03093888315081
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Amid the pandemic COVID-19, the world is facing unprecedented infodemic with the proliferation of both fake and real information. Considering the problematic consequences that the COVID-19 fake-news have brought, the scientific community has put effort to tackle it. To contribute to this fight against the infodemic, we aim to achieve a robust model for the COVID-19 fake-news detection task proposed at CONSTRAINT 2021 (FakeNews-19) by taking two separate approaches: 1) fine-tuning transformers based language models with robust loss functions and 2) removing harmful training instances through influence calculation. We further evaluate the robustness of our models by evaluating on different COVID-19 misinformation test set (Tweets-19) to understand model generalization ability. With the first approach, we achieve 98.13% for weighted F1 score (W-F1) for the shared task, whereas 38.18% W-F1 on the Tweets-19 highest. On the contrary, by performing influence data cleansing, our model with 99% cleansing percentage can achieve 54.33% W-F1 score on Tweets-19 with a trade-off. By evaluating our models on two COVID-19 fake-news test sets, we suggest the importance of model generalization ability in this task to step forward to tackle the COVID-19 fake-news problem in online social media platforms.

Related papers

• Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE [93.98660272309974]
  This report briefly describes our submission Vega v1 on the General Language Understanding Evaluation leaderboard. GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference. With our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3.
  arXiv  Detail & Related papers  (2023-02-18T09:26:35Z)
• Combat COVID-19 Infodemic Using Explainable Natural Language Processing Models [15.782463163357976]
  We propose an explainable natural language processing model based on DistilBERT and SHAP to combat misinformation about COVID-19. Our results provided good implications in detecting misinformation about COVID-19 and improving public trust.
  arXiv  Detail & Related papers  (2021-03-01T04:28:39Z)
• A transformer based approach for fighting COVID-19 fake news [0.8793721044482612]
  COVID-19 is the first pandemic in history when humanity is the most technologically advanced. Fake news and misinformation regarding this virus is also available to people and causing some massive problems. We present our solution for the "Constraint@AAAI2021 - COVID19 Fake News Detection in English" challenge in this work.
  arXiv  Detail & Related papers  (2021-01-28T14:43:42Z)
• End-2-End COVID-19 Detection from Breath & Cough Audio [68.41471917650571]
  We demonstrate the first attempt to diagnose COVID-19 using end-to-end deep learning from a crowd-sourced dataset of audio samples. We introduce a novel modelling strategy using a custom deep neural network to diagnose COVID-19 from a joint breath and cough representation.
  arXiv  Detail & Related papers  (2021-01-07T01:13:00Z)
• g2tmn at Constraint@AAAI2021: Exploiting CT-BERT and Ensembling Learning for COVID-19 Fake News Detection [0.0]
  We present our results at the Constraint@AAAI2021 Shared Task: COVID-19 Fake News Detection in English. We propose our approach using the transformer-based ensemble of COVID-Twitter-BERT (CT-BERT) models. As a result, our best model achieved the weighted F1-score of 98.69 on the test set.
  arXiv  Detail & Related papers  (2020-12-22T12:43:12Z)
• To be Robust or to be Fair: Towards Fairness in Adversarial Training [83.42241071662897]
  We find that adversarial training algorithms tend to introduce severe disparity of accuracy and robustness between different groups of data. We propose a Fair-Robust-Learning (FRL) framework to mitigate this unfairness problem when doing adversarial defenses.
  arXiv  Detail & Related papers  (2020-10-13T02:21:54Z)
• Not-NUTs at W-NUT 2020 Task 2: A BERT-based System in Identifying Informative COVID-19 English Tweets [0.0]
  We propose a model that, given an English tweet, automatically identifies whether that tweet bears informative content regarding COVID-19 or not. We have achieved competitive results that are only shy of those by top performing teams by roughly 1% in terms of F1 score on the informative class.
  arXiv  Detail & Related papers  (2020-09-14T15:49:16Z)
• BANANA at WNUT-2020 Task 2: Identifying COVID-19 Information on Twitter by Combining Deep Learning and Transfer Learning Models [0.0]
  This paper describes our prediction system for WNUT-2020 Task 2: Identification of Informative COVID-19 English Tweets. The dataset for this task contains size 10,000 tweets in English labeled by humans. The experimental result indicates that we have achieved F1 for the INFORMATIVE label on our systems at 88.81% on the test set.
  arXiv  Detail & Related papers  (2020-09-06T08:24:55Z)
• Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning [60.784641458579124]
  We show that fine-tuning effectively enhances model robustness under white-box FGSM attacks. We also propose a black-box attack method for transfer learning models which attacks the target model with the adversarial examples produced by its source model. To systematically measure the effect of both white-box and black-box attacks, we propose a new metric to evaluate how transferable are the adversarial examples produced by a source model to a target model.
  arXiv  Detail & Related papers  (2020-08-25T15:04:32Z)
• Deep F-measure Maximization for End-to-End Speech Understanding [52.36496114728355]
  We propose a differentiable approximation to the F-measure and train the network with this objective using standard backpropagation. We perform experiments on two standard fairness datasets, Adult, Communities and Crime, and also on speech-to-intent detection on the ATIS dataset and speech-to-image concept classification on the Speech-COCO dataset. In all four of these tasks, F-measure results in improved micro-F1 scores, with absolute improvements of up to 8% absolute, as compared to models trained with the cross-entropy loss function.
  arXiv  Detail & Related papers  (2020-08-08T03:02:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.