Hierarchical Multi-Label Classification of Online Vaccine Concerns
- URL: http://arxiv.org/abs/2402.01783v1
- Date: Thu, 1 Feb 2024 20:56:07 GMT
- Title: Hierarchical Multi-Label Classification of Online Vaccine Concerns
- Authors: Chloe Qinyu Zhu, Rickard Stureborg, Bhuwan Dhingra
- Abstract summary: Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic.
We explore the task of detecting vaccine concerns in online discourse using large language models (LLMs) in a zero-shot setting without the need for expensive training datasets.
- Score: 8.271202196208
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Vaccine concerns are an ever-evolving target, and can shift quickly as seen
during the COVID-19 pandemic. Identifying longitudinal trends in vaccine
concerns and misinformation might inform the healthcare space by helping public
health efforts strategically allocate resources or information campaigns. We
explore the task of detecting vaccine concerns in online discourse using large
language models (LLMs) in a zero-shot setting without the need for expensive
training datasets. Since real-time monitoring of online sources requires
large-scale inference, we explore cost-accuracy trade-offs of different
prompting strategies and offer concrete takeaways that may inform choices in
system designs for current applications. An analysis of different prompting
strategies reveals that classifying the concerns over multiple passes through
the LLM, each consisting a boolean question whether the text mentions a vaccine
concern or not, works the best. Our results indicate that GPT-4 can strongly
outperform crowdworker accuracy when compared to ground truth annotations
provided by experts on the recently introduced VaxConcerns dataset, achieving
an overall F1 score of 78.7%.
Related papers
- Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models [10.2201516537852]
We experimentally determine optimal strategies for scaling up social media content annotation for stance detection on HPV vaccine-related tweets.
In general, in-context learning outperforms fine-tuning in stance detection for HPV vaccine social media content.
arXiv Detail & Related papers (2024-11-22T04:19:32Z) - Leveraging Large Language Models and Weak Supervision for Social Media
data annotation: an evaluation using COVID-19 self-reported vaccination
tweets [1.9988653168573556]
Social media platforms have become a popular medium for discussions on vaccine-related topics.
In this study, we evaluate the usage of Large Language Models, in this case GPT-4, and weak supervision, to identify COVID-19 vaccine-related tweets.
arXiv Detail & Related papers (2023-09-12T18:18:23Z) - Self-Verification Improves Few-Shot Clinical Information Extraction [73.6905567014859]
Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning.
They still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health.
Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs.
arXiv Detail & Related papers (2023-05-30T22:05:11Z) - Evaluating COVID-19 vaccine allocation policies using Bayesian $m$-top
exploration [53.122045119395594]
We present a novel technique for evaluating vaccine allocation strategies using a multi-armed bandit framework.
$m$-top exploration allows the algorithm to learn $m$ policies for which it expects the highest utility.
We consider the Belgian COVID-19 epidemic using the individual-based model STRIDE, where we learn a set of vaccination policies.
arXiv Detail & Related papers (2023-01-30T12:22:30Z) - Dense Feature Memory Augmented Transformers for COVID-19 Vaccination
Search Classification [60.49594822215981]
This paper presents a classification model for detecting COVID-19 vaccination related search queries.
We propose a novel approach of considering dense features as memory tokens that the model can attend to.
We show that this new modeling approach enables a significant improvement to the Vaccine Search Insights (VSI) task.
arXiv Detail & Related papers (2022-12-16T13:57:41Z) - Classifying COVID-19 vaccine narratives [7.784326429148358]
Vaccine hesitancy is widespread, despite the government's information campaigns and the efforts of the World Health Organisation (WHO)
This paper introduces a novel vaccine narrative classification task, which categorises COVID-19 vaccine claims into one of seven categories.
The paper also presents a neural vaccine narrative classifier that achieves an accuracy of 84% under cross-validation.
arXiv Detail & Related papers (2022-07-18T11:37:47Z) - Insta-VAX: A Multimodal Benchmark for Anti-Vaccine and Misinformation
Posts Detection on Social Media [32.252687203366605]
Anti-vaccine posts on social media have been shown to create confusion and reduce the publics confidence in vaccines.
Insta-VAX is a new multi-modal dataset consisting of a sample of 64,957 Instagram posts related to human vaccines.
arXiv Detail & Related papers (2021-12-15T20:34:57Z) - Automatic Detection of COVID-19 Vaccine Misinformation with Graph Link
Prediction [2.0625936401496237]
Vaccine hesitancy fueled by social media misinformation about COVID-19 vaccines became a major hurdle.
We present CoVaxLies, a new dataset of tweets judged relevant to several misinformation targets about COVID-19 vaccines.
Our method organizes CoVaxLies in a Misinformation Knowledge Graph as it casts misinformation detection as a graph link prediction problem.
arXiv Detail & Related papers (2021-08-04T23:27:10Z) - Predicting Infectiousness for Proactive Contact Tracing [75.62186539860787]
Large-scale digital contact tracing is a potential solution to resume economic and social activity while minimizing spread of the virus.
Various DCT methods have been proposed, each making trade-offs between privacy, mobility restrictions, and public health.
This paper develops and test methods that can be deployed to a smartphone to proactively predict an individual's infectiousness.
arXiv Detail & Related papers (2020-10-23T17:06:07Z) - Epidemic mitigation by statistical inference from contact tracing data [61.04165571425021]
We develop Bayesian inference methods to estimate the risk that an individual is infected.
We propose to use probabilistic risk estimation in order to optimize testing and quarantining strategies for the control of an epidemic.
Our approaches translate into fully distributed algorithms that only require communication between individuals who have recently been in contact.
arXiv Detail & Related papers (2020-09-20T12:24:45Z) - COVI White Paper [67.04578448931741]
Contact tracing is an essential tool to change the course of the Covid-19 pandemic.
We present an overview of the rationale, design, ethical considerations and privacy strategy of COVI,' a Covid-19 public peer-to-peer contact tracing and risk awareness mobile application developed in Canada.
arXiv Detail & Related papers (2020-05-18T07:40:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.