Analyzing COVID-19 Tweets with Transformer-based Language Models

• URL: http://arxiv.org/abs/2104.10259v1
• Date: Tue, 20 Apr 2021 21:45:33 GMT
• Title: Analyzing COVID-19 Tweets with Transformer-based Language Models
• Authors: Philip Feldman, Sim Tiwari, Charissa S. L. Cheah, James R. Foulds, Shimei Pan
• Abstract summary: We train a set of GPT models on several COVID-19 tweet corpora. We then use prompt-based queries to probe these models to reveal insights into the opinions of social media users. Results resemble polling the public on diverse social, political and public health issues.
• Score: 11.726315753231667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes a method for using Transformer-based Language Models (TLMs) to understand public opinion from social media posts. In this approach, we train a set of GPT models on several COVID-19 tweet corpora. We then use prompt-based queries to probe these models to reveal insights into the opinions of social media users. We demonstrate how this approach can be used to produce results which resemble polling the public on diverse social, political and public health issues. The results on the COVID-19 tweet data show that transformer language models are promising tools that can help us understand public opinions on social media at scale.

Related papers

• Extending a Parliamentary Corpus with MPs' Tweets: Automatic Annotation and Evaluation Using MultiParTweet [37.712808276294]
  We present MultiParTweet, a tweet corpus from X that connects politicians' social media discourse with German political corpus GerParCor.<n>MultiParTweet contains 39 546 tweets, including 19 056 media items.<n>We enrich the annotation with nine text-based models and one vision-language model (VLM) to annotate MultiParTweet with emotion, sentiment, and topic annotations.
  arXiv  Detail & Related papers  (2025-12-12T13:55:02Z)
• Transformer Explainer: Interactive Learning of Text-Generative Models [65.91049787390692]
  Transformer Explainer is an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. It runs a live GPT-2 instance locally in the user's browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together.
  arXiv  Detail & Related papers  (2024-08-08T17:49:07Z)
• ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness [3.126776200660494]
  We show that ChatGPT-created paraphrases are more diverse and lead to at least as robust models. Traditionally, crowdsourcing has been used for acquiring solutions to a wide variety of human-intelligence tasks.
  arXiv  Detail & Related papers  (2023-05-22T11:46:32Z)
• Detection of depression on social networks using transformers and ensembles [3.997016051942249]
  We build several large pre-trained language model based classifiers for depression detection from social media posts. We analyze the performance of our models on two data sets of posts from social platforms Reddit and Twitter.
  arXiv  Detail & Related papers  (2023-05-09T10:21:14Z)
• TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter [31.698196219228024]
  We present TwHIN-BERT, a multilingual language model productionized at Twitter. Our model is trained on 7 billion tweets covering over 100 distinct languages. We evaluate our model on various multilingual social recommendation and semantic understanding tasks.
  arXiv  Detail & Related papers  (2022-09-15T19:01:21Z)
• 5q032e@SMM4H'22: Transformer-based classification of premise in tweets related to COVID-19 [2.3931689873603603]
  We propose a predictive model based on transformer architecture to classify the presence of premise in Twitter texts. Our experiments on a Twitter dataset showed that RoBERTa is superior to the other transformer models in the case of the premise prediction task.
  arXiv  Detail & Related papers  (2022-09-08T14:46:28Z)
• BERTuit: Understanding Spanish language in Twitter through a native transformer [70.77033762320572]
  We present bfBERTuit, the larger transformer proposed so far for Spanish language, pre-trained on a massive dataset of 230M Spanish tweets. Our motivation is to provide a powerful resource to better understand Spanish Twitter and to be used on applications focused on this social network.
  arXiv  Detail & Related papers  (2022-04-07T14:28:51Z)
• A New Generation of Perspective API: Efficient Multilingual Character-level Transformers [66.9176610388952]
  We present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings.
  arXiv  Detail & Related papers  (2022-02-22T20:55:31Z)
• Exploiting BERT For Multimodal Target SentimentClassification Through Input Space Translation [75.82110684355979]
  We introduce a two-stream model that translates images in input space using an object-aware transformer. We then leverage the translation to construct an auxiliary sentence that provides multimodal information to a language model. We achieve state-of-the-art performance on two multimodal Twitter datasets.
  arXiv  Detail & Related papers  (2021-08-03T18:02:38Z)
• COVID-19 Tweets Analysis through Transformer Language Models [0.0]
  In this study, we perform an in-depth, fine-grained sentiment analysis of tweets in COVID-19. A trained transformer model is able to correctly predict, with high accuracy, the tone of a tweet. We then leverage this model for predicting tones for 200,000 tweets on COVID-19.
  arXiv  Detail & Related papers  (2021-02-27T12:06:33Z)
• Improving Sentiment Analysis over non-English Tweets using Multilingual Transformers and Automatic Translation for Data-Augmentation [77.69102711230248]
  We propose the use of a multilingual transformer model, that we pre-train over English tweets and apply data-augmentation using automatic translation to adapt the model to non-English languages. Our experiments in French, Spanish, German and Italian suggest that the proposed technique is an efficient way to improve the results of the transformers over small corpora of tweets in a non-English language.
  arXiv  Detail & Related papers  (2020-10-07T15:44:55Z)
• Multimodal Matching Transformer for Live Commenting [97.06576354830736]
  Automatic live commenting aims to provide real-time comments on videos for viewers. Recent work on this task adopts encoder-decoder models to generate comments. We propose a multimodal matching transformer to capture the relationships among comments, vision, and audio.
  arXiv  Detail & Related papers  (2020-02-07T07:19:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.