Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning

• URL: http://arxiv.org/abs/2408.09914v1
• Date: Mon, 19 Aug 2024 11:40:20 GMT
• Title: Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning
• Authors: David Hanny, Sebastian Schmidt, Bernd Resch,
• Abstract summary: It is difficult to identify the disaster-related posts among the large amounts of unstructured data available. Previous methods often use keyword filtering, topic modelling or classification-based techniques to identify such posts. This study investigates the potential of Active Learning (AL) for identifying disaster-related Tweets.
• Score: 0.25602836891933073
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Information from social media can provide essential information for emergency response during natural disasters in near real-time. However, it is difficult to identify the disaster-related posts among the large amounts of unstructured data available. Previous methods often use keyword filtering, topic modelling or classification-based techniques to identify such posts. Active Learning (AL) presents a promising sub-field of Machine Learning (ML) that has not been used much in the field of text classification of social media content. This study therefore investigates the potential of AL for identifying disaster-related Tweets. We compare a keyword filtering approach, a RoBERTa model fine-tuned with generic data from CrisisLex, a base RoBERTa model trained with AL and a fine-tuned RoBERTa model trained with AL regarding classification performance. For testing, data from CrisisLex and manually labelled data from the 2021 flood in Germany and the 2023 Chile forest fires were considered. The results show that generic fine-tuning combined with 10 rounds of AL outperformed all other approaches. Consequently, a broadly applicable model for the identification of disaster-related Tweets could be trained with very little labelling effort. The model can be applied to use cases beyond this study and provides a useful tool for further research in social media analysis.

Related papers

• CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics [49.2719253711215]
  This study introduces a novel approach to disaster text classification by enhancing a pre-trained Large Language Model (LLM) Our methodology involves creating a comprehensive instruction dataset from disaster-related tweets, which is then used to fine-tune an open-source LLM. This fine-tuned model can classify multiple aspects of disaster-related information simultaneously, such as the type of event, informativeness, and involvement of human aid.
  arXiv  Detail & Related papers  (2024-06-16T23:01:10Z)
• CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification [51.58605842457186]
  We present a fine-grained disaster tweet classification model under the semi-supervised, few-shot learning setting. Our model, CrisisMatch, effectively classifies tweets into fine-grained classes of interest using few labeled data and large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2023-10-23T07:01:09Z)
• Unsupervised Sentiment Analysis of Plastic Surgery Social Media Posts [91.3755431537592]
  The massive collection of user posts across social media platforms is primarily untapped for artificial intelligence (AI) use cases. Natural language processing (NLP) is a subfield of AI that leverages bodies of documents, known as corpora, to train computers in human-like language understanding. This study demonstrates that the applied results of unsupervised analysis allow a computer to predict either negative, positive, or neutral user sentiment towards plastic surgery.
  arXiv  Detail & Related papers  (2023-07-05T20:16:20Z)
• Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models [2.9735729003555345]
  We propose a new targeted data reconstruction attack called the Mix And Match attack. This work highlights the importance of considering the privacy risks associated with data reconstruction attacks in classification models.
  arXiv  Detail & Related papers  (2023-06-23T21:25:38Z)
• Automated Labeling of German Chest X-Ray Radiology Reports using Deep Learning [50.591267188664666]
  We propose a deep learning-based CheXpert label prediction model, pre-trained on reports labeled by a rule-based German CheXpert model. Our results demonstrate the effectiveness of our approach, which significantly outperformed the rule-based model on all three tasks.
  arXiv  Detail & Related papers  (2023-06-09T16:08:35Z)
• VuLASTE: Long Sequence Model with Abstract Syntax Tree Embedding for vulnerability Detection [0.76146285961466]
  We build a model named VuLASTE, which regards vulnerability detection as a special text classification task. To solve the vocabulary explosion problem, VuLASTE uses a byte level BPE algorithm from natural language processing. To test our model performance on real-world source code, we build a cross-language and multi-repository vulnerability dataset.
  arXiv  Detail & Related papers  (2023-02-05T09:17:02Z)
• Selecting the suitable resampling strategy for imbalanced data classification regarding dataset properties [62.997667081978825]
  In many application domains such as medicine, information retrieval, cybersecurity, social media, etc., datasets used for inducing classification models often have an unequal distribution of the instances of each class. This situation, known as imbalanced data classification, causes low predictive performance for the minority class examples. Oversampling and undersampling techniques are well-known strategies to deal with this problem by balancing the number of examples of each class.
  arXiv  Detail & Related papers  (2021-12-15T18:56:39Z)
• Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLU [70.44344060176952]
  Intent classification is a major task in spoken language understanding (SLU) Recent works have shown that using extra data and labels can improve the OOD detection performance. This paper proposes to train a model with only IND data while supporting both IND intent classification and OOD detection.
  arXiv  Detail & Related papers  (2021-06-28T08:27:38Z)
• HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks [5.937482215664902]
  Social media content is often too noisy for direct use in any application. It is important to filter, categorize, and concisely summarize the available content to facilitate effective consumption and decision-making. We present a new large-scale dataset with 77K human-labeled tweets, sampled from a pool of 24 million tweets across 19 disaster events.
  arXiv  Detail & Related papers  (2021-04-07T12:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.