CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics

• URL: http://arxiv.org/abs/2406.15477v1
• Date: Sun, 16 Jun 2024 23:01:10 GMT
• Title: CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics
• Authors: Kai Yin, Chengkai Liu, Ali Mostafavi, Xia Hu,
• Abstract summary: This study introduces a novel approach to disaster text classification by enhancing a pre-trained Large Language Model (LLM) Our methodology involves creating a comprehensive instruction dataset from disaster-related tweets, which is then used to fine-tune an open-source LLM. This fine-tuned model can classify multiple aspects of disaster-related information simultaneously, such as the type of event, informativeness, and involvement of human aid.
• Score: 49.2719253711215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the field of crisis/disaster informatics, social media is increasingly being used for improving situational awareness to inform response and relief efforts. Efficient and accurate text classification tools have been a focal area of investigation in crisis informatics. However, current methods mostly rely on single-label text classification models, which fails to capture different insights embedded in dynamic and multifaceted disaster-related social media data. This study introduces a novel approach to disaster text classification by enhancing a pre-trained Large Language Model (LLM) through instruction fine-tuning targeted for multi-label classification of disaster-related tweets. Our methodology involves creating a comprehensive instruction dataset from disaster-related tweets, which is then used to fine-tune an open-source LLM, thereby embedding it with disaster-specific knowledge. This fine-tuned model can classify multiple aspects of disaster-related information simultaneously, such as the type of event, informativeness, and involvement of human aid, significantly improving the utility of social media data for situational awareness in disasters. The results demonstrate that this approach enhances the categorization of critical information from social media posts, thereby facilitating a more effective deployment for situational awareness during emergencies. This research paves the way for more advanced, adaptable, and robust disaster management tools, leveraging the capabilities of LLMs to improve real-time situational awareness and response strategies in disaster scenarios.

Related papers

• A Social Context-aware Graph-based Multimodal Attentive Learning Framework for Disaster Content Classification during Emergencies [0.0]
  CrisisSpot is a method that captures complex relationships between textual and visual modalities. IDEA captures both harmonious and contrasting patterns within the data to enhance multimodal interactions. CrisisSpot achieved an average F1-score gain of 9.45% and 5.01% compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-11T13:51:46Z)
• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification [51.58605842457186]
  We present a fine-grained disaster tweet classification model under the semi-supervised, few-shot learning setting. Our model, CrisisMatch, effectively classifies tweets into fine-grained classes of interest using few labeled data and large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2023-10-23T07:01:09Z)
• DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank [52.20298962359658]
  In crisis events, people often use social media platforms such as Twitter to disseminate information about the situation, warnings, advice, and support. fully-supervised approaches require annotating vast amounts of data and are impractical due to limited response time. Semi-supervised models can be biased, performing moderately well for certain classes while performing extremely poorly for others. We propose a simple but effective debiasing method, DeCrisisMB, that utilizes a Memory Bank to store and perform equal sampling for generated pseudo-labels from each class at each training.
  arXiv  Detail & Related papers  (2023-10-23T05:25:51Z)
• IKDSumm: Incorporating Key-phrases into BERT for extractive Disaster Tweet Summarization [5.299958874647294]
  We propose a disaster-specific tweet summarization framework, IKDSumm. IKDSumm identifies the crucial and important information from each tweet related to a disaster through key-phrases of that tweet. We utilize these key-phrases to automatically generate a summary of the tweets.
  arXiv  Detail & Related papers  (2023-05-19T11:05:55Z)
• Continual Distributed Learning for Crisis Management [0.0]
  Social media platforms such as Twitter provide an excellent resource for mobile communication during emergency events. Data present in such situations is ever-changing, and resources during such crisis may not be readily available. A low resource, continually learning system must be developed to incorporate and make NLP models robust against noisy and unordered data.
  arXiv  Detail & Related papers  (2021-04-26T21:01:29Z)
• Event-Related Bias Removal for Real-time Disaster Events [67.2965372987723]
  Social media has become an important tool to share information about crisis events such as natural disasters and mass attacks. Detecting actionable posts that contain useful information requires rapid analysis of huge volume of data in real-time. We train an adversarial neural model to remove latent event-specific biases and improve the performance on tweet importance classification.
  arXiv  Detail & Related papers  (2020-11-02T02:03:07Z)
• Predicting Themes within Complex Unstructured Texts: A Case Study on Safeguarding Reports [66.39150945184683]
  We focus on the problem of automatically identifying the main themes in a safeguarding report using supervised classification approaches. Our results show the potential of deep learning models to simulate subject-expert behaviour even for complex tasks with limited labelled data.
  arXiv  Detail & Related papers  (2020-10-27T19:48:23Z)
• Social Media Information Sharing for Natural Disaster Response [0.0]
  Social media has become an essential channel for posting disaster-related information, which provide governments and relief agencies real-time data for better disaster management. This paper aims to improve disaster relief efficiency via mining and analyzing social media data like public attitudes towards disaster response and public demands for targeted relief supplies during different types of disasters.
  arXiv  Detail & Related papers  (2020-05-08T21:11:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.