CrisiText: A dataset of warning messages for LLM training in emergency communication

• URL: http://arxiv.org/abs/2510.09243v2
• Date: Mon, 13 Oct 2025 18:43:09 GMT
• Title: CrisiText: A dataset of warning messages for LLM training in emergency communication
• Authors: Giacomo Gonella, Gian Maria Campedelli, Stefano Menini, Marco Guerini,
• Abstract summary: We present CrisiText, the first large-scale dataset for the generation of warning messages across 13 different types of crisis scenarios.<n>The dataset contains more than 400,000 warning messages aimed at assisting civilians during and after such events.
• Score: 8.549683389697126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effectively identifying threats and mitigating their potential damage during crisis situations, such as natural disasters or violent attacks, is paramount for safeguarding endangered individuals. To tackle these challenges, AI has been used in assisting humans in emergency situations. Still, the use of NLP techniques remains limited and mostly focuses on classification tasks. The significant potential of timely warning message generation using NLG architectures, however, has been largely overlooked. In this paper we present CrisiText, the first large-scale dataset for the generation of warning messages across 13 different types of crisis scenarios. The dataset contains more than 400,000 warning messages (spanning almost 18,000 crisis situations) aimed at assisting civilians during and after such events. To generate the dataset, we started from existing crisis descriptions and created chains of events related to the scenarios. Each event was then paired with a warning message. The generations follow experts' written guidelines to ensure correct terminology and factuality of their suggestions. Additionally, each message is accompanied by three suboptimal warning types to allow for the study of different NLG approaches. To this end, we conducted a series of experiments comparing supervised fine-tuning setups with preference alignment, zero-shot, and few-shot approaches. We further assessed model performance in out-of-distribution scenarios and evaluated the effectiveness of an automatic post-editor.

Related papers

• CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics [50.122541222825156]
  This study introduces a novel approach to disaster text classification by enhancing a pre-trained Large Language Model (LLM)<n>Our methodology involves creating a comprehensive instruction dataset from disaster-related tweets, which is then used to fine-tune an open-source LLM.<n>This fine-tuned model can classify multiple aspects of disaster-related information simultaneously, such as the type of event, informativeness, and involvement of human aid.
  arXiv  Detail & Related papers  (2024-06-16T23:01:10Z)
• CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification [51.58605842457186]
  We present a fine-grained disaster tweet classification model under the semi-supervised, few-shot learning setting. Our model, CrisisMatch, effectively classifies tweets into fine-grained classes of interest using few labeled data and large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2023-10-23T07:01:09Z)
• DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank [52.20298962359658]
  In crisis events, people often use social media platforms such as Twitter to disseminate information about the situation, warnings, advice, and support. fully-supervised approaches require annotating vast amounts of data and are impractical due to limited response time. Semi-supervised models can be biased, performing moderately well for certain classes while performing extremely poorly for others. We propose a simple but effective debiasing method, DeCrisisMB, that utilizes a Memory Bank to store and perform equal sampling for generated pseudo-labels from each class at each training.
  arXiv  Detail & Related papers  (2023-10-23T05:25:51Z)
• CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts [3.690904966341072]
  Social media platforms play an essential role in crisis communication, but analyzing crisis-related social media texts is challenging due to their informal nature. This study introduces CrisisTransformers, an ensemble of pre-trained language models and sentence encoders trained on an extensive corpus of over 15 billion word tokens from tweets.
  arXiv  Detail & Related papers  (2023-09-11T14:36:16Z)
• Coping with low data availability for social media crisis message categorisation [3.0255457622022495]
  This thesis focuses on addressing the challenge of low data availability when categorising crisis messages for emergency response. It first presents domain adaptation as a solution for this problem, which involves learning a categorisation model from annotated data from past crisis events. In many-to-many adaptation, where the model is trained on multiple past events and adapted to multiple ongoing events, a multi-task learning approach is proposed.
  arXiv  Detail & Related papers  (2023-05-26T19:08:24Z)
• CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization [62.77066949111921]
  This paper presents CrisisLTLSum, the largest dataset of local crisis event timelines available to date. CrisisLTLSum contains 1,000 crisis event timelines across four domains: wildfires, local fires, traffic, and storms. Our initial experiments indicate a significant gap between the performance of strong baselines compared to the human performance on both tasks.
  arXiv  Detail & Related papers  (2022-10-25T17:32:40Z)
• Event-Related Bias Removal for Real-time Disaster Events [67.2965372987723]
  Social media has become an important tool to share information about crisis events such as natural disasters and mass attacks. Detecting actionable posts that contain useful information requires rapid analysis of huge volume of data in real-time. We train an adversarial neural model to remove latent event-specific biases and improve the performance on tweet importance classification.
  arXiv  Detail & Related papers  (2020-11-02T02:03:07Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)
• CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding [2.7718973516070684]
  We propose an end-to-end transformer-based model for two crisis classification tasks, namely crisis detection and crisis recognition. We also proposed Crisis2Vec, an attention-based, document-level contextual embedding architecture for crisis embedding.
  arXiv  Detail & Related papers  (2020-05-11T09:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.