FloodBrain: Flood Disaster Reporting by Web-based Retrieval Augmented Generation with an LLM

• URL: http://arxiv.org/abs/2311.02597v1
• Date: Sun, 5 Nov 2023 08:34:26 GMT
• Title: FloodBrain: Flood Disaster Reporting by Web-based Retrieval Augmented Generation with an LLM
• Authors: Grace Colverd, Paul Darm, Leonard Silverberg, and Noah Kasmanoff
• Abstract summary: We introduce a sophisticated pipeline embodied in our tool FloodBrain (floodbrain.com) Our pipeline assimilates information from web search results to produce detailed and accurate reports on flood events. We find a notable correlation between the scores assigned by GPT-4 and the scores given by human evaluators when comparing our generated reports to human-authored ones.
• Score: 0.9374652839580183
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Fast disaster impact reporting is crucial in planning humanitarian assistance. Large Language Models (LLMs) are well known for their ability to write coherent text and fulfill a variety of tasks relevant to impact reporting, such as question answering or text summarization. However, LLMs are constrained by the knowledge within their training data and are prone to generating inaccurate, or "hallucinated", information. To address this, we introduce a sophisticated pipeline embodied in our tool FloodBrain (floodbrain.com), specialized in generating flood disaster impact reports by extracting and curating information from the web. Our pipeline assimilates information from web search results to produce detailed and accurate reports on flood events. We test different LLMs as backbones in our tool and compare their generated reports to human-written reports on different metrics. Similar to other studies, we find a notable correlation between the scores assigned by GPT-4 and the scores given by human evaluators when comparing our generated reports to human-authored ones. Additionally, we conduct an ablation study to test our single pipeline components and their relevancy for the final reports. With our tool, we aim to advance the use of LLMs for disaster impact reporting and reduce the time for coordination of humanitarian efforts in the wake of flood disasters.

Related papers

• TextMine: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action [4.990484801014005]
  Humanitarian Mine Action (HMA) addresses the challenge of detecting and removing landmines from conflict regions.<n>Much of the life-saving operational knowledge produced by HMA agencies is buried in unstructured reports.<n>To address this issue, we propose TextMine: the first dataset, evaluation framework and ontology-guided large language model (LLM) pipeline.
  arXiv  Detail & Related papers  (2025-09-18T15:55:19Z)
• A Multimodal, Multilingual, and Multidimensional Pipeline for Fine-grained Crowdsourcing Earthquake Damage Evaluation [5.5809992003597575]
  Rapid, fine-grained disaster damage assessment is essential for effective emergency response, yet remains challenging due to limited ground sensors and delays in official reporting.<n>Social media provides a rich, real-time source of human-centric observations, but its multimodal and unstructured nature presents challenges for traditional analytical methods.<n>We propose a structured Multimodal, Multilingual, and Multidimensional (3M) pipeline that leverages multimodal large language models (MLLMs) to assess disaster impacts.
  arXiv  Detail & Related papers  (2025-06-03T20:07:25Z)
• DisasterQA: A Benchmark for Assessing the performance of LLMs in Disaster Response [0.0]
  We evaluate the capabilities of Large Language Models (LLMs) in disaster response knowledge. The benchmark covers a wide range of disaster response topics. The results indicate that LLMs require improvement on disaster response knowledge.
  arXiv  Detail & Related papers  (2024-10-09T00:13:06Z)
• AutoRG-Brain: Grounded Report Generation for Brain MRI [57.22149878985624]
  Radiologists are tasked with interpreting a large number of images in a daily base, with the responsibility of generating corresponding reports. This demanding workload elevates the risk of human error, potentially leading to treatment delays, increased healthcare costs, revenue loss, and operational inefficiencies. We initiate a series of work on grounded Automatic Report Generation (AutoRG) This system supports the delineation of brain structures, the localization of anomalies, and the generation of well-organized findings.
  arXiv  Detail & Related papers  (2024-07-23T17:50:00Z)
• End-To-End Causal Effect Estimation from Unstructured Natural Language Data [23.484226791467478]
  We show how large, diverse observational text data can be mined with large language models (LLMs) to produce inexpensive causal effect estimates. We introduce NATURAL, a novel family of causal effect estimators built with LLMs that operate over datasets of unstructured text. Our results suggest that unstructured text data is a rich source of causal effect information, and NATURAL is a first step towards an automated pipeline to tap this resource.
  arXiv  Detail & Related papers  (2024-07-09T16:38:48Z)
• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• Are you still on track!? Catching LLM Task Drift with Activations [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users. We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set. We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Monitoring Critical Infrastructure Facilities During Disasters Using Large Language Models [8.17728833322492]
  Critical Infrastructure Facilities (CIFs) are vital for the functioning of a community, especially during large-scale emergencies. In this paper, we explore a potential application of Large Language Models (LLMs) to monitor the status of CIFs affected by natural disasters through information disseminated in social media networks. We analyze social media data from two disaster events in two different countries to identify reported impacts to CIFs as well as their impact severity and operational status.
  arXiv  Detail & Related papers  (2024-04-18T19:41:05Z)
• Causality Extraction from Nuclear Licensee Event Reports Using a Hybrid Framework [3.1139106894905972]
  This paper proposed a hybrid framework for causality detection and extraction from nuclear licensee event reports. We compiled an LER corpus with 20,129 text samples for causality analysis, developed an interactive tool for labeling cause effect pairs, and built a deep-learning-based approach for causal relation detection.
  arXiv  Detail & Related papers  (2024-04-08T16:39:34Z)
• CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification [51.58605842457186]
  We present a fine-grained disaster tweet classification model under the semi-supervised, few-shot learning setting. Our model, CrisisMatch, effectively classifies tweets into fine-grained classes of interest using few labeled data and large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2023-10-23T07:01:09Z)
• Source Attribution for Large Language Model-Generated Data [57.85840382230037]
  It is imperative to be able to perform source attribution by identifying the data provider who contributed to the generation of a synthetic text. We show that this problem can be tackled by watermarking. We propose a source attribution framework that satisfies these key properties due to our algorithmic designs.
  arXiv  Detail & Related papers  (2023-10-01T12:02:57Z)
• On the Risk of Misinformation Pollution with Large Language Models [127.1107824751703]
  We investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation. Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of Open-Domain Question Answering (ODQA) systems.
  arXiv  Detail & Related papers  (2023-05-23T04:10:26Z)
• Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning [3.964047152162558]
  We investigate the problem of localized flood detection using the social sensing model (Twitter) This study can immensely help in providing the flood-related updates and notifications to the city officials for emergency decision making, rescue operations, and early warnings, etc.
  arXiv  Detail & Related papers  (2020-02-10T20:17:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.