LLM-Powered Flood Depth Estimation from Social Media Imagery: A Vision-Language Model Framework with Mechanistic Interpretability for Transportation Resilience

• URL: http://arxiv.org/abs/2603.17108v2
• Date: Mon, 23 Mar 2026 20:39:33 GMT
• Title: LLM-Powered Flood Depth Estimation from Social Media Imagery: A Vision-Language Model Framework with Mechanistic Interpretability for Transportation Resilience
• Abstract summary: FloodLlama is a fine-tuned vision-language model for continuous flood depth estimation from single street-level images.<n>A TikTok-based data pipeline, validated on 676 annotated flood frames from Detroit, demonstrates the feasibility of real-time, crowd-sourced flood sensing.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Urban flooding poses an escalating threat to transportation network continuity, yet no operational system currently provides real-time, street-level flood depth information at the centimeter resolution required for dynamic routing, electric vehicle (EV) safety, and autonomous vehicle (AV) operations. This study presents FloodLlama, a fine-tuned open-source vision-language model (VLM) for continuous flood depth estimation from single street-level images, supported by a multimodal sensing pipeline using TikTok data. A synthetic dataset of approximately 190000 images was generated, covering seven vehicle types, four weather conditions, and 41 depth levels (0-40 cm at 1 cm resolution). Progressive curriculum training enabled coarse-to-fine learning, while LLaMA 3.2-11B Vision was fine-tuned using QLoRA. Evaluation across 34797 trials reveals a depth-dependent prompt effect: simple prompts perform better for shallow flooding, whereas chain-of-thought (CoT) reasoning improves performance at greater depths. FloodLlama achieves a mean absolute error (MAE) below 0.97 cm and Acc@5cm above 93.7% for deep flooding, exceeding 96.8% for shallow depths. A five-phase mechanistic interpretability framework identifies layer L23 as the critical depth-encoding transition and enables selective fine-tuning that reduces trainable parameters by 76-80% while maintaining accuracy. The Tier 3 configuration achieves 98.62% accuracy on real-world data and shows strong robustness under visual occlusion. A TikTok-based data pipeline, validated on 676 annotated flood frames from Detroit, demonstrates the feasibility of real-time, crowd-sourced flood sensing. The proposed framework provides a scalable, infrastructure-free solution with direct implications for EV safety, AV deployment, and resilient transportation management.

Related papers

• Mechanistic Interpretability-Guided Selective Fine-Tuning of Vision-Language Models for Centimeter-Level Flood Depth Estimation [4.766239287853751]
  Urban flooding poses an escalating threat to transportation infrastructure, yet no operational system provides real-time, street-level flood-depth estimates at centimeter resolution.<n>This paper presents three vision-language models fine-tuned for continuous flood-depth estimation from street-level imagery.<n>FloodLlama-Dense achieves an MAE of 0.40 cm, an RMSE of 1.97 cm, and an Acc@5cm of 97.59%.
  arXiv  Detail & Related papers  (2026-08-02T22:38:25Z)
• Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective [76.02772556944073]
  Environmental illusions are naturally existing yet overlooked phenomena in real-world driving environments.<n>They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving systems.<n>We introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions.
  arXiv  Detail & Related papers  (2026-07-07T03:15:00Z)
• LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation [53.88286427894783]
  Vision Language Models (VLMs) have emerged in the robotic domain as a powerful tool that enables environmental perception with language context.<n>Yet, their computational footprint typically confines them to cloud execution, hindering low-latency inference with local deployment on resource-constrained robots.<n>We present a distillation strategy that transfers complex spatial-semantic reasoning from large frontier models into a lightweight, 4B- parameter local VLM for edge execution on embedded GPU devices.
  arXiv  Detail & Related papers  (2026-06-26T09:11:59Z)
• LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People [4.3114959617830015]
  We present an agentic framework that converts a single floor plan image into a structured, retrievable knowledge base to generate navigation instructions.<n>We evaluate the system on the real-world UMBC Math and Psychology building (floors MP-1 and MP-3) and on the CVC-FP benchmark.
  arXiv  Detail & Related papers  (2026-04-27T02:32:44Z)
• Network Level Evaluation of Hangup Susceptibility of HRGCs using Deep Learning and Sensing Techniques: A Goal Towards Safer Future [0.43748379918040853]
  Steep profiled Highway Railway Grade Crossings pose safety hazards to vehicles with low ground clearance.<n>This research develops a framework for network level evaluation of hangup susceptibility of HRGCs.
  arXiv  Detail & Related papers  (2025-12-14T20:25:42Z)
• Floorplan2Guide: LLM-Guided Floorplan Parsing for BLV Indoor Navigation [4.3114959617830015]
  We propose a novel navigation approach that transforms floor plans into navigable knowledge graphs and generate human-readable navigation instructions.<n> Floorplan2Guide integrates a large language model (LLM) to extract spatial information from architectural layouts.<n>Results indicate that few-shot learning improves navigation accuracy in comparison to zero-shot learning on simulated and real-world evaluations.
  arXiv  Detail & Related papers  (2025-12-13T04:49:26Z)
• SAVANT: Semantic Analysis with Vision-Augmented Anomaly deTection [6.806105013817923]
  SAVANT is a structured reasoning framework that achieves high accuracy and recall in detecting anomalous driving scenarios.<n>By automatically labeling over 9,640 real-world images with high accuracy, SAVANT addresses the critical data scarcity problem in anomaly detection.
  arXiv  Detail & Related papers  (2025-10-20T19:14:29Z)
• FloodVision: Urban Flood Depth Estimation Using Foundation Vision-Language Models and Domain Knowledge Graph [1.0323063834827415]
  FloodVision is a zero-shot framework that combines the semantic reasoning abilities of the foundation vision-language model GPT-4o with a structured domain knowledge graph.<n>FloodVision achieves a mean absolute error of 8.17 cm, reducing the GPT-4o baseline 10.28 cm by 20.5% and surpassing prior CNN-based methods.
  arXiv  Detail & Related papers  (2025-09-05T03:05:18Z)
• Learning Underwater Active Perception in Simulation [51.205673783866146]
  Turbidity can jeopardise the whole mission as it may prevent correct visual documentation of the inspected structures.<n>Previous works have introduced methods to adapt to turbidity and backscattering.<n>We propose a simple yet efficient approach to enable high-quality image acquisition of assets in a broad range of water conditions.
  arXiv  Detail & Related papers  (2025-04-23T06:48:38Z)
• Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection [11.743721109110792]
  This paper introduces an integrated approach combining advanced deep learning techniques and Multimodal Large Language Models (MLLMs) for comprehensive road perception.<n>For traffic sign recognition, we evaluate ResNet-50, Yv8, and RT-DETR, achieving state-of-the-art performance of 99.8% with ResNet-50, 98.0% accuracy with YOLOv8, and achieved 96.6% accuracy in RT-DETR.<n>For lane detection, we propose a CNN-based segmentation method enhanced by curve fitting, which delivers high accuracy under favorable conditions.
  arXiv  Detail & Related papers  (2025-03-08T19:12:36Z)
• LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusions [61.87108000328186]
  Lane detection (LD) is an essential component of autonomous driving systems, providing fundamental functionalities like adaptive cruise control and automated lane centering. Existing LD benchmarks primarily focus on evaluating common cases, neglecting the robustness of LD models against environmental illusions. This paper studies the potential threats caused by these environmental illusions to LD and establishes the first comprehensive benchmark LanEvil.
  arXiv  Detail & Related papers  (2024-06-03T02:12:27Z)
• Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement [70.2429155741593]
  This paper presents a new dataset and general tracker enhancement method for Underwater Visual Object Tracking (UVOT) It poses distinct challenges; the underwater environment exhibits non-uniform lighting conditions, low visibility, lack of sharpness, low contrast, camouflage, and reflections from suspended particles. We propose a novel underwater image enhancement algorithm designed specifically to boost tracking quality. The method has resulted in a significant performance improvement, of up to 5.0% AUC, of state-of-the-art (SOTA) visual trackers.
  arXiv  Detail & Related papers  (2023-08-30T07:41:26Z)
• Vision Transformers, a new approach for high-resolution and large-scale mapping of canopy heights [50.52704854147297]
  We present a new vision transformer (ViT) model optimized with a classification (discrete) and a continuous loss function. This model achieves better accuracy than previously used convolutional based approaches (ConvNets) optimized with only a continuous loss function.
  arXiv  Detail & Related papers  (2023-04-22T22:39:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.