WeatherReasonSeg: A Benchmark for Weather-Aware Reasoning Segmentation in Visual Language Models

• URL: http://arxiv.org/abs/2603.17680v1
• Date: Wed, 18 Mar 2026 12:57:18 GMT
• Title: WeatherReasonSeg: A Benchmark for Weather-Aware Reasoning Segmentation in Visual Language Models
• Abstract summary: We introduce WeatherReasonSeg, a benchmark designed to evaluate VLM performance in reasoning-based segmentation under adverse weather conditions.<n>First, we construct a controllable reasoning dataset by applying synthetic weather with varying severity levels to existing segmentation datasets.<n>Second, to capture real-world complexity, we curate a real-world adverse-weather reasoning segmentation dataset with semantically consistent queries.
• Score: 15.970876520993954
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing vision-language models (VLMs) have demonstrated impressive performance in reasoning-based segmentation. However, current benchmarks are primarily constructed from high-quality images captured under idealized conditions. This raises a critical question: when visual cues are severely degraded by adverse weather conditions such as rain, snow, or fog, can VLMs sustain reliable reasoning segmentation capabilities? In response to this challenge, we introduce WeatherReasonSeg, a benchmark designed to evaluate VLM performance in reasoning-based segmentation under adverse weather conditions. It consists of two complementary components. First, we construct a controllable reasoning dataset by applying synthetic weather with varying severity levels to existing segmentation datasets, enabling fine-grained robustness analysis. Second, to capture real-world complexity, we curate a real-world adverse-weather reasoning segmentation dataset with semantically consistent queries generated via mask-guided LLM prompting. We further broaden the evaluation scope across five reasoning dimensions, including functionality, application scenarios, structural attributes, interactions, and requirement matching. Extensive experiments across diverse VLMs reveal two key findings: (1) VLM performance degrades monotonically with increasing weather severity, and (2) different weather types induce distinct vulnerability patterns. We hope WeatherReasonSeg will serve as a foundation for advancing robust, weather-aware reasoning.

Related papers

• Seek-and-Solve: Benchmarking MLLMs for Visual Clue-Driven Reasoning in Daily Scenarios [56.703500104126014]
  DailyClue is a benchmark designed for visual clue-driven reasoning in daily scenarios.<n>Our construction is guided by two core principles: (1) strict grounding in authentic daily activities, and (2) challenging query design.<n>Our analysis reveals several critical insights, emphasizing that the accurate identification of visual clues is essential for robust reasoning.
  arXiv  Detail & Related papers  (2026-04-15T16:22:10Z)
• A Semantic Decoupling-Based Two-Stage Rainy-Day Attack for Revealing Weather Robustness Deficiencies in Vision-Language Models [9.92574905741706]
  We introduce the first adversarial framework that exploits realistic weather to attack Vision-Language Models (VLMs)<n>We use a two-stage, parameterized perturbation model based on semantic decoupling to analyze rain-induced shifts in decision-making.<n>Our framework generates perturbations that are both physically grounded and interpretable.
  arXiv  Detail & Related papers  (2026-01-19T17:16:30Z)
• Zephyrus: An Agentic Framework for Weather Science [47.611521052984365]
  Foundation models for weather science are pre-trained on vast amounts of structured numerical data and outperform traditional weather forecasting systems.<n>Large language models (LLMs) excel at understanding and generating text but cannot reason about high-dimensional meteorological datasets.<n>We bridge this gap by building a novel agentic framework for weather science.<n>We design Zephyrus, a multi-turn LLM-based weather agent that iteratively analyzes weather datasets, observes results, and refines its approach through conversational feedback loops.
  arXiv  Detail & Related papers  (2025-10-05T03:34:08Z)
• RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditions [58.37558408672509]
  We propose a robust self-supervised training paradigm, consisting of two key steps: robust self-supervised scene correspondence learning and adverse weather distillation.<n>Experiments demonstrate the effectiveness and versatility of our proposed solution, which outperforms existing state-of-the-art self-supervised methods.
  arXiv  Detail & Related papers  (2025-09-23T15:41:40Z)
• Reinforcing Video Reasoning Segmentation to Think Before It Segments [67.5703457389657]
  We introduce Veason-R1, a specialized LVLM for video reasoning segmentation.<n>Veason-R1 is trained through Group Relative Policy Optimization (O) augmented with Chain-of-Thought trajectories.<n>We incorporate a holistic reward mechanism that enhances spatial alignment and temporal consistency.<n>Veason-R1 achieves state-of-the-art performance on multiple benchmarks, surpassing prior art by significant margins.
  arXiv  Detail & Related papers  (2025-08-15T15:34:56Z)
• Towards Generalized Range-View LiDAR Segmentation in Adverse Weather [65.22588361803942]
  We identify and analyze the unique challenges that affect the generalization of range-view LiDAR segmentation in severe weather.<n>We propose a modular and lightweight framework that enhances robustness without altering the core architecture of existing models.<n>Our approach significantly improves generalization to adverse weather with minimal inference overhead.
  arXiv  Detail & Related papers  (2025-06-10T16:48:27Z)
• WXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models [12.16198010132936]
  WXImpactBench is the first benchmark for evaluating the capacity of large language models (LLMs) on disruptive weather impacts.<n>The constructed dataset and the code for the evaluation framework are available to help society protect against vulnerabilities from disasters.
  arXiv  Detail & Related papers  (2025-05-26T17:23:29Z)
• GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [84.86699025256705]
  We present GEOBench-VLM, a benchmark specifically designed to evaluate Vision-Language Models (VLMs) on geospatial tasks.<n>Our benchmark features over 10,000 manually verified instructions and spanning diverse visual conditions, object types, and scales.<n>We evaluate several state-of-the-art VLMs to assess performance on geospatial-specific challenges.
  arXiv  Detail & Related papers  (2024-11-28T18:59:56Z)
• WeatherQA: Can Multimodal Language Models Reason about Severe Weather? [45.43764278625153]
  Severe convective weather events, such as hail, tornadoes, and thunderstorms, often occur quickly yet cause significant damage, costing billions of dollars every year. This highlights the importance of forecasting severe weather threats hours in advance to better prepare meteorologists and residents in at-risk areas. We introduce WeatherQA, the first multimodal dataset designed for machines to reason about complex combinations of weather parameters and predict severe weather in real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-17T05:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.