RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts

• URL: http://arxiv.org/abs/2508.12291v1
• Date: Sun, 17 Aug 2025 08:50:07 GMT
• Title: RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts
• Authors: Xuming He, Zhiyuan You, Junchao Gong, Couhua Liu, Xiaoyu Yue, Peiqin Zhuang, Wenlong Zhang, Lei Bai,
• Abstract summary: We introduce an MLLM-based weather forecast analysis method, RadarQA, integrating key physical attributes with detailed assessment reports.<n>We show that RadarQA outperforms existing general MLLMs across all evaluation settings.
• Score: 25.396253136103432
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Quality analysis of weather forecasts is an essential topic in meteorology. Although traditional score-based evaluation metrics can quantify certain forecast errors, they are still far from meteorological experts in terms of descriptive capability, interpretability, and understanding of dynamic evolution. With the rapid development of Multi-modal Large Language Models (MLLMs), these models become potential tools to overcome the above challenges. In this work, we introduce an MLLM-based weather forecast analysis method, RadarQA, integrating key physical attributes with detailed assessment reports. We introduce a novel and comprehensive task paradigm for multi-modal quality analysis, encompassing both single frame and sequence, under both rating and assessment scenarios. To support training and benchmarking, we design a hybrid annotation pipeline that combines human expert labeling with automated heuristics. With such an annotation method, we construct RQA-70K, a large-scale dataset with varying difficulty levels for radar forecast quality evaluation. We further design a multi-stage training strategy that iteratively improves model performance at each stage. Extensive experiments show that RadarQA outperforms existing general MLLMs across all evaluation settings, highlighting its potential for advancing quality analysis in weather prediction.

Related papers

• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• A Modular LLM-Agent System for Transparent Multi-Parameter Weather Interpretation [3.0354231393746685]
  This paper introduces AI-Meteorologist, an explainable LLM-agent framework that converts raw numerical forecasts into scientifically grounded narrative reports.<n>Unlike conventional forecast outputs presented as dense tables or unstructured time series, our system performs agent-based analysis across multiple meteorological variables.<n>We show how AI-Meteorologist not only communicates weather events but also reveals the underlying atmospheric drivers.
  arXiv  Detail & Related papers  (2025-11-28T22:24:40Z)
• EWE: An Agentic Framework for Extreme Weather Analysis [61.092871317626496]
  Extreme Weather Expert (EWE) is first intelligent agent framework dedicated to this task.<n>EWE emulates expert visualizations through knowledge-guided planning, closed-loop reasoning, and a domain-tailored meteorological toolkit.<n>To catalyze progress, we introduce the first benchmark for this emerging field, comprising a curated dataset of 103 high-impact events.
  arXiv  Detail & Related papers  (2025-11-26T14:37:25Z)
• Adaptive Spatio-Temporal Graphs with Self-Supervised Pretraining for Multi-Horizon Weather Forecasting [3.5137191090796054]
  We propose a self-supervised learning framework that leveragestemporal-temporal structures to improve multi-variable weather prediction.<n>Our approach achieves superior performance compared to traditional numerical prediction weather (NWP) models.<n>The framework provides a scalable and label-efficient solution for future data-driven weather systems.
  arXiv  Detail & Related papers  (2025-10-28T10:52:15Z)
• Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams [4.036372578802888]
  Vision-Language Models (VLMs) have shown promise in other scientific domains, but their application to meteorological diagram interpretation remains largely unexplored.<n>We present a lightweight AI assistant that interprets Skew-T diagrams using a small language model (LM) and a small VLM fine-tuned to emulate human forecasters.
  arXiv  Detail & Related papers  (2025-08-17T01:36:31Z)
• TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop [79.5773512667468]
  TimeXL is a multi-modal prediction framework that integrates a prototype-based time series encoder with three collaborating Large Language Models.<n>A reflection LLM compares the predicted values against the ground truth, identifying textual inconsistencies or noise.<n>This closed-loop workflow-prediction, critique (reflect), and refinement-continuously boosts the framework's performance and interpretability.
  arXiv  Detail & Related papers  (2025-03-02T20:40:53Z)
• WeatherQA: Can Multimodal Language Models Reason about Severe Weather? [45.43764278625153]
  Severe convective weather events, such as hail, tornadoes, and thunderstorms, often occur quickly yet cause significant damage, costing billions of dollars every year. This highlights the importance of forecasting severe weather threats hours in advance to better prepare meteorologists and residents in at-risk areas. We introduce WeatherQA, the first multimodal dataset designed for machines to reason about complex combinations of weather parameters and predict severe weather in real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-17T05:23:18Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.