SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future

• URL: http://arxiv.org/abs/2604.16451v1
• Date: Tue, 07 Apr 2026 20:17:49 GMT
• Title: SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future
• Abstract summary: We present SynopticBench, a high-quality dataset consisting of 1,367,041 text samples of Area Forecast Discussions created by the National Weather Service over the continental United States.<n>We also present Synoptic Phenomena Alignment and Coverage Evaluation (SPACE), a novel evaluation framework that can be used to estimate the quality of text descriptions of synoptic weather phenomena.
• Score: 13.422349825308316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in visual-language models (VLMs) have led to significant improvements in a plethora of complex multimodal tasks like image captioning, report generation, and visual perception. However, generating text from meteorological data is highly challenging because the atmosphere is a chaotic system that is rapidly changing at various spatial and temporal scales. Given the complexity of atmospheric phenomena, it is critical to verifiably quantify the effectiveness of existing VLMs on weather forecasting data. In this work, we present SynopticBench, a high-quality dataset consisting of 1,367,041 text samples of Area Forecast Discussions created by the National Weather Service over the continental United States paired to images of 500mb geopotential height, 2 meter temperature, and 850mb wind velocity in weather forecasts. We also present Synoptic Phenomena Alignment and Coverage Evaluation (SPACE), a novel evaluation framework that can be used to effectively estimate the quality of text descriptions of synoptic weather phenomena. Extensive experiments on generating forecast discussions using state-of-the-art VLMs show the sensitivity of existing evaluation metrics in this domain and enable further exploration into synoptic weather and climate text generation.

Related papers

• Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Data [5.582207674658708]
  We introduce MeteoCap-3B, a billion-scale weather dataset paired with expert-level captions constructed via a Collaborative Captioning (MACC) pipeline.<n>We then propose MTransformer, a diffusion-based model that enables precise semantic control by mapping textual descriptions into multi-band spectral priors.<n>Experiments on real-world benchmarks demonstrate state-of-the-art generation quality, accurate cross-modal alignment, strong semantic controllability, and substantial gains in downstream forecasting.
  arXiv  Detail & Related papers  (2026-03-28T04:55:00Z)
• Adaptive Spatio-Temporal Graphs with Self-Supervised Pretraining for Multi-Horizon Weather Forecasting [3.5137191090796054]
  We propose a self-supervised learning framework that leveragestemporal-temporal structures to improve multi-variable weather prediction.<n>Our approach achieves superior performance compared to traditional numerical prediction weather (NWP) models.<n>The framework provides a scalable and label-efficient solution for future data-driven weather systems.
  arXiv  Detail & Related papers  (2025-10-28T10:52:15Z)
• Zephyrus: An Agentic Framework for Weather Science [47.611521052984365]
  Foundation models for weather science are pre-trained on vast amounts of structured numerical data and outperform traditional weather forecasting systems.<n>Large language models (LLMs) excel at understanding and generating text but cannot reason about high-dimensional meteorological datasets.<n>We bridge this gap by building a novel agentic framework for weather science.<n>We design Zephyrus, a multi-turn LLM-based weather agent that iteratively analyzes weather datasets, observes results, and refines its approach through conversational feedback loops.
  arXiv  Detail & Related papers  (2025-10-05T03:34:08Z)
• PeakWeather: MeteoSwiss Weather Station Measurements for Spatiotemporal Deep Learning [19.620793566349185]
  We introduce PeakWeather, a high-quality dataset of surface weather observations collected every 10 minutes over more than 8 years.<n>The dataset includes a diverse set of meteorological variables from 302 station locations distributed across Switzerland's complex topography.<n>PeakWeather serves as a real-world benchmark to advance both foundational machine learning research, meteorology, and sensor-based applications.
  arXiv  Detail & Related papers  (2025-06-16T16:16:42Z)
• Air Quality Prediction with A Meteorology-Guided Modality-Decoupled Spatio-Temporal Network [47.699409089023696]
  Air quality prediction plays a crucial role in public health and environmental protection.<n>Existing works underestimate the critical role atmospheric conditions in air quality prediction.<n> MDSTNet is an encoder framework explicitly that captures atmosphere-pollution dependencies for prediction.<n>ChinaAirNet is the first dataset combining air quality records with multi-pressure-level meteorological observations.
  arXiv  Detail & Related papers  (2025-04-14T09:18:11Z)
• ClimateBench-M: A Multi-Modal Climate Data Benchmark with a Simple Generative Method [61.76389719956301]
  We contribute a multi-modal climate benchmark, i.e., ClimateBench-M, which aligns time series climate data from ERA5, extreme weather events data from NOAA, and satellite image data from NASA.<n>Under each data modality, we also propose a simple but strong generative method that could produce competitive performance in weather forecasting, thunderstorm alerts, and crop segmentation tasks.
  arXiv  Detail & Related papers  (2025-04-10T02:22:23Z)
• WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning [69.82211470647349]
  We introduce the first generalist weather foundation model (WeatherGFM)<n>It addresses a wide spectrum of weather understanding tasks in a unified manner.<n>Our model can effectively handle up to ten weather understanding tasks, including weather forecasting, super-resolution, weather image translation, and post-processing.
  arXiv  Detail & Related papers  (2024-11-08T09:14:19Z)
• Advancing Meteorological Forecasting: AI-based Approach to Synoptic Weather Map Analysis [3.686808512438363]
  Our study proposes a novel preprocessing method and convolutional autoencoder model to improve the interpretation of synoptic weather maps. This model could recognize historical synoptic weather maps that nearly match current atmospheric conditions.
  arXiv  Detail & Related papers  (2024-11-08T07:46:50Z)
• Generating Fine-Grained Causality in Climate Time Series Data for Forecasting and Anomaly Detection [67.40407388422514]
  We design a conceptual fine-grained causal model named TBN Granger Causality. Second, we propose an end-to-end deep generative model called TacSas, which discovers TBN Granger Causality in a generative manner. We test TacSas on climate benchmark ERA5 for climate forecasting and the extreme weather benchmark of NOAA for extreme weather alerts.
  arXiv  Detail & Related papers  (2024-08-08T06:47:21Z)
• WeatherQA: Can Multimodal Language Models Reason about Severe Weather? [45.43764278625153]
  Severe convective weather events, such as hail, tornadoes, and thunderstorms, often occur quickly yet cause significant damage, costing billions of dollars every year. This highlights the importance of forecasting severe weather threats hours in advance to better prepare meteorologists and residents in at-risk areas. We introduce WeatherQA, the first multimodal dataset designed for machines to reason about complex combinations of weather parameters and predict severe weather in real-world scenarios.
  arXiv  Detail & Related papers  (2024-06-17T05:23:18Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Federated Prompt Learning for Weather Foundation Models on Devices [37.88417074427373]
  On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing. This paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD) FedPoD enables devices to obtain highly customized models while maintaining communication efficiency.
  arXiv  Detail & Related papers  (2023-05-23T16:59:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.