Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Data

• URL: http://arxiv.org/abs/2603.27135v1
• Date: Sat, 28 Mar 2026 04:55:00 GMT
• Title: Spectral-Aware Text-to-Time Series Generation with Billion-Scale Multimodal Meteorological Data
• Abstract summary: We introduce MeteoCap-3B, a billion-scale weather dataset paired with expert-level captions constructed via a Collaborative Captioning (MACC) pipeline.<n>We then propose MTransformer, a diffusion-based model that enables precise semantic control by mapping textual descriptions into multi-band spectral priors.<n>Experiments on real-world benchmarks demonstrate state-of-the-art generation quality, accurate cross-modal alignment, strong semantic controllability, and substantial gains in downstream forecasting.
• Score: 5.582207674658708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-time-series generation is particularly important in meteorology, where natural language offers intuitive control over complex, multi-scale atmospheric dynamics. Existing approaches are constrained by the lack of large-scale, physically grounded multimodal datasets and by architectures that overlook the spectral-temporal structure of weather signals. We address these challenges with a unified framework for text-guided meteorological time-series generation. First, we introduce MeteoCap-3B, a billion-scale weather dataset paired with expert-level captions constructed via a Multi-agent Collaborative Captioning (MACC) pipeline, yielding information-dense and physically consistent annotations. Building on this dataset, we propose MTransformer, a diffusion-based model that enables precise semantic control by mapping textual descriptions into multi-band spectral priors through a Spectral Prompt Generator, which guides generation via frequency-aware attention. Extensive experiments on real-world benchmarks demonstrate state-of-the-art generation quality, accurate cross-modal alignment, strong semantic controllability, and substantial gains in downstream forecasting under data-sparse and zero-shot settings. Additional results on general time-series benchmarks indicate that the proposed framework generalizes beyond meteorology.

Related papers

• SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version [5.00208619516796]
  We propose SCENARIODIFF, a hierarchical contextual reasoning framework for multimodal time series forecasting.<n>SCENARIODIFF organizes contextual information into three levels: a Historical Context Agent extracts stepwise evidence from raw documents, a Scenario Agent produces a qualitative scenario description for the forecast horizon, and an Anchor Guidance Agent generates sparse anchor points for event-relevant future regions.<n> Experiments on the Time-MMD benchmark show that SCENARIODIFF is especially effective in event-driven domains.
  arXiv  Detail & Related papers  (2026-08-17T22:05:09Z)
• Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting [52.53742727060265]
  Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets.<n>Here, we introduce Earth System Foundation Model (ESFM), a fully open model building on the 3Dwin UNet backbone of the pioneering model.<n>We present findings using dense gridded data (ERA5, CMIP6), regionally masked dense data, sparse gridded MODIS satellite data, and station data.<n>Results demonstrate competitive or superior performance relative to state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T11:40:39Z)
• Optimizing Multi-Agent Weather Captioning via Text Gradient Descent: A Training-Free Approach with Consensus-Aware Gradient Fusion [0.0]
  We introduce WeatherTGD, a training-free multi-agent framework that reinterprets collaborative caption refinement through the lens of Text Gradient Descent (TGD)<n>Our system deploys three specialized LLM agents including a Statistical Analyst, a Physics Interpreter, and a Meteorology Expert that generate domain-specific textual gradients from weather time series observations.<n> Experiments on real-world meteorological datasets demonstrate that WeatherTGD achieves significant improvements in both LLM-based evaluation and human expert evaluation.
  arXiv  Detail & Related papers  (2026-03-23T07:55:45Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Hierarchical AI-Meteorologist: LLM-Agent System for Multi-Scale and Explainable Weather Forecast Reporting [3.0354231393746685]
  Hierarchical AI-Meteorologist generates explainable weather reports using a hierarchical forecast reasoning and weather keyword generation.<n>Our framework performs multi-scale reasoning across hourly, 6-hour, and daily aggregations to capture both short-term dynamics and long-term trends.
  arXiv  Detail & Related papers  (2025-11-28T17:27:06Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Multi-modal Adaptive Estimation for Temporal Respiratory Disease Outbreak [8.861941883057098]
  MAESTRO is a novel, unified framework that integrates spectro-temporal modeling with multi-modal data fusion.<n> Evaluated on over 11 years of Hong Kong data, MAESTRO achieves a superior model fit with an R-square of 0.956.<n>The modular and reproducible pipeline is made publicly available to facilitate deployment and extension to other regions and pathogens.
  arXiv  Detail & Related papers  (2025-09-10T13:27:40Z)
• MeteorPred: A Meteorological Multimodal Large Model and Dataset for Severe Weather Event Prediction [44.09814361538848]
  We introduce MP-Bench, the first large-scale temporal multimodal dataset for severe weather events prediction.<n>On top of this dataset, we develop a multimodal large model (MMLM) that directly ingests 4D meteorological inputs.<n> MMLM performs exceptionally well across multiple tasks, highlighting its effectiveness in severe weather understanding.
  arXiv  Detail & Related papers  (2025-08-09T06:54:41Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [42.60778405812048]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting [0.0]
  Climate stands as one of the most pressing global challenges of the twenty-first century, with far-reaching consequences such as rising sea levels, melting glaciers, and increasingly extreme weather patterns.<n> Accurate forecasting is critical for monitoring these phenomena and supporting mitigation strategies.<n>Recent data-driven models for time-series forecasting, including CNNs, RNNs, and attention-based transformers, have shown promise but struggle with dependencies and limited parallelization.<n>In this work, we present Modulated Attention Focal (FATE) for reliable time-series forecasting.
  arXiv  Detail & Related papers  (2024-08-21T04:40:18Z)
• Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach [71.67506068703314]
  Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Traditionally, urban mobility data has been structured as videos, treating longitude and latitude as fundamental pixels. In our research, we introduce a fresh perspective on urban mobility prediction. Instead of oversimplifying urban mobility data as traditional video data, we regard it as a complex time series.
  arXiv  Detail & Related papers  (2023-12-04T07:39:05Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.