Solar-VLM: Multimodal Vision-Language Models for Augmented Solar Power Forecasting

• URL: http://arxiv.org/abs/2604.04145v1
• Date: Sun, 05 Apr 2026 15:14:26 GMT
• Title: Solar-VLM: Multimodal Vision-Language Models for Augmented Solar Power Forecasting
• Abstract summary: This paper proposes Solar-VLM, a large-language-model-driven framework for multimodal PV power forecasting.<n>To capture spatial dependencies across geographically distributed stations, a cross-site feature fusion mechanism is introduced.<n>Experiments conducted on data from eight PV stations in a northern province of China demonstrate the effectiveness of the proposed framework.
• Score: 8.479205315463522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photovoltaic (PV) power forecasting plays a critical role in power system dispatch and market participation. Because PV generation is highly sensitive to weather conditions and cloud motion, accurate forecasting requires effective modeling of complex spatiotemporal dependencies across multiple information sources. Although recent studies have advanced AI-based forecasting methods, most fail to fuse temporal observations, satellite imagery, and textual weather information in a unified framework. This paper proposes Solar-VLM, a large-language-model-driven framework for multimodal PV power forecasting. First, modality-specific encoders are developed to extract complementary features from heterogeneous inputs. The time-series encoder adopts a patch-based design to capture temporal patterns from multivariate observations at each site. The visual encoder, built upon a Qwen-based vision backbone, extracts cloud-cover information from satellite images. The text encoder distills historical weather characteristics from textual descriptions. Second, to capture spatial dependencies across geographically distributed PV stations, a cross-site feature fusion mechanism is introduced. Specifically, a Graph Learner models inter-station correlations through a graph attention network constructed over a K-nearest-neighbor (KNN) graph, while a cross-site attention module further facilitates adaptive information exchange among sites. Finally, experiments conducted on data from eight PV stations in a northern province of China demonstrate the effectiveness of the proposed framework. Our proposed model is publicly available at https://github.com/rhp413/Solar-VLM.

Related papers

• AI and Open-data Driven Scalable Solar Power Profiling [48.72868465736126]
  This paper presents an open, scalable framework for detecting solar panels from open data and generating city-level solar power profiles.<n>We leverage foundation vision AI models to detect solar panel from open-source satellite imagery.<n>The framework reduces dependency on proprietary imagery, manual labeling, and closed-source models, and offers a transparent and scalable approach for solar planning and analysis.
  arXiv  Detail & Related papers  (2026-05-04T15:37:52Z)
• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model [53.15040805435013]
  Vision-language models (VLMs) provide strong semantic grounding and general knowledge by reasoning over uniformly sampled frames.<n>We propose a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics modeling with long-horizon semantic guidance.
  arXiv  Detail & Related papers  (2026-03-23T17:59:42Z)
• Ultra-short-term solar power forecasting by deep learning and data reconstruction [60.200987006598524]
  We propose a deep-learning based ultra-short-term solar power prediction with data reconstruction.<n>We employ deep-learning models to capture long- and short-term dependencies towards the target prediction period.
  arXiv  Detail & Related papers  (2025-09-21T14:22:35Z)
• Solar Forecasting with Causality: A Graph-Transformer Approach to Spatiotemporal Dependencies [5.291394959125948]
  SolarCAST is a causally informed model predicting future global horizontal radiance (GHI) at a target site using only historical GHI from site X and nearby stations S.<n>It outperforms leading time-series and multimodal baselines across diverse geographical conditions.<n>It achieves a 25.9% error reduction over the top commercial forecaster, Solcast.
  arXiv  Detail & Related papers  (2025-09-18T22:57:07Z)
• S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting [67.93713728260646]
  Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
  arXiv  Detail & Related papers  (2025-09-10T05:33:28Z)
• Exploring Multimodal AI Reasoning for Meteorological Forecasting from Skew-T Diagrams [4.036372578802888]
  Vision-Language Models (VLMs) have shown promise in other scientific domains, but their application to meteorological diagram interpretation remains largely unexplored.<n>We present a lightweight AI assistant that interprets Skew-T diagrams using a small language model (LM) and a small VLM fine-tuned to emulate human forecasters.
  arXiv  Detail & Related papers  (2025-08-17T01:36:31Z)
• VideoMolmo: Spatio-Temporal Grounding Meets Pointing [66.19964563104385]
  VideoMolmo is a model tailored for fine-grained pointing of video sequences.<n>A novel temporal mask fusion employs SAM2 for bidirectional point propagation.<n>To evaluate the generalization of VideoMolmo, we introduce VPoMolS-temporal, a challenging out-of-distribution benchmark spanning five real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-05T17:59:29Z)
• BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents [56.33989853438012]
  We propose BEVWorld, a framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View latent space for holistic environment modeling.<n>The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• FusionSF: Fuse Heterogeneous Modalities in a Vector Quantized Framework for Robust Solar Power Forecasting [24.57911612111109]
  We propose a multi-modality fusion framework to integrate historical power data, numerical weather prediction, and satellite images. Our framework demonstrates strong zero-shot forecasting capability, which is especially useful for those newly installed plants. Our model not only operates with robustness but also boosts accuracy in both zero-shot forecasting and scenarios rich with training data, surpassing leading models.
  arXiv  Detail & Related papers  (2024-02-08T17:03:10Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• Improving day-ahead Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Context [46.72071291175356]
  Solar power harbors immense potential in mitigating climate change by substantially reducing CO$_2$ emissions. However, the inherent variability of solar irradiance poses a significant challenge for seamlessly integrating solar power into the electrical grid. In this paper, we put forth a deep learning architecture designed to harnesstemporal context using satellite data.
  arXiv  Detail & Related papers  (2023-06-01T19:54:39Z)
• SKIPP'D: a SKy Images and Photovoltaic Power Generation Dataset for Short-term Solar Forecasting [0.0]
  There are few publicly available standardized benchmark datasets for image-based solar forecasting. We introduce SKIPP'D -- a SKy Images and Photovoltaic Power Generation dataset. The dataset contains quality-controlled down-sampled sky images and PV power generation data ready-to-use for short-term solar forecasting using deep learning.
  arXiv  Detail & Related papers  (2022-07-02T21:52:50Z)
• Spatio-temporal graph neural networks for multi-site PV power forecasting [0.0]
  We present two novel graph neural network models for deterministic multi-site forecasting. The proposed models outperform state-of-the-art multi-site forecasting methods for prediction horizons of six hours ahead.
  arXiv  Detail & Related papers  (2021-07-29T10:15:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.