A visual big data system for the prediction of weather-related variables: Jordan-Spain case study

• URL: http://arxiv.org/abs/2510.21176v1
• Date: Fri, 24 Oct 2025 05:59:08 GMT
• Title: A visual big data system for the prediction of weather-related variables: Jordan-Spain case study
• Authors: Shadi Aljawarneh, Juan A. Lara, Muneer Bani Yassein,
• Abstract summary: We propose a visual big data system that is designed to deal with high amounts of weather-related data.<n>The proposed system collects open data and loads them onto a local database fusing them at different levels of temporal and spatial aggregation.<n>The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Meteorology is a field where huge amounts of data are generated, mainly collected by sensors at weather stations, where different variables can be measured. Those data have some particularities such as high volume and dimensionality, the frequent existence of missing values in some stations, and the high correlation between collected variables. In this regard, it is crucial to make use of Big Data and Data Mining techniques to deal with those data and extract useful knowledge from them that can be used, for instance, to predict weather phenomena. In this paper, we propose a visual big data system that is designed to deal with high amounts of weather-related data and lets the user analyze those data to perform predictive tasks over the considered variables (temperature and rainfall). The proposed system collects open data and loads them onto a local NoSQL database fusing them at different levels of temporal and spatial aggregation in order to perform a predictive analysis using univariate and multivariate approaches as well as forecasting based on training data from neighbor stations in cases with high rates of missing values. The system has been assessed in terms of usability and predictive performance, obtaining an overall normalized mean squared error value of 0.00013, and an overall directional symmetry value of nearly 0.84. Our system has been rated positively by a group of experts in the area (all aspects of the system except graphic desing were rated 3 or above in a 1-5 scale). The promising preliminary results obtained demonstrate the validity of our system and invite us to keep working on this area.

Related papers

• Towards Data-Efficient Pretraining for Atomic Property Prediction [51.660835328611626]
  We show that pretraining on a task-relevant dataset can match or surpass large-scale pretraining.<n>We introduce the Chemical Similarity Index (CSI), a novel metric inspired by computer vision's Fr'echet Inception Distance.
  arXiv  Detail & Related papers  (2025-02-16T11:46:23Z)
• Tackling Data Heterogeneity in Federated Time Series Forecasting [61.021413959988216]
  Time series forecasting plays a critical role in various real-world applications, including energy consumption prediction, disease transmission monitoring, and weather forecasting. Most existing methods rely on a centralized training paradigm, where large amounts of data are collected from distributed devices to a central cloud server. We propose a novel framework, Fed-TREND, to address data heterogeneity by generating informative synthetic data as auxiliary knowledge carriers.
  arXiv  Detail & Related papers  (2024-11-24T04:56:45Z)
• Local Off-Grid Weather Forecasting with Multi-Modal Earth Observation Data [5.147520971893645]
  Wildfire management and renewable energy generation require precise, localized weather forecasts near the Earth's surface.<n>Forecasts produced by machine learning models or numerical weather prediction systems are typically generated on large-scale regular grids.<n>We propose a multi-grained transformer model trained end-to-end to downscale gridded forecasts to off-grid locations of interest.
  arXiv  Detail & Related papers  (2024-10-16T18:25:43Z)
• Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales [5.427841765899196]
  We demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather.<n>By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained.<n>It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.
  arXiv  Detail & Related papers  (2024-06-19T10:28:11Z)
• Predicting unobserved climate time series data at distant areas via spatial correlation using reservoir computing [5.008022566317949]
  This study focuses on a prediction of climatic elements, specifically near-surface temperature and pressure, at a target location apart from a data observation point. Our results show that the accuracy of the predictions degrades with the distance between the observation and target locations.
  arXiv  Detail & Related papers  (2024-06-05T08:39:10Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Predicting Temperature of Major Cities Using Machine Learning and Deep Learning [0.0]
  We use the database made by University of Dayton which consists the change of temperature in major cities to predict the temperature of different cities during any time in future. This document contains our methodology for being able to make such predictions.
  arXiv  Detail & Related papers  (2023-09-23T10:23:00Z)
• Federated Prompt Learning for Weather Foundation Models on Devices [37.88417074427373]
  On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing. This paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD) FedPoD enables devices to obtain highly customized models while maintaining communication efficiency.
  arXiv  Detail & Related papers  (2023-05-23T16:59:20Z)
• A Meta-Analysis of Solar Forecasting Based on Skill Score [0.0]
  We conduct the first comprehensive meta-analysis of deterministic solar forecasting based on skill score. A database of 4,687 points was built and analyzed. More improvement is observed for intra-hour and intra-day than day-ahead forecasts.
  arXiv  Detail & Related papers  (2022-08-22T18:37:01Z)
• TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
  estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. We propose a versatile method that estimates joint distributions using an attention-based decoder. We show that our model produces state-of-the-art predictions on several real-world datasets.
  arXiv  Detail & Related papers  (2022-02-07T21:37:29Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Multi-Source Causal Inference Using Control Variates [81.57072928775509]
  We propose a general algorithm to estimate causal effects from emphmultiple data sources. We show theoretically that this reduces the variance of the ATE estimate. We apply this framework to inference from observational data under an outcome selection bias.
  arXiv  Detail & Related papers  (2021-03-30T21:20:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.