Merging satellite and gauge-measured precipitation using LightGBM with an emphasis on extreme quantiles

• URL: http://arxiv.org/abs/2302.03606v2
• Date: Thu, 3 Aug 2023 15:25:51 GMT
• Title: Merging satellite and gauge-measured precipitation using LightGBM with an emphasis on extreme quantiles
• Authors: Hristos Tyralis, Georgia Papacharalampous, Nikolaos Doulamis, Anastasios Doulamis
• Abstract summary: Knowing actual precipitation in space and time is critical in hydrological modelling applications. Gridded satellite precipitation datasets offer an alternative option for estimating the actual precipitation. To improve precipitation estimates, machine learning is applied to merge rain gauge-based measurements and gridded satellite precipitation products.
• Score: 7.434517639563671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowing the actual precipitation in space and time is critical in hydrological modelling applications, yet the spatial coverage with rain gauge stations is limited due to economic constraints. Gridded satellite precipitation datasets offer an alternative option for estimating the actual precipitation by covering uniformly large areas, albeit related estimates are not accurate. To improve precipitation estimates, machine learning is applied to merge rain gauge-based measurements and gridded satellite precipitation products. In this context, observed precipitation plays the role of the dependent variable, while satellite data play the role of predictor variables. Random forests is the dominant machine learning algorithm in relevant applications. In those spatial predictions settings, point predictions (mostly the mean or the median of the conditional distribution) of the dependent variable are issued. The aim of the manuscript is to solve the problem of probabilistic prediction of precipitation with an emphasis on extreme quantiles in spatial interpolation settings. Here we propose, issuing probabilistic spatial predictions of precipitation using Light Gradient Boosting Machine (LightGBM). LightGBM is a boosting algorithm, highlighted by prize-winning entries in prediction and forecasting competitions. To assess LightGBM, we contribute a large-scale application that includes merging daily precipitation measurements in contiguous US with PERSIANN and GPM-IMERG satellite precipitation data. We focus on extreme quantiles of the probability distribution of the dependent variable, where LightGBM outperforms quantile regression forests (QRF, a variant of random forests) in terms of quantile score at extreme quantiles. Our study offers understanding of probabilistic predictions in spatial settings using machine learning.

Related papers

• Uncertainty estimation in satellite precipitation spatial prediction by combining distributional regression algorithms [3.8623569699070353]
  We introduce the concept of distributional regression for the engineering task of creating precipitation datasets through data merging. We propose new ensemble learning methods that can be valuable not only for spatial prediction but also for prediction problems in general.
  arXiv  Detail & Related papers  (2024-06-29T05:58:00Z)
• Enhanced Precision in Rainfall Forecasting for Mumbai: Utilizing Physics Informed ConvLSTM2D Models for Finer Spatial and Temporal Resolution [0.0]
  This study introduces deep learning spatial model aimed at enhancing rainfall prediction accuracy on a finer scale. To test this hypothesis, we introduce a physics informed ConvLSTM2D model to predict precipitation 6hr and 12hr ahead for Mumbai, India.
  arXiv  Detail & Related papers  (2024-04-01T13:56:12Z)
• Uncertainty estimation in satellite precipitation interpolation with machine learning [4.2193475197905705]
  Merging satellite and gauge data with machine learning produces high-resolution precipitation datasets, but uncertainty estimates are often missing. We address this gap by benchmarking six algorithms, mostly novel for this task, for quantifying predictive uncertainty in spatial data. We propose a suite of machine learning algorithms for estimating uncertainty in interpolating spatial data, supported with a formal evaluation framework.
  arXiv  Detail & Related papers  (2023-11-13T17:55:28Z)
• Residual Diffusion Modeling for Km-scale Atmospheric Downscaling [51.061954281398116]
  A cost-effective downscaling model is trained from a high-resolution 2-km weather model over Taiwan. textitCorrDiff exhibits skillful RMSE and CRPS and faithfully recovers spectra and distributions even for extremes. Downscaling global forecasts successfully retains many of these benefits, foreshadowing the potential of end-to-end, global-to-km-scales machine learning weather predictions.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• Towards replacing precipitation ensemble predictions systems using machine learning [0.0]
  We propose a new approach to generating ensemble weather predictions for high-resolution precipitation. The method uses generative adversarial networks to learn the complex patterns of precipitation. We demonstrate the feasibility of generating realistic precipitation ensemble members on unseen higher resolutions.
  arXiv  Detail & Related papers  (2023-04-20T12:20:35Z)
• Prediction-Powered Inference [68.97619568620709]
  Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
  arXiv  Detail & Related papers  (2023-01-23T18:59:28Z)
• Comparison of machine learning algorithms for merging gridded satellite and earth-observed precipitation data [7.434517639563671]
  We use monthly earth-observed precipitation data from the Global Historical Climatology Network monthly database, version 2. Results suggest that extreme gradient boosting and random forests are the most accurate in terms of the squared error scoring function.
  arXiv  Detail & Related papers  (2022-12-17T09:39:39Z)
• Deep Learning Based Cloud Cover Parameterization for ICON [55.49957005291674]
  We train NN based cloud cover parameterizations with coarse-grained data based on realistic regional and global ICON simulations. Globally trained NNs can reproduce sub-grid scale cloud cover of the regional simulation. We identify an overemphasis on specific humidity and cloud ice as the reason why our column-based NN cannot perfectly generalize from the global to the regional coarse-grained data.
  arXiv  Detail & Related papers  (2021-12-21T16:10:45Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression [51.770998056563094]
  Probabilistic Gradient Boosting Machines (PGBM) is a method to create probabilistic predictions with a single ensemble of decision trees. We empirically demonstrate the advantages of PGBM compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-03T08:32:13Z)
• SLPC: a VRNN-based approach for stochastic lidar prediction and completion in autonomous driving [63.87272273293804]
  We propose a new LiDAR prediction framework that is based on generative models namely Variational Recurrent Neural Networks (VRNNs) Our algorithm is able to address the limitations of previous video prediction frameworks when dealing with sparse data by spatially inpainting the depth maps in the upcoming frames. We present a sparse version of VRNNs and an effective self-supervised training method that does not require any labels.
  arXiv  Detail & Related papers  (2021-02-19T11:56:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.