Uncertainty estimation in satellite precipitation interpolation with
machine learning
- URL: http://arxiv.org/abs/2311.07511v2
- Date: Mon, 26 Feb 2024 19:33:25 GMT
- Title: Uncertainty estimation in satellite precipitation interpolation with
machine learning
- Authors: Georgia Papacharalampous, Hristos Tyralis, Nikolaos Doulamis,
Anastasios Doulamis
- Abstract summary: Merging satellite and gauge data with machine learning produces high-resolution precipitation datasets, but uncertainty estimates are often missing.
We address this gap by benchmarking six algorithms, mostly novel for this task, for quantifying predictive uncertainty in spatial data.
We propose a suite of machine learning algorithms for estimating uncertainty in interpolating spatial data, supported with a formal evaluation framework.
- Score: 4.2193475197905705
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Merging satellite and gauge data with machine learning produces
high-resolution precipitation datasets, but uncertainty estimates are often
missing. We address this gap by benchmarking six algorithms, mostly novel for
this task, for quantifying predictive uncertainty in spatial interpolation. On
15 years of monthly data over the contiguous United States (CONUS), we compared
quantile regression (QR), quantile regression forests (QRF), generalized random
forests (GRF), gradient boosting machines (GBM), light gradient boosting
machines (LightGBM), and quantile regression neural networks (QRNN). Their
ability to issue predictive precipitation quantiles at nine quantile levels
(0.025, 0.050, 0.100, 0.250, 0.500, 0.750, 0.900, 0.950, 0.975), approximating
the full probability distribution, was evaluated using quantile scoring
functions and the quantile scoring rule. Feature importance analysis revealed
satellite precipitation (PERSIANN (Precipitation Estimation from Remotely
Sensed Information using Artificial Neural Networks) and IMERG (Integrated
Multi-satellitE Retrievals) datasets) as the most informative predictor,
followed by gauge elevation and distance to satellite grid points. Compared to
QR, LightGBM showed improved performance with respect to the quantile scoring
rule by 11.10%, followed by QRF (7.96%), GRF (7.44%), GBM (4.64%) and QRNN
(1.73%). Notably, LightGBM outperformed all random forest variants, the current
standard in spatial interpolation with machine learning. To conclude, we
propose a suite of machine learning algorithms for estimating uncertainty in
interpolating spatial data, supported with a formal evaluation framework based
on scoring functions and scoring rules.
Related papers
- Uncertainty estimation in satellite precipitation spatial prediction by combining distributional regression algorithms [3.8623569699070353]
We introduce the concept of distributional regression for the engineering task of creating precipitation datasets through data merging.
We propose new ensemble learning methods that can be valuable not only for spatial prediction but also for prediction problems in general.
arXiv Detail & Related papers (2024-06-29T05:58:00Z) - Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs.
We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint.
We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
arXiv Detail & Related papers (2024-06-05T13:36:38Z) - Uncertainty estimation in spatial interpolation of satellite precipitation with ensemble learning [3.8623569699070353]
We introduce nine quantile-based ensemble learners and apply them to large precipitation datasets.
Our ensemble learners include six stacking and three simple methods (mean, median, best combiner)
Stacking with QR and QRNN yielded the best results across quantile levels of interest.
arXiv Detail & Related papers (2024-03-14T17:45:56Z) - Minimally Supervised Learning using Topological Projections in
Self-Organizing Maps [55.31182147885694]
We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs)
Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU)
Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
arXiv Detail & Related papers (2024-01-12T22:51:48Z) - Ensemble learning for blending gridded satellite and gauge-measured
precipitation data [4.2193475197905705]
This study proposes 11 new ensemble learners for improving the accuracy of satellite precipitation products.
We apply the ensemble learners to monthly data from the PERSIANN and IMERG gridded datasets.
We also use gauge-measured precipitation data from the Global Historical Climatology Network monthly database.
arXiv Detail & Related papers (2023-07-09T17:54:46Z) - Merging satellite and gauge-measured precipitation using LightGBM with
an emphasis on extreme quantiles [7.434517639563671]
Knowing actual precipitation in space and time is critical in hydrological modelling applications.
Gridded satellite precipitation datasets offer an alternative option for estimating the actual precipitation.
To improve precipitation estimates, machine learning is applied to merge rain gauge-based measurements and gridded satellite precipitation products.
arXiv Detail & Related papers (2023-02-02T20:03:21Z) - Comparison of machine learning algorithms for merging gridded satellite
and earth-observed precipitation data [7.434517639563671]
We use monthly earth-observed precipitation data from the Global Historical Climatology Network monthly database, version 2.
Results suggest that extreme gradient boosting and random forests are the most accurate in terms of the squared error scoring function.
arXiv Detail & Related papers (2022-12-17T09:39:39Z) - Rethinking Spatial Invariance of Convolutional Networks for Object
Counting [119.83017534355842]
We try to use locally connected Gaussian kernels to replace the original convolution filter to estimate the spatial position in the density map.
Inspired by previous work, we propose a low-rank approximation accompanied with translation invariance to favorably implement the approximation of massive Gaussian convolution.
Our methods significantly outperform other state-of-the-art methods and achieve promising learning of the spatial position of objects.
arXiv Detail & Related papers (2022-06-10T17:51:25Z) - ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked
Models [56.21470608621633]
We propose a time estimation framework to decouple the architectural search from the target hardware.
The proposed methodology extracts a set of models from micro- kernel and multi-layer benchmarks and generates a stacked model for mapping and network execution time estimation.
We compare estimation accuracy and fidelity of the generated mixed models, statistical models with the roofline model, and a refined roofline model for evaluation.
arXiv Detail & Related papers (2021-05-07T11:39:05Z) - Flexible Model Aggregation for Quantile Regression [92.63075261170302]
Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions.
We investigate methods for aggregating any number of conditional quantile models.
All of the models we consider in this paper can be fit using modern deep learning toolkits.
arXiv Detail & Related papers (2021-02-26T23:21:16Z) - APQ: Joint Search for Network Architecture, Pruning and Quantization
Policy [49.3037538647714]
We present APQ for efficient deep learning inference on resource-constrained hardware.
Unlike previous methods that separately search the neural architecture, pruning policy, and quantization policy, we optimize them in a joint manner.
With the same accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ.
arXiv Detail & Related papers (2020-06-15T16:09:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.