DL-Corrector-Remapper: A grid-free bias-correction deep learning
methodology for data-driven high-resolution global weather forecasting
- URL: http://arxiv.org/abs/2210.12293v1
- Date: Fri, 21 Oct 2022 23:04:44 GMT
- Title: DL-Corrector-Remapper: A grid-free bias-correction deep learning
methodology for data-driven high-resolution global weather forecasting
- Authors: Tao Ge and Jaideep Pathak and Akshay Subramaniam and Karthik Kashinath
- Abstract summary: We develop a methodology to correct, remap, and fine-tune gridded uniform forecasts of FourCastNet (FCN)
This is akin to bias correction and post-processing of numerical weather prediction (NWP)
We call this network the Deep-Learning-Corrector-Remapper (DLCR)
- Score: 11.334341754942917
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Data-driven models, such as FourCastNet (FCN), have shown exemplary
performance in high-resolution global weather forecasting. This performance,
however, is based on supervision on mesh-gridded weather data without the
utilization of raw climate observational data, the gold standard ground truth.
In this work we develop a methodology to correct, remap, and fine-tune gridded
uniform forecasts of FCN so it can be directly compared against observational
ground truth, which is sparse and non-uniform in space and time. This is akin
to bias correction and post-processing of numerical weather prediction (NWP), a
routine operation at meteorological and weather forecasting centers across the
globe. The Adaptive Fourier Neural Operator (AFNO) architecture is used as the
backbone to learn continuous representations of the atmosphere. The spatially
and temporally non-uniform output is evaluated by the non-uniform discrete
inverse Fourier transform (NUIDFT) given the output query locations. We call
this network the Deep-Learning-Corrector-Remapper (DLCR). The improvement in
DLCR's performance against the gold standard ground truth over the baseline's
performance shows its potential to correct, remap, and fine-tune the
mesh-gridded forecasts under the supervision of observations.
Related papers
- A Generalized Transformer-based Radio Link Failure Prediction Framework in 5G RANs [2.519319150166215]
This paper proposes GenTrap, a novel RLF prediction framework that introduces a graph neural network (GNN)-based learnable weather effect aggregation module.
We evaluate GenTrap on two real-world datasets with 2.6 million data points and show that GenTrap offers a significantly higher F1-score (0.93 for rural and 0.79 for urban) compared to its counterparts.
arXiv Detail & Related papers (2024-07-06T21:57:23Z) - Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales [5.453657018459705]
We demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather.
By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained.
It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.
arXiv Detail & Related papers (2024-06-19T10:28:11Z) - ACE Metric: Advection and Convection Evaluation for Accurate Weather Forecasting [7.016835396874093]
We propose the advection and convection Error (ACE) metric to assess how well models predict advection and convection.
We have validated the ACE evaluation metric on the WeatherBench2 and MovingMNIST datasets.
arXiv Detail & Related papers (2024-06-07T06:49:59Z) - Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for
Video Adverse Weather Removal [53.15046196592023]
We introduce test-time adaptation into adverse weather removal in videos.
We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
arXiv Detail & Related papers (2024-03-12T14:21:30Z) - ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast.
We also introduce a training-free extreme value enhancement strategy named ExEnsemble, which increases the variance of pixel values and improves the forecast robustness.
Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
arXiv Detail & Related papers (2024-02-02T10:34:13Z) - Observation-Guided Meteorological Field Downscaling at Station Scale: A
Benchmark and a New Method [66.80344502790231]
We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset.
Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors.
Our proposed method outperforms other specially designed baseline models on multiple surface variables.
arXiv Detail & Related papers (2024-01-22T14:02:56Z) - Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies.
Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
arXiv Detail & Related papers (2023-11-30T08:22:08Z) - Verification against in-situ observations for Data-Driven Weather
Prediction [0.14504054468850663]
Data-driven weather prediction models (DDWPs) have made rapid strides in recent years.
There remains work to be done in rigorously evaluating DDWPs in a true operational setting.
We present a robust dataset of in-situ observations derived from the NOAA MADIS program to serve as a benchmark to validate DDWPs in an operational setting.
arXiv Detail & Related papers (2023-04-28T18:58:36Z) - Forecasting large-scale circulation regimes using deformable
convolutional neural networks and global spatiotemporal climate data [86.1450118623908]
We investigate a supervised machine learning approach based on deformable convolutional neural networks (deCNNs)
We forecast the North Atlantic-European weather regimes during extended boreal winter for 1 to 15 days into the future.
Due to its wider field of view, we also observe deCNN achieving considerably better performance than regular convolutional neural networks at lead times beyond 5-6 days.
arXiv Detail & Related papers (2022-02-10T11:37:00Z) - Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of
Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars.
They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
arXiv Detail & Related papers (2021-07-14T21:10:47Z) - Improving data-driven global weather prediction using deep convolutional
neural networks on a cubed sphere [7.918783985810551]
We present a significantly-improved data-driven global weather forecasting framework using a deep convolutional neural network (CNN)
New developments in this framework include an offline volume-conservative mapping to a cubed-sphere grid.
Our model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables.
arXiv Detail & Related papers (2020-03-15T19:57:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.