DL-Corrector-Remapper: A grid-free bias-correction deep learning methodology for data-driven high-resolution global weather forecasting

• URL: http://arxiv.org/abs/2210.12293v1
• Date: Fri, 21 Oct 2022 23:04:44 GMT
• Title: DL-Corrector-Remapper: A grid-free bias-correction deep learning methodology for data-driven high-resolution global weather forecasting
• Authors: Tao Ge and Jaideep Pathak and Akshay Subramaniam and Karthik Kashinath
• Abstract summary: We develop a methodology to correct, remap, and fine-tune gridded uniform forecasts of FourCastNet (FCN) This is akin to bias correction and post-processing of numerical weather prediction (NWP) We call this network the Deep-Learning-Corrector-Remapper (DLCR)
• Score: 11.334341754942917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-driven models, such as FourCastNet (FCN), have shown exemplary performance in high-resolution global weather forecasting. This performance, however, is based on supervision on mesh-gridded weather data without the utilization of raw climate observational data, the gold standard ground truth. In this work we develop a methodology to correct, remap, and fine-tune gridded uniform forecasts of FCN so it can be directly compared against observational ground truth, which is sparse and non-uniform in space and time. This is akin to bias correction and post-processing of numerical weather prediction (NWP), a routine operation at meteorological and weather forecasting centers across the globe. The Adaptive Fourier Neural Operator (AFNO) architecture is used as the backbone to learn continuous representations of the atmosphere. The spatially and temporally non-uniform output is evaluated by the non-uniform discrete inverse Fourier transform (NUIDFT) given the output query locations. We call this network the Deep-Learning-Corrector-Remapper (DLCR). The improvement in DLCR's performance against the gold standard ground truth over the baseline's performance shows its potential to correct, remap, and fine-tune the mesh-gridded forecasts under the supervision of observations.

Related papers

• Multi-modal graph neural networks for localized off-grid weather forecasting [3.890177521606208]
  Weather forecast products from machine learning or numerical weather models are currently generated on a global regular grid. In this work, we train a heterogeneous graph neural network (GNN) end-to-end to downscale gridded forecasts to off-grid locations of interest. Our approach demonstrates how the gap between global large-scale weather models and locally accurate predictions can be bridged to inform localized decision-making.
  arXiv  Detail & Related papers  (2024-10-16T18:25:43Z)
• SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal [57.52777076116241]
  Adverse weather removal aims to restore clear vision under adverse weather conditions. This paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network.
  arXiv  Detail & Related papers  (2024-09-29T12:12:22Z)
• Generating Fine-Grained Causality in Climate Time Series Data for Forecasting and Anomaly Detection [67.40407388422514]
  We design a conceptual fine-grained causal model named TBN Granger Causality. Second, we propose an end-to-end deep generative model called TacSas, which discovers TBN Granger Causality in a generative manner. We test TacSas on climate benchmark ERA5 for climate forecasting and the extreme weather benchmark of NOAA for extreme weather alerts.
  arXiv  Detail & Related papers  (2024-08-08T06:47:21Z)
• A Generalized Transformer-based Radio Link Failure Prediction Framework in 5G RANs [2.519319150166215]
  This paper proposes GenTrap, a novel RLF prediction framework that introduces a graph neural network (GNN)-based learnable weather effect aggregation module. We evaluate GenTrap on two real-world datasets with 2.6 million data points and show that GenTrap offers a significantly higher F1-score (0.93 for rural and 0.79 for urban) compared to its counterparts.
  arXiv  Detail & Related papers  (2024-07-06T21:57:23Z)
• How far are today's time-series models from real-world weather forecasting applications? [22.68937280154092]
  WEATHER-5K is a comprehensive collection of observational weather data that better reflects real-world scenarios. It enables a better training of models and a more accurate assessment of the real-world forecasting capabilities of TSF models. We provide researchers with a clear assessment of the gap between academic TSF models and real-world weather forecasting applications.
  arXiv  Detail & Related papers  (2024-06-20T15:18:52Z)
• Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales [5.453657018459705]
  We demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.
  arXiv  Detail & Related papers  (2024-06-19T10:28:11Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Forecasting large-scale circulation regimes using deformable convolutional neural networks and global spatiotemporal climate data [86.1450118623908]
  We investigate a supervised machine learning approach based on deformable convolutional neural networks (deCNNs) We forecast the North Atlantic-European weather regimes during extended boreal winter for 1 to 15 days into the future. Due to its wider field of view, we also observe deCNN achieving considerably better performance than regular convolutional neural networks at lead times beyond 5-6 days.
  arXiv  Detail & Related papers  (2022-02-10T11:37:00Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere [7.918783985810551]
  We present a significantly-improved data-driven global weather forecasting framework using a deep convolutional neural network (CNN) New developments in this framework include an offline volume-conservative mapping to a cubed-sphere grid. Our model is able to learn to forecast complex surface temperature patterns from few input atmospheric state variables.
  arXiv  Detail & Related papers  (2020-03-15T19:57:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.