A Machine Learning Outlook: Post-processing of Global Medium-range Forecasts

• URL: http://arxiv.org/abs/2303.16301v1
• Date: Tue, 28 Mar 2023 20:48:01 GMT
• Title: A Machine Learning Outlook: Post-processing of Global Medium-range Forecasts
• Authors: Shreya Agrawal, Rob Carver, Cenk Gazen, Eric Maddy, Vladimir Krasnopolsky, Carla Bromberg, Zack Ontiveros, Tyler Russell, Jason Hickey, and Sid Boukabara
• Abstract summary: Post-processing typically takes the outputs of a Numerical Weather Prediction (NWP) model and applies linear statistical techniques. We show that we can achieve accuracy improvements of up to 12% (RMSE) in a field such as temperature at 850hPa for a 7 day forecast. We discuss the challenges of using standard metrics such as root mean squared error (RMSE) or anomaly correlation coefficient (ACC)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-processing typically takes the outputs of a Numerical Weather Prediction (NWP) model and applies linear statistical techniques to produce improve localized forecasts, by including additional observations, or determining systematic errors at a finer scale. In this pilot study, we investigate the benefits and challenges of using non-linear neural network (NN) based methods to post-process multiple weather features -- temperature, moisture, wind, geopotential height, precipitable water -- at 30 vertical levels, globally and at lead times up to 7 days. We show that we can achieve accuracy improvements of up to 12% (RMSE) in a field such as temperature at 850hPa for a 7 day forecast. However, we recognize the need to strengthen foundational work on objectively measuring a sharp and correct forecast. We discuss the challenges of using standard metrics such as root mean squared error (RMSE) or anomaly correlation coefficient (ACC) as we move from linear statistical models to more complex non-linear machine learning approaches for post-processing global weather forecasts.

Related papers

• FuXi Weather: An end-to-end machine learning weather data assimilation and forecasting system [13.824417759272785]
  This paper introduces FuXi Weather, an end-to-end machine learning based weather forecasting system. FuXi Weather employs specialized data preprocessing and multi-modal data fusion techniques to integrate information from diverse sources. It independently generates robust and accurate 10-day global weather forecasts at a spatial resolution of 0.25text.
  arXiv  Detail & Related papers  (2024-08-10T07:42:01Z)
• Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling [55.13352174687475]
  This paper proposes a physics-AI hybrid model (i.e., WeatherGFT) which Generalizes weather forecasts to Finer-grained Temporal scales. Specifically, we employ a carefully designed PDE kernel to simulate physical evolution on a small time scale. We introduce a lead time-aware training framework to promote the generalization of the model at different lead times.
  arXiv  Detail & Related papers  (2024-05-22T16:21:02Z)
• Aardvark weather: end-to-end data-driven weather forecasting [30.219727555662267]
  Aardvark Weather is an end-to-end data-driven weather prediction system. It ingests raw observations and outputs global gridded forecasts and local station forecasts. It can be optimised end-to-end to maximise performance over quantities of interest.
  arXiv  Detail & Related papers  (2024-03-30T16:41:24Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Towards an end-to-end artificial intelligence driven global weather forecasting system [57.5191940978886]
  We present an AI-based data assimilation model, i.e., Adas, for global weather variables. We demonstrate that Adas can assimilate global observations to produce high-quality analysis, enabling the system operate stably for long term. We are the first to apply the methods to real-world scenarios, which is more challenging and has considerable practical application potential.
  arXiv  Detail & Related papers  (2023-12-18T09:05:28Z)
• FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead [93.67314652898547]
  We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI) FengWu is able to accurately reproduce the atmospheric dynamics and predict the future land and atmosphere states at 37 vertical levels on a 0.25deg latitude-longitude resolution. The results suggest that FengWu can significantly improve the forecast skill and extend the skillful global medium-range weather forecast out to 10.75 days lead.
  arXiv  Detail & Related papers  (2023-04-06T09:16:39Z)
• Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast [91.9372563527801]
  We present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy. Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast and large-member ensemble forecast in real-time.
  arXiv  Detail & Related papers  (2022-11-03T17:19:43Z)
• Forecasting large-scale circulation regimes using deformable convolutional neural networks and global spatiotemporal climate data [86.1450118623908]
  We investigate a supervised machine learning approach based on deformable convolutional neural networks (deCNNs) We forecast the North Atlantic-European weather regimes during extended boreal winter for 1 to 15 days into the future. Due to its wider field of view, we also observe deCNN achieving considerably better performance than regular convolutional neural networks at lead times beyond 5-6 days.
  arXiv  Detail & Related papers  (2022-02-10T11:37:00Z)
• Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison [0.0]
  Postprocessing ensemble weather predictions to correct systematic errors has become a standard practice in research and operations. Here, we provide a comprehensive review and systematic comparison of eight statistical and machine learning methods for probabilistic wind gust forecasting. We propose a flexible framework of locally adaptive neural networks with different probabilistic forecast types as output.
  arXiv  Detail & Related papers  (2021-06-17T14:03:29Z)
• Deep Learning for Post-Processing Ensemble Weather Forecasts [14.622977874836298]
  We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. We show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble.
  arXiv  Detail & Related papers  (2020-05-18T14:23:26Z)
• Machine learning for total cloud cover prediction [0.0]
  We investigate the performance of post-processing using multilayer perceptron (MLP) neural networks, gradient boosting machines (GBM) and random forest (RF) methods. Compared to the raw ensemble, all calibration methods result in a significant improvement in forecast skill. RF models provide the smallest increase in predictive performance, while POLR and GBM approaches perform best.
  arXiv  Detail & Related papers  (2020-01-16T17:13:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.