A HEART for the environment: Transformer-Based Spatiotemporal Modeling for Air Quality Prediction

• URL: http://arxiv.org/abs/2502.19042v1
• Date: Wed, 26 Feb 2025 10:54:27 GMT
• Title: A HEART for the environment: Transformer-Based Spatiotemporal Modeling for Air Quality Prediction
• Authors: Norbert Bodendorfer,
• Abstract summary: llull-environment is a sophisticated and scalable forecasting system for air pollution.<n>It contains an encoder-decoder convolutional neural network to forecast mean pollution levels for four key pollutants.<n>This paper investigates the augmentation of this neural network with an attention mechanism to improve predictive accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate and reliable air pollution forecasting is crucial for effective environmental management and policy-making. llull-environment is a sophisticated and scalable forecasting system for air pollution, inspired by previous models currently operational in Madrid and Valladolid (Spain). It contains (among other key components) an encoder-decoder convolutional neural network to forecast mean pollution levels for four key pollutants (NO$_2$, O$_3$, PM$_{10}$, PM$_{2.5}$) using historical data, external forecasts, and other contextual features. This paper investigates the augmentation of this neural network with an attention mechanism to improve predictive accuracy. The proposed attention mechanism pre-processes tensors containing the input features before passing them to the existing mean forecasting model. The resulting model is a combination of several architectures and ideas and can be described as a "Hybrid Enhanced Autoregressive Transformer", or HEART. The effectiveness of the approach is evaluated by comparing the mean square error (MSE) across different attention layouts against the system without such a mechanism. We observe a significant reduction in MSE of up to 22%, with an average of 7.5% across tested cities and pollutants. The performance of a given attention mechanism turns out to depend on the pollutant, highlighting the differences in their creation and dissipation processes. Our findings are not restricted to optimizing air quality prediction models, but are applicable generally to (fixed length) time series forecasting.

Related papers

• Air Quality Prediction with A Meteorology-Guided Modality-Decoupled Spatio-Temporal Network [47.699409089023696]
  Air quality prediction plays a crucial role in public health and environmental protection. Existing works underestimate the critical role atmospheric conditions in air quality prediction. MDSTNet is an encoder framework explicitly that captures atmosphere-pollution dependencies for prediction. ChinaAirNet is the first dataset combining air quality records with multi-pressure-level meteorological observations.
  arXiv  Detail & Related papers  (2025-04-14T09:18:11Z)
• Hybrid Forecasting of Geopolitical Events [71.73737011120103]
  SAGE is a hybrid forecasting system that combines human and machine generated forecasts.<n>The system aggregates human and machine forecasts weighting both for propinquity and based on assessed skill.<n>We show that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data.
  arXiv  Detail & Related papers  (2024-12-14T22:09:45Z)
• Variable importance measure for spatial machine learning models with application to air pollution exposure prediction [2.633085745593072]
  The objective is to predict air pollution exposures for study subjects at locations without data in order to optimize our ability to learn about health effects of air pollution. We tackle these challenges in two datasets: sulfur (S) from regulatory United States national PM2.5 sub-species data and ultrafine particles (UFP) from a new Seattle-area traffic-related air pollution dataset. Our key contribution is a leave-one-out approach for variable importance that leads to interpretable and comparable measures for a broad class of models.
  arXiv  Detail & Related papers  (2024-06-04T05:51:36Z)
• WaveCatBoost for Probabilistic Forecasting of Regional Air Quality Data [0.0]
  This letter presents a novel WaveCatBoost architecture designed to forecast the real-time concentrations of air pollutants. This hybrid approach efficiently transforms time series into high-frequency and low-frequency components, thereby extracting signal from noise.
  arXiv  Detail & Related papers  (2024-04-08T13:01:25Z)
• Back to the Future: GNN-based NO$_2$ Forecasting via Future Covariates [49.93577170464313]
  We deal with air quality observations in a city-wide network of ground monitoring stations. We propose a conditioning block that embeds past and future covariates into the current observations. We find that conditioning on future weather information has a greater impact than considering past traffic conditions.
  arXiv  Detail & Related papers  (2024-04-08T09:13:16Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Generative ensemble deep learning severe weather prediction from a deterministic convection-allowing model [0.0]
  Method combines conditional generative adversarial networks (CGANs) with a convolutional neural network (CNN) to post-process convection-allowing model (CAM) forecasts. The CGANs are designed to create synthetic ensemble members from deterministic CAM forecasts. The method produced skillful predictions with up to 20% Brier Skill Score (BSS) increases compared to other neural-network-based reference methods.
  arXiv  Detail & Related papers  (2023-10-09T18:02:11Z)
• Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling [58.456404022536425]
  State of the art for physical hazard prediction from weather and climate requires expensive km-scale numerical simulations driven by coarser resolution global inputs. Here, a generative diffusion architecture is explored for downscaling such global inputs to km-scale, as a cost-effective machine learning alternative. The model is trained to predict 2km data from a regional weather model over Taiwan, conditioned on a 25km global reanalysis.
  arXiv  Detail & Related papers  (2023-09-24T19:57:22Z)
• Statistical post-processing of visibility ensemble forecasts [0.0]
  We investigate the predictive performance of locally, semi-locally and regionally trained proportional odds logistic regression (POLR) and multilayer perceptron (MLP) neural network classifiers. We show that while climatological forecasts outperform the raw ensemble by a wide margin, post-processing results in further substantial improvement in forecast skill.
  arXiv  Detail & Related papers  (2023-05-24T16:41:36Z)
• Beyond S-curves: Recurrent Neural Networks for Technology Forecasting [60.82125150951035]
  We develop an autencoder approach that employs recent advances in machine learning and time series forecasting. S-curves forecasts largely exhibit a mean average percentage error (MAPE) comparable to a simple ARIMA baseline. Our autoencoder approach improves the MAPE by 13.5% on average over the second-best result.
  arXiv  Detail & Related papers  (2022-11-28T14:16:22Z)
• Machine learning for total cloud cover prediction [0.0]
  We investigate the performance of post-processing using multilayer perceptron (MLP) neural networks, gradient boosting machines (GBM) and random forest (RF) methods. Compared to the raw ensemble, all calibration methods result in a significant improvement in forecast skill. RF models provide the smallest increase in predictive performance, while POLR and GBM approaches perform best.
  arXiv  Detail & Related papers  (2020-01-16T17:13:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.