Inflation forecasting with attention based transformer neural networks

• URL: http://arxiv.org/abs/2303.15364v2
• Date: Wed, 29 Mar 2023 08:08:52 GMT
• Title: Inflation forecasting with attention based transformer neural networks
• Authors: Maximilian Tschuchnig and Petra Tschuchnig and Cornelia Ferner and Michael Gadermayr
• Abstract summary: This paper investigates the potential of the transformer deep neural network architecture to forecast different inflation rates. We show that our adapted transformer, on average, outperforms the baseline in 6 out of 16 experiments.
• Score: 1.6822770693792823
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Inflation is a major determinant for allocation decisions and its forecast is a fundamental aim of governments and central banks. However, forecasting inflation is not a trivial task, as its prediction relies on low frequency, highly fluctuating data with unclear explanatory variables. While classical models show some possibility of predicting inflation, reliably beating the random walk benchmark remains difficult. Recently, (deep) neural networks have shown impressive results in a multitude of applications, increasingly setting the new state-of-the-art. This paper investigates the potential of the transformer deep neural network architecture to forecast different inflation rates. The results are compared to a study on classical time series and machine learning models. We show that our adapted transformer, on average, outperforms the baseline in 6 out of 16 experiments, showing best scores in two out of four investigated inflation rates. Our results demonstrate that a transformer based neural network can outperform classical regression and machine learning models in certain inflation rates and forecasting horizons.

Related papers

• Powerformer: A Transformer with Weighted Causal Attention for Time-series Forecasting [50.298817606660826]
  We introduce Powerformer, a novel Transformer variant that replaces noncausal attention weights with causal weights that are reweighted according to a smooth heavy-tailed decay. Our empirical results demonstrate that Powerformer achieves state-of-the-art accuracy on public time-series benchmarks. Our analyses show that the model's locality bias is amplified during training, demonstrating an interplay between time-series data and power-law-based attention.
  arXiv  Detail & Related papers  (2025-02-10T04:42:11Z)
• GARCH-Informed Neural Networks for Volatility Prediction in Financial Markets [0.0]
  We present a new, hybrid Deep Learning model that captures and forecasting market volatility more accurately than either class of models are capable of on their own. When compared to other time series models, GINN showed superior out-of-sample prediction performance in terms of the Coefficient of Determination ($R2$), Mean Squared Error (MSE), and Mean Absolute Error (MAE)
  arXiv  Detail & Related papers  (2024-09-30T23:53:54Z)
• Interval Forecasts for Gas Prices in the Face of Structural Breaks -- Statistical Models vs. Neural Networks [0.0]
  We investigate whether modern machine learning methods such as neural networks are more resilient against such changes. We see that, during the shock period, most models underestimate the variance while overestimating the variance in the after-shock period. Interestingly, the widely-used long-short term neural network is outperformed by its competitors.
  arXiv  Detail & Related papers  (2024-07-23T11:34:13Z)
• Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures [93.17009514112702]
  Pruning, setting a significant subset of the parameters of a neural network to zero, is one of the most popular methods of model compression. Despite existing evidence for this phenomenon, the relationship between neural network pruning and induced bias is not well-understood.
  arXiv  Detail & Related papers  (2023-04-25T07:42:06Z)
• Volatility forecasting using Deep Learning and sentiment analysis [0.0]
  This paper presents a composite model that merges a deep learning approach with sentiment analysis for predicting market volatility. We then describe a composite forecasting model, a Long-Short-Term-Memory Neural Network method, to use historical sentiment and the previous day's volatility to make forecasts.
  arXiv  Detail & Related papers  (2022-10-22T14:54:33Z)
• Multi-scale Feature Learning Dynamics: Insights for Double Descent [71.91871020059857]
  We study the phenomenon of "double descent" of the generalization error. We find that double descent can be attributed to distinct features being learned at different scales.
  arXiv  Detail & Related papers  (2021-12-06T18:17:08Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Neural networks in day-ahead electricity price forecasting: Single vs. multiple outputs [0.0]
  In electricity price forecasting, neural networks are the most popular machine learning method. This paper provides a comprehensive comparison of two most common structures when using the deep neural networks. Results show a significant accuracy advantage of using the latter, confirmed on data from five distinct power exchanges.
  arXiv  Detail & Related papers  (2020-08-18T16:20:31Z)
• Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
  We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation. We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
  arXiv  Detail & Related papers  (2020-08-01T00:58:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.