Transformer Based Time-Series Forecasting for Stock

• URL: http://arxiv.org/abs/2502.09625v1
• Date: Wed, 29 Jan 2025 00:26:47 GMT
• Title: Transformer Based Time-Series Forecasting for Stock
• Authors: Shuozhe Li, Zachery B Schulwol, Risto Miikkulainen,
• Abstract summary: It is one of the most difficult forecasting tasks that hundreds of millions of retail traders and professional traders around the world try to do every second even before the market opens.<n>With recent advances in the development of machine learning and the amount of data the market generated over years, applying machine learning techniques such as deep learning neural networks is unavoidable.
• Score: 9.437599568164869
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To the naked eye, stock prices are considered chaotic, dynamic, and unpredictable. Indeed, it is one of the most difficult forecasting tasks that hundreds of millions of retail traders and professional traders around the world try to do every second even before the market opens. With recent advances in the development of machine learning and the amount of data the market generated over years, applying machine learning techniques such as deep learning neural networks is unavoidable. In this work, we modeled the task as a multivariate forecasting problem, instead of a naive autoregression problem. The multivariate analysis is done using the attention mechanism via applying a mutated version of the Transformer, "Stockformer", which we created.

Related papers

• Transformer Encoder and Multi-features Time2Vec for Financial Prediction [1.1399577852929503]
  We develop a novel neural network architecture by integrating Time2Vec with the Transformer model. Based on the study of different markets, we propose a novel correlation feature selection method. We conclude that our method outperforms other state-of-the-art encoding methods such as positional encoding.
  arXiv  Detail & Related papers  (2025-04-18T17:07:41Z)
• Higher Order Transformers: Enhancing Stock Movement Prediction On Multimodal Time-Series Data [10.327160288730125]
  We tackle the challenge of predicting stock movements in financial markets by introducing Higher Order Transformers.<n>We extend the self-attention mechanism and the transformer architecture to a higher order, effectively capturing complex market dynamics across time and variables.<n>We present an encoder-decoder model that integrates technical and fundamental analysis, utilizing multimodal signals from historical prices and related tweets.
  arXiv  Detail & Related papers  (2024-12-13T20:26:35Z)
• MCI-GRU: Stock Prediction Model Based on Multi-Head Cross-Attention and Improved GRU [15.232546605091818]
  This paper proposes a stock prediction model, MCI-GRU, based on a multi-head cross-attention mechanism and an improved GRU. Experiments on four main stock markets show that the proposed method outperforms SOTA techniques across multiple metrics.
  arXiv  Detail & Related papers  (2024-09-25T14:37:49Z)
• Quantformer: from attention to profit with a quantitative transformer trading strategy [1.6006550105523192]
  This work collects more than 5,000,000 rolling data of 4,601 stocks in the Chinese capital market from 2010 to 2019. The results of this study demonstrated the model's superior performance in predicting stock trends compared with other 100 factor-based quantitative strategies.
  arXiv  Detail & Related papers  (2024-03-30T17:18:00Z)
• In-Context Convergence of Transformers [63.04956160537308]
  We study the learning dynamics of a one-layer transformer with softmax attention trained via gradient descent. For data with imbalanced features, we show that the learning dynamics take a stage-wise convergence process.
  arXiv  Detail & Related papers  (2023-10-08T17:55:33Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Augmented Bilinear Network for Incremental Multi-Stock Time-Series Classification [83.23129279407271]
  We propose a method to efficiently retain the knowledge available in a neural network pre-trained on a set of securities. In our method, the prior knowledge encoded in a pre-trained neural network is maintained by keeping existing connections fixed. This knowledge is adjusted for the new securities by a set of augmented connections, which are optimized using the new data.
  arXiv  Detail & Related papers  (2022-07-23T18:54:10Z)
• Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets [84.90242084523565]
  Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics. By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention. By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
  arXiv  Detail & Related papers  (2022-03-07T18:59:54Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• Taking Over the Stock Market: Adversarial Perturbations Against Algorithmic Traders [47.32228513808444]
  We present a realistic scenario in which an attacker influences algorithmic trading systems by using adversarial learning techniques. We show that when added to the input stream, our perturbation can fool the trading algorithms at future unseen data points.
  arXiv  Detail & Related papers  (2020-10-19T06:28:05Z)
• Multi-future Merchant Transaction Prediction [11.479583812869645]
  The capability of predicting merchants' future is crucial for fraud detection and recommendation systems. We propose a new model using convolutional neural networks and a simple yet effective encoder-decoder structure to learn the time series pattern.
  arXiv  Detail & Related papers  (2020-07-10T11:07:32Z)
• Time-varying neural network for stock return prediction [0.0]
  We show that a neural network trained using an online early stopping algorithm can track a function changing with unknown dynamics. We also show that prominent factors (such as the size and momentum effects) and industry indicators, exhibit time varying stock return predictiveness.
  arXiv  Detail & Related papers  (2020-03-05T10:16:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.