From Index to Equity: Pre-Training Transformers for Stock Return Prediction

• URL: http://arxiv.org/abs/2605.23962v1
• Date: Tue, 12 May 2026 13:18:48 GMT
• Title: From Index to Equity: Pre-Training Transformers for Stock Return Prediction
• Abstract summary: This research aims to leverage machine learning to improve stock price prediction and support informed investment decisions.<n>It investigates transformer-based models for stock prediction and examines the impact of pre-training strategies on forecasting performance.
• Score: 2.814741750783267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This research aims to leverage machine learning to improve stock price prediction and support informed investment decisions related to buying, selling, and holding assets. Specifically, this work investigates transformer-based models for stock prediction and examines the impact of pre-training strategies on forecasting performance. A transformer model was first pre-trained on the Toronto Stock Exchange Index (TSX) to predict intra-day return direction and subsequently fine-tuned on individual TSX stocks. The model was further adapted for return-value regression tasks. Performance was benchmarked against Long Short-Term Memory (LSTM) and XGBoost models. Pre-training on the market index improved the binary cross-entropy loss for individual stock prediction from 0.69 to 0.64. The fine-tuned transformer regression model achieved lower mean squared error than the benchmark models, although the ensemble and XGBoost models achieved higher average daily returns. In addition, a practical application was developed to deliver real-time stock predictions for trading support. Future work will focus on increasing transformer model capacity, incorporating broader global technical indicators, and filtering out stocks with low predictability.

Related papers

• Adaptive Temporal Fusion Transformers for Cryptocurrency Price Prediction [0.0]
  This paper introduces an adaptive TFT modeling approach leveraging dynamic subseries lengths and pattern-based categorization to enhance short-term forecasting.<n>Our results on ETH-USDT 10-minute data over a two-month test period demonstrate that our approach significantly outperforms baseline fixed-length TFT and LSTM models in prediction accuracy and simulated trading profitability.
  arXiv  Detail & Related papers  (2025-09-06T20:04:46Z)
• An Evaluation of Deep Learning Models for Stock Market Trend Prediction [0.3277163122167433]
  This study investigates the efficacy of advanced deep learning models for short-term trend forecasting using daily and hourly closing prices from the S&P 500 index and the Brazilian ETF EWZ. We introduce the Extended Long Short-Term Memory for Time Series (xLSTM-TS) model, an xLSTM adaptation optimised for time series prediction. Among the models tested, xLSTM-TS consistently outperformed others. For example, it achieved a test accuracy of 72.82% and an F1 score of 73.16% on the EWZ daily dataset.
  arXiv  Detail & Related papers  (2024-08-22T13:58:55Z)
• Gated recurrent neural network with TPE Bayesian optimization for enhancing stock index prediction accuracy [0.0]
  The aim is to improve the prediction accuracy of the next day's closing price of the NIFTY 50 index, a prominent Indian stock market index. A combination of eight influential factors is carefully chosen from fundamental stock data, technical indicators, crude oil price, and macroeconomic data to train the models.
  arXiv  Detail & Related papers  (2024-06-02T06:39:01Z)
• StockGPT: A GenAI Model for Stock Prediction and Trading [0.0]
  This paper introduces StockGPT, an autoregressive number'' model trained and tested on 70 million daily U.S. stock returns over nearly 100 years. Treating each return series as a sequence of tokens, StockGPT automatically learns the hidden patterns predictive of future returns. On a held-out test sample from 2001 to 2023, daily and monthly rebalanced long-short portfolios formed from StockGPT predictions yield strong performance.
  arXiv  Detail & Related papers  (2024-04-07T22:53:43Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Support for Stock Trend Prediction Using Transformers and Sentiment Analysis [3.147603836269998]
  We develop a Transformer based model that uses technical stock data and sentiment analysis to conduct accurate stock trend prediction over long time windows. This paper also introduces a novel dataset containing daily technical stock data and top news headline data spanning almost three years.
  arXiv  Detail & Related papers  (2023-05-18T03:26:39Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [48.87381259980254]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock market reactions from news headlines without direct financial training.<n>Using post-knowledge-cutoff headlines, GPT-4 captures initial market responses, achieving approximately 90% portfolio-day hit rates for the non-tradable initial reaction.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• DeepVol: Volatility Forecasting from High-Frequency Data with Dilated Causal Convolutions [53.37679435230207]
  We propose DeepVol, a model based on Dilated Causal Convolutions that uses high-frequency data to forecast day-ahead volatility. Our empirical results suggest that the proposed deep learning-based approach effectively learns global features from high-frequency data.
  arXiv  Detail & Related papers  (2022-09-23T16:13:47Z)
• Transformer-Based Deep Learning Model for Stock Price Prediction: A Case Study on Bangladesh Stock Market [0.0]
  This paper concentrates on the application of transformer-based model to predict the price movement of eight specific stocks listed in Dhaka Stock Exchange (DSE) Our experiments demonstrate promising results and acceptable root mean squared error on most of the stocks.
  arXiv  Detail & Related papers  (2022-08-17T14:03:28Z)
• Stock Price Prediction Under Anomalous Circumstances [81.37657557441649]
  This paper aims to capture the movement pattern of stock prices under anomalous circumstances. We train ARIMA and LSTM models at the single-stock level, industry level, and general market level. Based on 100 companies' stock prices in the period of 2016 to 2020, the models achieve an average prediction accuracy of 98%.
  arXiv  Detail & Related papers  (2021-09-14T18:50:38Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• Deep Stock Predictions [58.720142291102135]
  We consider the design of a trading strategy that performs portfolio optimization using Long Short Term Memory (LSTM) neural networks. We then customize the loss function used to train the LSTM to increase the profit earned. We find the LSTM model with the customized loss function to have an improved performance in the training bot over a regressive baseline such as ARIMA.
  arXiv  Detail & Related papers  (2020-06-08T23:37:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.