Transformer Encoder and Multi-features Time2Vec for Financial Prediction

• URL: http://arxiv.org/abs/2504.13801v1
• Date: Fri, 18 Apr 2025 17:07:41 GMT
• Title: Transformer Encoder and Multi-features Time2Vec for Financial Prediction
• Authors: Nguyen Kim Hai Bui, Nguyen Duy Chien, Péter Kovács, Gergő Bognár,
• Abstract summary: We develop a novel neural network architecture by integrating Time2Vec with the Transformer model.<n>Based on the study of different markets, we propose a novel correlation feature selection method.<n>We conclude that our method outperforms other state-of-the-art encoding methods such as positional encoding.
• Score: 1.1399577852929503
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Financial prediction is a complex and challenging task of time series analysis and signal processing, expected to model both short-term fluctuations and long-term temporal dependencies. Transformers have remarkable success mostly in natural language processing using attention mechanism, which also influenced the time series community. The ability to capture both short and long-range dependencies helps to understand the financial market and to recognize price patterns, leading to successful applications of Transformers in stock prediction. Although, the previous research predominantly focuses on individual features and singular predictions, that limits the model's ability to understand broader market trends. In reality, within sectors such as finance and technology, companies belonging to the same industry often exhibit correlated stock price movements. In this paper, we develop a novel neural network architecture by integrating Time2Vec with the Encoder of the Transformer model. Based on the study of different markets, we propose a novel correlation feature selection method. Through a comprehensive fine-tuning of multiple hyperparameters, we conduct a comparative analysis of our results against benchmark models. We conclude that our method outperforms other state-of-the-art encoding methods such as positional encoding, and we also conclude that selecting correlation features enhance the accuracy of predicting multiple stock prices.

Related papers

• An Advanced Ensemble Deep Learning Framework for Stock Price Prediction Using VAE, Transformer, and LSTM Model [4.097563258332958]
  This research proposes a cutting-edge ensemble deep learning framework for stock price prediction by combining three advanced neural network architectures.<n>The framework uses rich set of technical indicators and it scales its predictors based on the current market situation.<n>It has a very important application in algorithmic trading, risk analysis, and control and decision-making for finance professions and scholars.
  arXiv  Detail & Related papers  (2025-03-28T07:20:40Z)
• Higher Order Transformers: Enhancing Stock Movement Prediction On Multimodal Time-Series Data [10.327160288730125]
  We tackle the challenge of predicting stock movements in financial markets by introducing Higher Order Transformers.<n>We extend the self-attention mechanism and the transformer architecture to a higher order, effectively capturing complex market dynamics across time and variables.<n>We present an encoder-decoder model that integrates technical and fundamental analysis, utilizing multimodal signals from historical prices and related tweets.
  arXiv  Detail & Related papers  (2024-12-13T20:26:35Z)
• STORM: A Spatio-Temporal Factor Model Based on Dual Vector Quantized Variational Autoencoders for Financial Trading [55.02735046724146]
  In financial trading, factor models are widely used to price assets and capture excess returns from mispricing.<n>We propose a Spatio-Temporal factOR Model based on dual vector quantized variational autoencoders, named STORM.<n>Storm extracts features of stocks from temporal and spatial perspectives, then fuses and aligns these features at the fine-grained and semantic level, and represents the factors as multi-dimensional embeddings.
  arXiv  Detail & Related papers  (2024-12-12T17:15:49Z)
• Stockformer: A Price-Volume Factor Stock Selection Model Based on Wavelet Transform and Multi-Task Self-Attention Networks [3.7608255115473592]
  This paper introduces Stockformer, a price-volume factor stock selection model that integrates wavelet transformation and a multitask self-attention network. Stockformer decomposes stock returns into high and low frequencies, meticulously capturing long-term market trends and abrupt events. Experimental results show that Stockformer outperforms existing advanced methods on multiple real stock market datasets.
  arXiv  Detail & Related papers  (2023-11-23T04:33:47Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting [50.23240107430597]
  We design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue.
  arXiv  Detail & Related papers  (2023-05-20T05:16:31Z)
• Financial Time Series Forecasting using CNN and Transformer [16.57996422431636]
  We propose to harness the power of CNNs and Transformers to model both short-term and long-term dependencies within a time series. In our experiments, we demonstrated the success of the proposed method in comparison to commonly adopted statistical and deep learning methods.
  arXiv  Detail & Related papers  (2023-04-11T00:56:57Z)
• Stock Trend Prediction: A Semantic Segmentation Approach [3.718476964451589]
  We present a novel approach to predict long-term daily stock price change trends with fully 2D-convolutional encoder-decoders. Our hierarchical structure of CNNs makes it capable of capturing both long and short-term temporal relationships effectively.
  arXiv  Detail & Related papers  (2023-03-09T01:29:09Z)
• Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets [84.90242084523565]
  Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics. By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention. By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
  arXiv  Detail & Related papers  (2022-03-07T18:59:54Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• Transformer Hawkes Process [79.16290557505211]
  We propose a Transformer Hawkes Process (THP) model, which leverages the self-attention mechanism to capture long-term dependencies. THP outperforms existing models in terms of both likelihood and event prediction accuracy by a notable margin. We provide a concrete example, where THP achieves improved prediction performance for learning multiple point processes when incorporating their relational information.
  arXiv  Detail & Related papers  (2020-02-21T13:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.