Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning

• URL: http://arxiv.org/abs/2605.30652v1
• Date: Thu, 28 May 2026 23:14:45 GMT
• Title: Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning
• Abstract summary: This paper explores high-dimensional representation learning by replacing discrete polarity ratings with dense FinBERT embeddings within a Transformer-based forecasting architecture.<n>We benchmarked various embedding strategies on the FNSPID dataset, including raw embeddings, attention-weighted aggregation, and a custom Siamese network.
• Score: 2.1956992918521805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional multi-modal financial forecasting often relies on scalar sentiment scores, which fail to capture the nuances of financial news. To address this information loss, this paper explores high-dimensional representation learning by replacing discrete polarity ratings with dense FinBERT embeddings within a Transformer-based forecasting architecture. We benchmarked various embedding strategies on the FNSPID dataset, including raw embeddings, attention-weighted aggregation, and a custom Siamese network. While the attention-based mechanism struggled with the low signal-to-noise ratio typical of financial data, the integration of Siamese-optimized embeddings outperformed both the scalar baseline and raw embedding approaches, demonstrating that preserving high-dimensional narrative context yields improved predictive accuracy for short-term stock price movements.

Related papers

• Context-Integrated Adversarial Learning for Predictive Modelling of Stock Price Dynamics [0.0]
  This paper introduces a context-sensitive adversarial learning model to predict equity prices.<n>The architecture uses adversarial training to model future price movements and incorporates contextual sentiment features derived from financial textual data.<n> Empirical evaluation of a sample of U.S. equities testifies that the presented approach outperforms the traditional ARIMA and LSTM baselines in a range of measures of error.
  arXiv  Detail & Related papers  (2026-04-13T18:51:01Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• Generalized Stock Price Prediction for Multiple Stocks Combined with News Fusion [55.80546131166094]
  This paper introduces an approach that integrates Large Language Models (LLMs) with daily financial news for stock price prediction.<n>We encode news articles using a pre-trained LLM and implement three attention-based pooling techniques to filter news based on stock relevance.<n> Experimental results demonstrate a 7.11% reduction in Mean Absolute Error (MAE) compared to the baseline.
  arXiv  Detail & Related papers  (2026-03-08T09:05:10Z)
• ASTIF: Adaptive Semantic-Temporal Integration for Cryptocurrency Price Forecasting [6.12055122337183]
  ASTIF is a hybrid intelligent system that adapts its forecasting strategy in real time through confidence-based meta-learning.<n>A confidence-aware meta-learner functions as an adaptive inference layer, modulating each predictor's contribution based on its real-time uncertainty.<n>The research contributes a scalable, knowledge-based solution for fusing quantitative and qualitative data in non-stationary environments.
  arXiv  Detail & Related papers  (2025-12-21T09:17:36Z)
• Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data [49.36938105983916]
  Unsupervised domain adaptation methods seek to generalize effectively on unlabeled test data.<n>We propose incorporating multi-scale feature extraction and uncertainty estimation to improve the model's generalization and robustness across domains.
  arXiv  Detail & Related papers  (2025-08-26T03:13:08Z)
• An Advanced Ensemble Deep Learning Framework for Stock Price Prediction Using VAE, Transformer, and LSTM Model [4.097563258332958]
  This research proposes a cutting-edge ensemble deep learning framework for stock price prediction by combining three advanced neural network architectures.<n>The framework uses rich set of technical indicators and it scales its predictors based on the current market situation.<n>It has a very important application in algorithmic trading, risk analysis, and control and decision-making for finance professions and scholars.
  arXiv  Detail & Related papers  (2025-03-28T07:20:40Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Enhancing Financial Data Visualization for Investment Decision-Making [0.04096453902709291]
  This paper delves into the potential of Long Short-Term Memory (LSTM) networks for predicting stock dynamics. The study incorporates multiple features to enhance LSTM's capacity in capturing complex patterns. The meticulously crafted LSTM incorporates crucial price and volume attributes over a 25-day time step.
  arXiv  Detail & Related papers  (2023-12-09T07:53:25Z)
• Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices [3.1466086042810884]
  Transformer-based language models have set new benchmarks across a wide range of NLP tasks. reliably estimating the uncertainty of their predictions remains a significant challenge. We tackle these limitations by harnessing the geometry of attention maps across multiple heads and layers to assess model confidence. Our method significantly outperforms existing uncertainty estimation techniques on benchmarks for acceptability judgments and artificial text detection.
  arXiv  Detail & Related papers  (2023-08-22T09:17:45Z)
• Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets [84.90242084523565]
  Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics. By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention. By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
  arXiv  Detail & Related papers  (2022-03-07T18:59:54Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.