Forecast collapse of transformer-based models under squared loss in financial time series

• URL: http://arxiv.org/abs/2604.00064v1
• Date: Tue, 31 Mar 2026 09:06:52 GMT
• Title: Forecast collapse of transformer-based models under squared loss in financial time series
• Abstract summary: We study trajectory forecasting under squared loss for time series with weak conditional structure.<n>We emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate.<n>Increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study trajectory forecasting under squared loss for time series with weak conditional structure, using highly expressive prediction models. Building on the classical characterization of squared-loss risk minimization, we emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate, leading to trivial Bayes-optimal predictors (flat for prices and zero for returns in standard financial settings). In this regime, increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor. These fluctuations arise from the reuse of noise and result in increased prediction variance without any reduction in bias. This provides a process-level explanation for the degradation of Transformerbased forecasts on financial time series. We complement these theoretical results with numerical experiments on high-frequency EUR/USD exchange rate data, analyzing the distribution of trajectory-level forecasting errors. The results show that Transformer-based models yield larger errors than a simple linear benchmark on a large majority of forecasting windows, consistent with the variance-driven mechanism identified by the theory.

Related papers

• Repurposing Deep Limit Order Book Forecasting for Scenario-Conditioned Market Impact Modeling [49.590388401787834]
  We introduce a model-agnostic framework that compares a trained forecaster's predictive distributions before and after injecting mechanically valid counterfactual messages.<n>A Transformer-based forecaster recovered scenario rankings with a Spearman correlation of 0.99 and 97.2% directional agreement with realized historical outcomes.
  arXiv  Detail & Related papers  (2026-09-15T10:06:03Z)
• Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting [64.2640656619583]
  Time series foundation models (TSFMs) have recently achieved strong zero-shot forecasting performance through large-scale pretraining and retrieval-augmented prediction.<n>We propose a Retrieval-guided Invariant-Dynamic DEcomposition framework for time series forecasting.
  arXiv  Detail & Related papers  (2026-05-24T07:31:55Z)
• Forecasting the U.S. Treasury Yield Curve: A Distributionally Robust Machine Learning Approach [0.12891210250935145]
  We study U.S. Treasury yield curve forecasting under distributional uncertainty.<n>Rather than minimizing average forecast error, the forecaster selects a decision rule that minimizes worst case expected loss.<n>We propose a distributionally robust ensemble forecasting framework that integrates factor models with high dimensional nonparametric machine learning models.
  arXiv  Detail & Related papers  (2026-01-08T05:26:43Z)
• Neural CDEs as Correctors for Learned Time Series Models [0.0]
  We propose a Predictor-Corrector mechanism where the Predictor is any learned time-series model and the Corrector is a neural controlled differential equation.<n>The proposed Corrector works with irregularly sampled time series and continuous- and discrete-time Predictors.<n>We evaluate our Corrector with diverse Predictors on synthetic, physics simulation, and real-world forecasting datasets.
  arXiv  Detail & Related papers  (2025-12-13T01:17:05Z)
• Enforcing tail calibration when training probabilistic forecast models [0.0]
  We study how the loss function used to train probabilistic forecast models can be adapted to improve the reliability of forecasts made for extreme events.<n>We demonstrate that state-of-the-art models do not issue calibrated forecasts for extreme wind speeds, and that the calibration of forecasts for extreme events can be improved by suitable adaptations to the loss function during model training.
  arXiv  Detail & Related papers  (2025-06-16T16:51:06Z)
• $K^2$VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting [11.83736650205371]
  Probabilistic Time Series Forecasting (PTSF) plays a crucial role in decision-making across various fields, including economics, energy, and transportation.<n>We introduce $K2$VAE, an efficient VAE-based generative model that transforms nonlinear time series into a linear dynamical system.<n>$K2$VAE outperforms state-of-the-art methods in both short- and long-term PTSF, providing a more efficient and accurate solution.
  arXiv  Detail & Related papers  (2025-05-29T02:52:59Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.