LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities

• URL: http://arxiv.org/abs/2408.10111v3
• Date: Tue, 21 Oct 2025 12:42:19 GMT
• Title: LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities
• Authors: Yuanjian Xu, Anxian Liu, Jianing Hao, Zhenzhuo Li, Shichang Meng, Guang Zhang,
• Abstract summary: We propose textbfLENS, a pre-trained model for financial time series.<n>textbfLENS effectively captures the complexity of financial systems through a carefully crafted model architecture.<n>Our work offers practical insights into developing pre-trained time series models in high-noise environments.
• Score: 3.475989206546412
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling large-scale time series has gained significant attention in recent years. However, its direct application in finance remains challenging due to substantial differences in data characteristics across domains. Specifically, financial systems feature inherent stochasticity and low signal-to-noise ratios, rendering traditional methods and pre-training approaches ineffective. This underscores the urgent need for a foundation model tailored to financial time series. To bridge this gap, we propose \textbf{LENS}, a pre-trained model for this domain. \textbf{LENS} effectively captures the complexity of financial stochastic systems through a carefully crafted model architecture and mitigates noise during pre-training by using an invertible embedding module. We provide a rigorous theoretical explanation of the model's effectiveness and validate its performance through extensive experiments. Pre-trained on a dataset comprising 100 billion financial observations, \textbf{LENS} achieves exceptional results across a wide range of critical downstream tasks. Moreover, our work offers practical insights into developing pre-trained time series models in high-noise environments, paving the way for further advancements in this pivotal research domain.

Related papers

• How to Set the Learning Rate for Large-Scale Pre-training? [73.03133634525635]
  We formalize this investigation into two distinct research paradigms: Fitting and Transfer.<n>Within the Fitting Paradigm, we introduce a Scaling Law for search factor, effectively reducing the search complexity from O(n3) to O(n*C_D*C_) via predictive modeling.<n>We extend the principles of $$Transfer to the Mixture of Experts (MoE) architecture, broadening its applicability to encompass model depth, weight decay, and token horizons.
  arXiv  Detail & Related papers  (2026-01-08T15:55:13Z)
• Re(Visiting) Time Series Foundation Models in Finance [3.295157175236371]
  Financial time series forecasting is central to trading, portfolio optimization, and risk management.<n>Recent advances in time series foundation models (TSFMs) offer a new paradigm for learning generalizable temporal representations from large and diverse datasets.<n>This paper presents the first comprehensive empirical study of TSFMs in global financial markets.
  arXiv  Detail & Related papers  (2025-11-23T18:44:19Z)
• Intention-Conditioned Flow Occupancy Models [69.79049994662591]
  Large-scale pre-training has fundamentally changed how machine learning research is done today.<n>Applying this same framework to reinforcement learning is appealing because it offers compelling avenues for addressing core challenges in RL.<n>Recent advances in generative AI have provided new tools for modeling highly complex distributions.
  arXiv  Detail & Related papers  (2025-06-10T15:27:46Z)
• Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions [0.0]
  Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering.<n>We introduce $textbfCHARM$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations.<n>The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability.
  arXiv  Detail & Related papers  (2025-05-20T15:58:54Z)
• DELPHYNE: A Pre-Trained Model for General and Financial Time Series [2.601248228220401]
  Time-series data is valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data.<n>Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains.<n>However, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings.
  arXiv  Detail & Related papers  (2025-05-12T16:53:29Z)
• LLM4FTS: Enhancing Large Language Models for Financial Time Series Prediction [0.0]
  Traditional machine learning models exhibit limitations in this forecasting task constrained by their restricted model capacity.<n>We propose $LLM4FTS$, a novel framework that enhances temporal sequence modeling through learnable patch segmentation and dynamic wavelet convolution modules.<n>Experiments on real-world financial datasets substantiate the framework's efficacy, demonstrating superior performance in capturing complex market patterns and achieving state-of-the-art results in stock return prediction.
  arXiv  Detail & Related papers  (2025-05-05T06:48:34Z)
• ZiGong 1.0: A Large Language Model for Financial Credit [8.49779245416985]
  Large Language Models (LLMs) have demonstrated strong performance across various general Natural Language Processing (NLP) tasks.<n>However, their effectiveness in financial credit assessment applications remains suboptimal.<n>We propose ZiGong, a Mistral-based model enhanced through multi-task supervised fine-tuning.
  arXiv  Detail & Related papers  (2025-02-22T09:27:56Z)
• Sundial: A Family of Highly Capable Time Series Foundation Models [64.6322079384575]
  We introduce Sundial, a family of native, flexible, and scalable time series foundation models.<n>Our models are pre-trained without specifying any prior distribution and can generate multiple probable predictions.<n>Sundial achieves state-of-the-art results on both point and probabilistic forecasting benchmarks with a just-in-time inference speed.
  arXiv  Detail & Related papers  (2025-02-02T14:52:50Z)
• BreakGPT: Leveraging Large Language Models for Predicting Asset Price Surges [55.2480439325792]
  This paper introduces BreakGPT, a novel large language model (LLM) architecture adapted specifically for time series forecasting and the prediction of sharp upward movements in asset prices. We showcase BreakGPT as a promising solution for financial forecasting with minimal training and as a strong competitor for capturing both local and global temporal dependencies.
  arXiv  Detail & Related papers  (2024-11-09T05:40:32Z)
• Large Language Models for Financial Aid in Financial Time-series Forecasting [0.4218593777811082]
  Time series forecasting in financial aid is difficult due to limited historical datasets and high dimensional financial information. We use state-of-the-art time series models including pre-trained LLMs (GPT-2 as the backbone), transformers, and linear models to demonstrate their ability to outperform traditional approaches.
  arXiv  Detail & Related papers  (2024-10-24T12:41:47Z)
• Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts [25.503695417712997]
  Time-MoE is a scalable and unified architecture designed to pre-train larger, more capable forecasting foundation models. Time-MoE enhances computational efficiency by activating only a subset of networks for each prediction. For the first time, we scaled a time series foundation model up to 2.4 billion parameters, achieving significantly improved forecasting precision.
  arXiv  Detail & Related papers  (2024-09-24T12:42:18Z)
• Impact of Noisy Supervision in Foundation Model Learning [91.56591923244943]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets.<n>We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences [0.0]
  We present a generative pretraining method that can be used to obtain contextualised embeddings of financial transactions. We additionally perform large-scale pretraining of an embedding model using a corpus of data from 180 issuing banks containing 5.1 billion transactions.
  arXiv  Detail & Related papers  (2024-01-03T09:32:48Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• Bilinear Input Normalization for Neural Networks in Financial Forecasting [101.89872650510074]
  We propose a novel data-driven normalization method for deep neural networks that handle high-frequency financial time-series. The proposed normalization scheme takes into account the bimodal characteristic of financial time-series. Our experiments, conducted with state-of-the-arts neural networks and high-frequency data, show significant improvements over other normalization techniques.
  arXiv  Detail & Related papers  (2021-09-01T07:52:03Z)
• Low-Rank Temporal Attention-Augmented Bilinear Network for financial time-series forecasting [93.73198973454944]
  Deep learning models have led to significant performance improvements in many problems coming from different domains, including prediction problems of financial time-series data. The Temporal Attention-Augmented Bilinear network was recently proposed as an efficient and high-performing model for Limit Order Book time-series forecasting. In this paper, we propose a low-rank tensor approximation of the model to further reduce the number of trainable parameters and increase its speed.
  arXiv  Detail & Related papers  (2021-07-05T10:15:23Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)
• Modeling Financial Time Series using LSTM with Trainable Initial Hidden States [0.0]
  We introduce a novel approach to modeling financial time series using a deep learning model. We use a Long Short-Term Memory (LSTM) network equipped with the trainable initial hidden states.
  arXiv  Detail & Related papers  (2020-07-14T06:36:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.