DELPHYNE: A Pre-Trained Model for General and Financial Time Series

• URL: http://arxiv.org/abs/2506.06288v1
• Date: Mon, 12 May 2025 16:53:29 GMT
• Title: DELPHYNE: A Pre-Trained Model for General and Financial Time Series
• Authors: Xueying Ding, Aakriti Mittal, Achintya Gopal,
• Abstract summary: Time-series data is valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data.<n>Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains.<n>However, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings.
• Score: 2.601248228220401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time-series data is a vital modality within data science communities. This is particularly valuable in financial applications, where it helps in detecting patterns, understanding market behavior, and making informed decisions based on historical data. Recent advances in language modeling have led to the rise of time-series pre-trained models that are trained on vast collections of datasets and applied to diverse tasks across financial domains. However, across financial applications, existing time-series pre-trained models have not shown boosts in performance over simple finance benchmarks in both zero-shot and fine-tuning settings. This phenomenon occurs because of a i) lack of financial data within the pre-training stage, and ii) the negative transfer effect due to inherently different time-series patterns across domains. Furthermore, time-series data is continuous, noisy, and can be collected at varying frequencies and with varying lags across different variables, making this data more challenging to model than languages. To address the above problems, we introduce a Pre-trained MoDEL for FINance TimE-series (Delphyne). Delphyne achieves competitive performance to existing foundation and full-shot models with few fine-tuning steps on publicly available datasets, and also shows superior performances on various financial tasks.

Related papers

• Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models [104.17057231661371]
  Time series analysis is crucial for understanding dynamics of complex systems.<n>Recent advances in foundation models have led to task-agnostic Time Series Foundation Models (TSFMs) and Large Language Model-based Time Series Models (TSLLMs)<n>Their success depends on large, diverse, and high-quality datasets, which are challenging to build due to regulatory, diversity, quality, and quantity constraints.<n>This survey provides a comprehensive review of synthetic data for TSFMs and TSLLMs, analyzing data generation strategies, their role in model pretraining, fine-tuning, and evaluation, and identifying future research directions.
  arXiv  Detail & Related papers  (2025-03-14T13:53:46Z)
• Large Language Models for Financial Aid in Financial Time-series Forecasting [0.4218593777811082]
  Time series forecasting in financial aid is difficult due to limited historical datasets and high dimensional financial information. We use state-of-the-art time series models including pre-trained LLMs (GPT-2 as the backbone), transformers, and linear models to demonstrate their ability to outperform traditional approaches.
  arXiv  Detail & Related papers  (2024-10-24T12:41:47Z)
• Recent Trends in Modelling the Continuous Time Series using Deep Learning: A Survey [0.18434042562191813]
  Continuous-time series is essential for different modern application areas, e.g. healthcare, automobile, energy, finance, Internet of things (IoT) This paper has described the general problem domain of time series and reviewed the challenges of modelling the continuous time series.
  arXiv  Detail & Related papers  (2024-09-13T14:19:44Z)
• PLUTUS: A Well Pre-trained Large Unified Transformer can Unveil Financial Time Series Regularities [0.848210898747543]
  Financial time series modeling is crucial for understanding and predicting market behaviors. Traditional models struggle to capture complex patterns due to non-linearity, non-stationarity, and high noise levels. Inspired by the success of large language models in NLP, we introduce $textbfPLUTUS$, a $textbfP$re-trained $textbfL$arge. PLUTUS is the first open-source, large-scale, pre-trained financial time series model with over one billion parameters.
  arXiv  Detail & Related papers  (2024-08-19T15:59:46Z)
• Text2TimeSeries: Enhancing Financial Forecasting through Time Series Prediction Updates with Event-Driven Insights from Large Language Models [9.991327369572819]
  We propose a collaborative modeling framework that incorporates textual information about relevant events for predictions. We leverage the intuition of large language models about future changes to update real number time series predictions.
  arXiv  Detail & Related papers  (2024-07-04T07:21:38Z)
• MOMENT: A Family of Open Time-series Foundation Models [19.0845213853369]
  We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. We compile a collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges. We build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings.
  arXiv  Detail & Related papers  (2024-02-06T10:48:46Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting [59.11817101030137]
  This research advocates for a unified model paradigm that transcends domain boundaries. Learning an effective cross-domain model presents the following challenges. We propose UniTime for effective cross-domain time series learning.
  arXiv  Detail & Related papers  (2023-10-15T06:30:22Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Low-Rank Temporal Attention-Augmented Bilinear Network for financial time-series forecasting [93.73198973454944]
  Deep learning models have led to significant performance improvements in many problems coming from different domains, including prediction problems of financial time-series data. The Temporal Attention-Augmented Bilinear network was recently proposed as an efficient and high-performing model for Limit Order Book time-series forecasting. In this paper, we propose a low-rank tensor approximation of the model to further reduce the number of trainable parameters and increase its speed.
  arXiv  Detail & Related papers  (2021-07-05T10:15:23Z)
• Gaussian process imputation of multiple financial series [71.08576457371433]
  Multiple time series such as financial indicators, stock prices and exchange rates are strongly coupled due to their dependence on the latent state of the market. We focus on learning the relationships among financial time series by modelling them through a multi-output Gaussian process.
  arXiv  Detail & Related papers  (2020-02-11T19:18:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.