TRACE: Time SeRies PArameter EffiCient FinE-tuning

• URL: http://arxiv.org/abs/2503.16991v1
• Date: Fri, 21 Mar 2025 09:55:43 GMT
• Title: TRACE: Time SeRies PArameter EffiCient FinE-tuning
• Authors: Yuze Li, Wei Zhu,
• Abstract summary: We propose an efficient fine-tuning method for time series foundation models, termed TRACE: Time Series Efficient Fine-tuning.<n>For long-term forecasting tasks, tailored fine-tuning can significantly enhance performance.
• Score: 4.398852373014641
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose an efficient fine-tuning method for time series foundation models, termed TRACE: Time Series Parameter Efficient Fine-tuning. While pretrained time series foundation models are gaining popularity, they face the following challenges: (1) Unlike natural language tasks, time series data vary in frequency, channel numbers, historical/prediction lengths. For long-term forecasting tasks in particular, tailored fine-tuning can significantly enhance performance.(2) Existing parameter-efficient tuning methods like LoRA remain applicable but require adaptation to temporal characteristics. To address these challenges, our TRACE framework introduces two key innovations: (1) Gated DSIC (Gated Dynamic Simulation Importance Calculation), an unbiased LoRA module importance selection mechanism that ensures conditional parameter consistency before and after masking. Experiments demonstrate that Gated DSIC outperforms common fine-tuning. (2) Reconstructed prediction heads for long-term forecasting tasks, which achieve comparable or superior performance to linear probing heads while drastically reducing parameter counts. Extensive experiments on long-/short-term forecasting and anomaly detection tasks across diverse datasets, coupled with ablation studies, validate the effectiveness of our method.

Related papers

• STTS-EAD: Improving Spatio-Temporal Learning Based Time Series Prediction via [7.247017092359663]
  We propose STTS-EAD, an end-to-end method that seamlessly integrates anomaly into the training process of time series forecasting. Our proposed STTS-EAD leveragestemporal information for forecasting and anomaly detection, with the two parts alternately executed and optimized for each other. Our experiments show that our proposed method can effectively process anomalies detected in the training stage to improve forecasting performance in the inference stage and significantly outperform baselines.
  arXiv  Detail & Related papers  (2025-01-14T03:26:05Z)
• MATEY: multiscale adaptive foundation models for spatiotemporal physical systems [2.7767126393602726]
  We propose two adaptive tokenization schemes that dynamically adjust patch sizes based on local features.<n>We evaluate the performance of a proposed multiscale adaptive model, MATEY, in a sequence of experiments.<n>We also demonstrate fine-tuning tasks featuring different physics that models pretrained on PDE data.
  arXiv  Detail & Related papers  (2024-12-29T22:13:16Z)
• Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation [69.60321475454843]
  We propose DPCPL, the first pre-training and prompt-tuning paradigm tailored for Multi-Behavior Sequential Recommendation. In the pre-training stage, we propose a novel Efficient Behavior Miner (EBM) to filter out the noise at multiple time scales. Subsequently, we propose to tune the pre-trained model in a highly efficient manner with the proposed Customized Prompt Learning (CPL) module.
  arXiv  Detail & Related papers  (2024-08-21T06:48:38Z)
• Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models [68.23649978697027]
  Forecast-PEFT is a fine-tuning strategy that freezes the majority of the model's parameters, focusing adjustments on newly introduced prompts and adapters. Our experiments show that Forecast-PEFT outperforms traditional full fine-tuning methods in motion prediction tasks. Forecast-FT further improves prediction performance, evidencing up to a 9.6% enhancement over conventional baseline methods.
  arXiv  Detail & Related papers  (2024-07-28T19:18:59Z)
• TimeSieve: Extracting Temporal Dynamics through Information Bottlenecks [31.10683149519954]
  We propose an innovative time series forecasting model TimeSieve. Our approach employs wavelet transforms to preprocess time series data, effectively capturing multi-scale features. Our results validate the effectiveness of our approach in addressing the key challenges in time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T15:58:12Z)
• Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting [46.63798583414426]
  Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key to containing excessive model inflation. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks.
  arXiv  Detail & Related papers  (2024-01-22T13:15:40Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning [55.50908600818483]
  Fine-tuning large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. We propose an Effective and Efficient Visual Prompt Tuning (E2VPT) approach for large-scale transformer-based model adaptation. Our approach outperforms several state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2023-07-25T19:03:21Z)
• Mlinear: Rethink the Linear Model for Time-series Forecasting [9.841293660201261]
  Mlinear is a simple yet effective method based mainly on linear layers. We introduce a new loss function that significantly outperforms the widely used mean squared error (MSE) on multiple datasets. Our method significantly outperforms PatchTST with a ratio of 21:3 at 336 sequence length input and 29:10 at 512 sequence length input.
  arXiv  Detail & Related papers  (2023-05-08T15:54:18Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• Grouped self-attention mechanism for a memory-efficient Transformer [64.0125322353281]
  Real-world tasks such as forecasting weather, electricity consumption, and stock market involve predicting data that vary over time. Time-series data are generally recorded over a long period of observation with long sequences owing to their periodic characteristics and long-range dependencies over time. We propose two novel modules, Grouped Self-Attention (GSA) and Compressed Cross-Attention (CCA) Our proposed model efficiently exhibited reduced computational complexity and performance comparable to or better than existing methods.
  arXiv  Detail & Related papers  (2022-10-02T06:58:49Z)
• Conformal Prediction Bands for Two-Dimensional Functional Time Series [0.0]
  Time evolving surfaces can be modeled as two-dimensional Functional time series, exploiting the tools of Functional data analysis. The main focus revolves around Conformal Prediction, a versatile non-parametric paradigm used to quantify uncertainty in prediction problems. A probabilistic forecasting scheme for two-dimensional functional time series is presented, while providing an extension of Functional Autoregressive Processes of order one to this setting.
  arXiv  Detail & Related papers  (2022-07-27T17:23:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.