R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models

• URL: http://arxiv.org/abs/2511.11685v1
• Date: Wed, 12 Nov 2025 08:32:08 GMT
• Title: R-Tuning: Wavelet-Decomposed Replay and Semantic Alignment for Continual Adaptation of Pretrained Time-Series Models
• Abstract summary: Replay Tuning (R-Tuning) is a novel framework designed for the continual adaptation of pre-trained time-series models.<n>R-Tuning constructs a unified latent space that captures both prior and current task knowledge through a frequency-aware replay strategy.
• Score: 15.189716355297326
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-trained models have demonstrated exceptional generalization capabilities in time-series forecasting; however, adapting them to evolving data distributions remains a significant challenge. A key hurdle lies in accessing the original training data, as fine-tuning solely on new data often leads to catastrophic forgetting. To address this issue, we propose Replay Tuning (R-Tuning), a novel framework designed for the continual adaptation of pre-trained time-series models. R-Tuning constructs a unified latent space that captures both prior and current task knowledge through a frequency-aware replay strategy. Specifically, it augments model-generated samples via wavelet-based decomposition across multiple frequency bands, generating trend-preserving and fusion-enhanced variants to improve representation diversity and replay efficiency. To further reduce reliance on synthetic samples, R-Tuning introduces a latent consistency constraint that aligns new representations with the prior task space. This constraint guides joint optimization within a compact and semantically coherent latent space, ensuring robust knowledge retention and adaptation. Extensive experimental results demonstrate the superiority of R-Tuning, which reduces MAE and MSE by up to 46.9% and 46.8%, respectively, on new tasks, while preserving prior knowledge with gains of up to 5.7% and 6.0% on old tasks. Notably, under few-shot settings, R-Tuning outperforms all state-of-the-art baselines even when synthetic proxy samples account for only 5% of the new task dataset.

Related papers

• Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data [119.14301046837822]
  We show that Mixture-of-Experts (MoEs) degrade more rapidly under data repetition.<n>MoEs can outperform dense models even when data is repeated more than 64 times.<n>We analyze internal mechanisms correlated with MoE overfitting in high repetition regimes.
  arXiv  Detail & Related papers  (2026-09-10T17:57:33Z)
• Counterfactual Residual Data Augmentation for Regression [25.42873557456817]
  Real-world regression tasks often suffer from limited training samples, high collection costs, and noisy observations.<n>Inspired by the impact of data augmentation in vision and language, we propose a novel Counterfactual Residual Data Augmentation technique.<n>Our method offers a simple and efficient remedy for noise-prone, small-sample regression settings.
  arXiv  Detail & Related papers  (2026-06-26T13:04:37Z)
• DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay [57.80564154223355]
  Existing experience replay methods address this by reusing accurate samples for direct policy updates.<n>We argue that historical data should prioritize sustaining diversity rather than simply reinforcing accuracy.<n>We propose Dynamic Jensen-Shannon Replay (LEPJR), a simple yet effective regularization framework.
  arXiv  Detail & Related papers  (2026-03-17T06:20:56Z)
• MSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning [36.119948355819]
  We propose an experience replay framework that estimates sample-level memory strength and schedules rehearsal at adaptive intervals to mitigate catastrophic forgetting.<n>Extensive experiments across three backbone models and 11 sequential tasks show that MSSR consistently outperforms state-of-the-art replay baselines.
  arXiv  Detail & Related papers  (2026-03-10T16:49:44Z)
• FOZO: Forward-Only Zeroth-Order Prompt Optimization for Test-Time Adaptation [9.28697795097814]
  Test-Time Adaptation is essential for enabling deep learning models to handle real-world data distribution shifts.<n>Backpropagation-based methods are not suitable for low-end deployment devices.<n>We propose Forward-Only Zeroth-Order Optimization (FOZO), a novel and practical backpropagation-free paradigm for TTA.
  arXiv  Detail & Related papers  (2026-03-05T02:12:48Z)
• Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols [123.73663884421272]
  Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation algorithms.<n>We establish FEWTRANS, a comprehensive benchmark containing 10 diverse datasets.<n>By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research.
  arXiv  Detail & Related papers  (2026-02-28T05:41:57Z)
• AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs [20.237470399970448]
  Test time adaptation (TTA) has emerged as a promising solution to adapt pre-trained models to new, unseen data distributions.<n>This paper presents AdaNODEs, an innovative source-free TTA method tailored explicitly for time series forecasting.
  arXiv  Detail & Related papers  (2026-01-19T09:46:54Z)
• Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning [71.30276778807068]
  We propose a unified framework that strategically coordinates sample pruning and token pruning.<n>Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data.
  arXiv  Detail & Related papers  (2025-09-28T13:27:38Z)
• Automated Tuning for Diffusion Inverse Problem Solvers without Generative Prior Retraining [4.511561231517167]
  Diffusion/score-based models have emerged as powerful generative priors for solving inverse problems.<n>We propose Zero-shot Adaptive Diffusion Sampling (ZADS), a test-time optimization method that tunes fidelity weights across arbitrary noise schedules.<n>ZADS consistently outperforms both traditional compressed sensing and recent diffusion-based methods.
  arXiv  Detail & Related papers  (2025-09-11T22:22:32Z)
• TSPulse: Dual Space Tiny Pre-Trained Models for Rapid Time-Series Analysis [12.034816114258803]
  TSPulse is an ultra-compact time-series pre-trained model with only 1M parameters.<n>It performs strongly across classification, anomaly detection, imputation, and retrieval tasks.<n>Results are achieved with just 1M parameters (10-100X smaller than existing SOTA models)
  arXiv  Detail & Related papers  (2025-05-19T12:18:53Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• ReAugment: Model Zoo-Guided RL for Few-Shot Time Series Augmentation and Forecasting [74.00765474305288]
  We present a pilot study on using reinforcement learning (RL) for time series data augmentation.<n>Our method, ReAugment, tackles three critical questions: which parts of the training set should be augmented, how the augmentation should be performed, and what advantages RL brings to the process.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability [73.34532767873785]
  We propose the concept of Information Density'' (ID) to indicate whether a matrix strongly belongs to certain feature spaces. We introduce the Dense Information Prompt (DIP) to enhance information density to improve generalization. DIP significantly reduces the number of tunable parameters and the requisite storage space, making it particularly advantageous in resource-constrained settings.
  arXiv  Detail & Related papers  (2023-12-17T20:42:43Z)
• Efficient Stitchable Task Adaptation [47.94819192325723]
  We present a novel framework, Efficient Stitchable Task Adaptation (ESTA), to efficiently produce a palette of fine-tuned models. Specifically, we first tailor parameter-efficient fine-tuning to share low-rank updates among the stitches. We streamline a simple yet effective one-stage deployment pipeline, which estimates the important stitches to deploy.
  arXiv  Detail & Related papers  (2023-11-29T04:31:35Z)
• Financial Time Series Data Augmentation with Generative Adversarial Networks and Extended Intertemporal Return Plots [2.365537081046599]
  We apply state-of-the art image-based generative models for the task of data augmentation. We introduce the extended intertemporal return plot (XIRP), a new image representation for time series. Our approach proves to be effective in reducing the return forecast error by 7% on 79% of the financial data sets.
  arXiv  Detail & Related papers  (2022-05-18T13:39:27Z)
• Investigating Tradeoffs in Real-World Video Super-Resolution [90.81396836308085]
  Real-world video super-resolution (VSR) models are often trained with diverse degradations to improve generalizability. To alleviate the first tradeoff, we propose a degradation scheme that reduces up to 40% of training time without sacrificing performance. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences.
  arXiv  Detail & Related papers  (2021-11-24T18:58:21Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.