In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks

• URL: http://arxiv.org/abs/2602.20307v1
• Date: Mon, 23 Feb 2026 19:48:47 GMT
• Title: In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks
• Abstract summary: Time-series foundation models (TSFMs) have demonstrated strong generalization capabilities across diverse datasets and tasks.<n>We propose augmenting TSFMs with In-Context Learning (ICL) capabilities, enabling them to perform test-time inference.<n>Our framework, In-Context Time-series Pre-training (ICTP), restructures the original pre-training data to equip the backbone TSFM with ICL capabilities.
• Score: 22.20716752724626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time-series foundation models (TSFMs) have demonstrated strong generalization capabilities across diverse datasets and tasks. However, existing foundation models are typically pre-trained to enhance performance on specific tasks and often struggle to generalize to unseen tasks without fine-tuning. To address this limitation, we propose augmenting TSFMs with In-Context Learning (ICL) capabilities, enabling them to perform test-time inference by dynamically adapting to input-output relationships provided within the context. Our framework, In-Context Time-series Pre-training (ICTP), restructures the original pre-training data to equip the backbone TSFM with ICL capabilities, enabling adaptation to unseen tasks. Experiments demonstrate that ICT improves the performance of state-of-the-art TSFMs by approximately 11.4% on unseen tasks without requiring fine-tuning.

Related papers

• SkillTFM: Gated Skill Evolution for Training-Free Adaptation of Tabular Foundation Models [59.078670951794415]
  We propose SkillTFM, a training-free system that shifts Tabular Foundation Models from gated evolution of agentic skills to reusable skills.<n>Across simulated boundary settings and real-world electricity-price forecasting, SkillTFM improves AUC by 0.128--0.142, raises nonlinear-boundary AUC from 0.699 to 0.898.
  arXiv  Detail & Related papers  (2026-08-06T15:09:02Z)
• Post-Training in Time Series Foundation Models: A Unifying Framework [42.66753732684461]
  Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis.<n>Pretraining alone is often insufficient for reliable downstream deployment.<n>Post-training is a broad class of methods to adapt, augment, compose, calibrate, or specialize pretrained TSFMs for downstream tasks.
  arXiv  Detail & Related papers  (2026-07-22T10:37:10Z)
• Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning [8.604749005323557]
  We propose ETAP (Ensemble Task Affinity Predictor), a scalable framework that integrates principled and data-driven estimators to predict MTL performance gains.<n>We demonstrate that ETAP improves MTL gain prediction and enables more effective task grouping, outperforming state-of-the-art baselines across diverse application domains.
  arXiv  Detail & Related papers  (2026-02-20T19:57:38Z)
• Dynamic Routing Between Experts: A Data-Efficient Approach to Continual Learning in Vision-Language Models [10.431923437214719]
  Vision-Language Models (VLMs) suffer from catastrophic forgetting when sequentially fine-tuned on new tasks.<n>We introduce a routing-based approach that enables the integration of new tasks while preserving the foundational knowledge acquired during pretraining.
  arXiv  Detail & Related papers  (2025-11-03T18:39:32Z)
• Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models [64.02612380298228]
  Recent studies have explored the idea of continuing to train a model at test-time for a given task, known as test-time training (TTT)<n>We propose a model in which TTT achieves a substantially smaller in-distribution test error than global training.<n>We empirically validate our model's key assumptions by training a sparse autoencoder on ImageNet.
  arXiv  Detail & Related papers  (2025-09-29T09:24:52Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• Can Time-Series Foundation Models Perform Building Energy Management Tasks? [5.450531952940644]
  Building energy management tasks require processing and learning from a variety of time-series data.<n>Existing solutions rely on bespoke task- and data-specific models to perform these tasks.<n>Inspired by the transformative success of Large Language Models (LLMs), Time-Series Foundation Models (TSFMs) have the potential to change this.
  arXiv  Detail & Related papers  (2025-06-12T19:45:10Z)
• UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplines [64.84631333071728]
  We introduce bfUnistage, a unified Transformer-based framework fortemporal modeling.<n>Our work demonstrates that a task-specific vision-text can build a generalizable model fortemporal learning.<n>We also introduce a temporal module to incorporate temporal dynamics explicitly.
  arXiv  Detail & Related papers  (2025-03-26T17:33:23Z)
• Federated Continual Instruction Tuning [39.344583304181135]
  Federated learning (FL) has the potential to leverage all distributed data and training resources to reduce the overhead of joint training.<n>In this work, we introduce the Federated Continual Instruction Tuning (FCIT) benchmark to model this real-world challenge.
  arXiv  Detail & Related papers  (2025-03-17T07:58:06Z)
• The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities [51.594836904623534]
  We investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples.<n>We show that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts.<n>Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve.
  arXiv  Detail & Related papers  (2025-01-15T10:57:55Z)
• Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective [125.00228936051657]
  We introduce NTK-CL, a novel framework that eliminates task-specific parameter storage while adaptively generating task-relevant features.<n>By fine-tuning optimizable parameters with appropriate regularization, NTK-CL achieves state-of-the-art performance on established PEFT-CL benchmarks.
  arXiv  Detail & Related papers  (2024-07-24T09:30:04Z)
• Hierarchical Few-Shot Imitation with Skill Transition Models [66.81252581083199]
  Few-shot Imitation with Skill Transition Models (FIST) is an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments.
  arXiv  Detail & Related papers  (2021-07-19T15:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.