TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification

• URL: http://arxiv.org/abs/2511.19694v2
• Date: Wed, 26 Nov 2025 07:35:29 GMT
• Title: TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification
• Authors: Chin-Chia Michael Yeh, Uday Singh Saini, Junpeng Wang, Xin Dai, Xiran Fan, Jiarui Sun, Yujie Fan, Yan Zheng,
• Abstract summary: We introduce TiCT, a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification.<n> TiCT achieves competitive performance against state-of-the-art supervised methods.
• Score: 24.157185193971856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ubiquity of time series data creates a strong demand for general-purpose foundation models, yet developing them for classification remains a significant challenge, largely due to the high cost of labeled data. Foundation models capable of in-context learning (ICL) offer a powerful solution, adapting to new tasks with minimal examples and reducing the need for extensive retraining. However, prior work on large-scale time series models has predominantly focused on forecasting, leaving a critical gap for versatile, fine-tuning-free classification. To address this, we introduce TiCT (Time-series in-Context Transformer), a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification. We make two primary technical contributions: 1) a novel architecture featuring a scalable bit-based label encoding and a special output attention mechanism to handle an arbitrary number of classes; and 2) a synthetic pre-training framework that combines a Mixup-inspired process with data augmentation to foster generalization and noise invariance. Extensive evaluations on the UCR Archive show that TiCT achieves competitive performance against state-of-the-art supervised methods. Crucially, this is accomplished using only in-context examples at inference time, without updating a single model weight.

Related papers

• Bagging-Based Model Merging for Robust General Text Embeddings [73.51674133699196]
  General-purpose text embedding models underpin a wide range of NLP and information retrieval applications.<n>We present a systematic study of multi-task training for text embeddings from two perspectives: data scheduling and model merging.<n>We propose Bagging-based rObust mOdel Merging (BOOM), which trains multiple embedding models on sampled subsets and merges them into a single model.
  arXiv  Detail & Related papers  (2026-02-05T15:45:08Z)
• Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learning [19.12792297140574]
  Continual learning aims to incrementally train a model on a sequence of tasks while retaining performance on prior ones.<n> storing and replaying data is often infeasible due to privacy or security constraints.<n>We propose Per-layer Model Inversion (PMI), inspired by faster convergence in single-layer optimization.
  arXiv  Detail & Related papers  (2025-10-30T09:58:48Z)
• Scaling Transformer-Based Novel View Synthesis Models with Token Disentanglement and Synthetic Data [53.040873127309766]
  We propose a token disentanglement process within the transformer architecture, enhancing feature separation and ensuring more effective learning.<n>Our method outperforms existing models on both in-dataset and cross-dataset evaluations.
  arXiv  Detail & Related papers  (2025-09-08T17:58:06Z)
• RoFt-Mol: Benchmarking Robust Fine-Tuning with Molecular Graph Foundation Models [15.62650736139546]
  We classify eight fine-tuning methods into three mechanisms: weight-based, representation-based, and partial fine-tuning.<n>We benchmark these methods on downstream regression and classification tasks across supervised and self-supervised pre-trained models in diverse labeling settings.<n>This evaluation provides valuable insights and informs the design of a refined robust fine-tuning method, ROFT-MOL.
  arXiv  Detail & Related papers  (2025-08-30T21:35:57Z)
• Large EEG-U-Transformer for Time-Step Level Detection Without Pre-Training [1.3254304182988286]
  We propose a simple U-shaped model to efficiently learn representations by capturing both local and global features.<n>Compared to other window-level classification models, our method directly outputs predictions at the time-step level.<n>Our model won 1st place in the 2025 "seizure detection challenge" organized in the International Conference on Artificial Intelligence in Epilepsy and Other Neurological Disorders.
  arXiv  Detail & Related papers  (2025-04-01T01:33:42Z)
• Measuring Pre-training Data Quality without Labels for Time Series Foundation Models [10.64362760848387]
  We introduce contrastive accuracy, a new measure to evaluate the quality of the representation space learned by the foundation model.<n>Our experiments reveal the positive correlation between the proposed measure and the accuracy of the model on a collection of downstream tasks.
  arXiv  Detail & Related papers  (2024-12-09T10:38:30Z)
• TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series [57.4208255711412]
  Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS) We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-02T16:45:19Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Class-Incremental Learning with Strong Pre-trained Models [97.84755144148535]
  Class-incremental learning (CIL) has been widely studied under the setting of starting from a small number of classes (base classes) We explore an understudied real-world setting of CIL that starts with a strong model pre-trained on a large number of base classes. Our proposed method is robust and generalizes to all analyzed CIL settings.
  arXiv  Detail & Related papers  (2022-04-07T17:58:07Z)
• Data Summarization via Bilevel Optimization [48.89977988203108]
  A simple yet powerful approach is to operate on small subsets of data. In this work, we propose a generic coreset framework that formulates the coreset selection as a cardinality-constrained bilevel optimization problem.
  arXiv  Detail & Related papers  (2021-09-26T09:08:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.