Tabby: An Open Pretraining Recipe for Time Series Foundation Models

• URL: http://arxiv.org/abs/2609.13956v1
• Date: Sat, 12 Sep 2026 13:58:39 GMT
• Title: Tabby: An Open Pretraining Recipe for Time Series Foundation Models
• Abstract summary: Tabby is a long context probabilistic time series foundation model.<n>It adopts an encoder-only patch Transformer architecture and concentrates the contributions on the data and the training procedure.<n>We release training pipeline and model as open source at huawei-noah/trustworthyAI.
• Score: 36.79272249788465
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this report, we release Tabby, a long context probabilistic time series foundation model, together with a complete and open recipe of how it was built. Tabby adopts an encoder-only patch Transformer architecture and concentrates the contributions on the data and the training procedure. The pretraining corpus combines an extended real-world collection, GIFT-Eval-Pretrain+ and BLAST, with synthetic data from KernelSynth and CauKerV2, an online generator that composes temporal dynamics through randomly sampled structural causal models. Training couples a progressive convergence schedule, which yields reusable intermediate checkpoints, with a deep quantile supervision objective for intermediate layers. The resulting 145M parameter backbone supports contexts of up to 8,192 observations and serves forecasting, classification, and anomaly detection, while a prompt-tuning module further improves in-distribution forecasting performance with the pretrained weights frozen. Tabby achieves competitive zero-shot forecasting performance on GIFT-Eval and the out-of-distribution TIME benchmark, while the same pretrained backbone also supports classification on the UCR Archive and zero-shot anomaly detection on TSB-AD-U. We release training pipeline and model as open source at huawei-noah/trustworthyAI.

Related papers

• Beyond Extrapolation: Knowledge Utilization Paradigm with Bidirectional Inspiration for Time Series Forecasting [81.44658512649633]
  KUP-BI is a new time-series modeling paradigm that distills continuation-style knowledge from a emphtrain-only historical library.<n>It consistently improves the forecasting performance of state-of-the-art models, with small additional overhead.
  arXiv  Detail & Related papers  (2026-05-19T01:43:00Z)
• Latent Chain-of-Thought Improves Structured-Data Transformers [26.742304109602614]
  This paper explores latent chain-of-thought and the impact of depth and looping for structured data.<n>We show that latent chain-of-thought is a useful axis for scaling test-time compute for structured data.
  arXiv  Detail & Related papers  (2026-05-11T21:41:21Z)
• Inference-Time Refinement Closes the Synthetic-Real Gap in Tabular Diffusion [8.745106905496282]
  We introduce an inferencetime refinement framework that operates on a frozen pre-trained backbone.<n>Inference-time refinement reaches and exceeds real-data utility in 1 to 80 minutes on a single consumer-grade GPU.
  arXiv  Detail & Related papers  (2026-05-07T13:37:03Z)
• Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling [51.78972657142583]
  We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K.<n>To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions.
  arXiv  Detail & Related papers  (2026-03-05T04:13:57Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer [2.1677183904102257]
  We present an Adversarially Pre-trained Transformer (APT) that is able to perform zero-shot meta-learning on tabular prediction tasks without pre-training on any real-world dataset.<n>APT is pre-trained with adversarial synthetic data agents, who deliberately challenge the model with different synthetic datasets.<n>We show that our framework matches state-of-the-art performance on small classification tasks without filtering on dataset characteristics.
  arXiv  Detail & Related papers  (2025-02-06T23:58:11Z)
• Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization [74.3339999119713]
  We develop a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies.<n>Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon.
  arXiv  Detail & Related papers  (2024-12-06T18:22:59Z)
• Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data [39.40116554523575]
  We present Drift-Resilient TabPFN, a fresh approach based on In-Context Learning with a Prior-Data Fitted Network. It learns to approximate Bayesian inference on synthetic datasets drawn from a prior. It improves accuracy from 0.688 to 0.744 and ROC AUC from 0.786 to 0.832 while maintaining stronger calibration.
  arXiv  Detail & Related papers  (2024-11-15T23:49:23Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.