When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

• URL: http://arxiv.org/abs/2607.04919v1
• Date: Mon, 06 Jul 2026 10:48:11 GMT
• Title: When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
• Abstract summary: Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets.<n>One robust deployment rule requires no model training: if n_train 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We provide the first systematic break-even analysis answering when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned foundation models (Chronos, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets - GPU deployment is unconditionally justified on these regardless of data volume. On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data (21-2,768 samples); on the remaining 9, break-even ranges from 24 to 8,361 samples. One robust deployment rule requires no model training: if n_train < 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning - this resolves 10 of 30 deployment decisions immediately. Contrary to common practice, LoRA fine-tuning can actively degrade performance on short series. We operationalise these findings as a two-step decision framework - compute dataset length and seasonality strength, run a brief 5-10% pilot only if needed - enabling practitioners to make the FM-versus-classical decision before committing to full infrastructure. Four dataset features motivate mechanistic hypotheses for the remaining cases, though reliable automated prediction at this benchmark scale remains an open problem. Code, benchmark, and decision tools are available at https://github.com/nicolaisi/fm-breakeven.

Related papers

• Learning to Fine-tune Foundation Models under Resource Limitations [2.7071541526963805]
  We study the problem of optimal continual fine-tuning for a pre-trained Foundation Model deployed at a resource-limited device.<n>Our objective is to learn an optimal policy that determines emphwhen to fine-tune the model under a finite compute budget.<n> Experiments with a large pre-trained model on a widely-used text classification dataset demonstrate that our method consistently outperforms fine-tuning approaches with the same compute budget.
  arXiv  Detail & Related papers  (2026-07-12T10:32:32Z)
• ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response [33.890280378630884]
  Time series question-answering (TSQA) is a promising yet underexplored capability of foundation models.<n>ARFBench consists of 750 questions across 142 time series and 5.38M data points from 63 production incidents sourced exclusively from internal telemetry at Datadog.<n>We develop a novel TSFM + VLM hybrid prototype which we post-train on a small set of synthetic and real data that yields comparable overall F1 and accuracy with frontier models.
  arXiv  Detail & Related papers  (2026-04-23T01:45:10Z)
• Pioneer Agent: Continual Improvement of Small Language Models in Production [6.452260317191361]
  Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization.<n>We present Pioneer Agent, a closed-loop system that automates the process of adapting small language models to a specific task.
  arXiv  Detail & Related papers  (2026-04-10T18:13:09Z)
• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• Intention-Conditioned Flow Occupancy Models [80.42634994902858]
  Large-scale pre-training has fundamentally changed how machine learning research is done today.<n>Applying this same framework to reinforcement learning is appealing because it offers compelling avenues for addressing core challenges in RL.<n>Recent advances in generative AI have provided new tools for modeling highly complex distributions.
  arXiv  Detail & Related papers  (2025-06-10T15:27:46Z)
• FRoundation: Are Foundation Models Ready for Face Recognition? [8.045296450065019]
  We investigate whether foundation models are suitable for the specific domain of face recognition (FR)<n>We propose and demonstrate the adaptation of these models for FR across different levels of data availability, including synthetic data.<n>Our results indicate that, despite their versatility, pre-trained foundation models tend to underperform in FR in comparison with similar architectures trained specifically for this task.
  arXiv  Detail & Related papers  (2024-10-31T11:21:21Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance [68.18779562801762]
  multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance. Our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found.
  arXiv  Detail & Related papers  (2024-04-04T17:58:02Z)
• Revisiting Few-Shot Object Detection with Vision-Language Models [49.79495118650838]
  We revisit the task of few-shot object detection (FSOD) in the context of recent foundational vision-language models (VLMs) We propose Foundational FSOD, a new benchmark protocol that evaluates detectors pre-trained on any external data. We discuss our recent CVPR 2024 Foundational FSOD competition and share insights from the community.
  arXiv  Detail & Related papers  (2023-12-22T07:42:00Z)
• Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models [75.9543301303586]
  Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Fine-tuning and ensembling are also commonly adopted to better fit the downstream tasks. However, we argue that prior work has overlooked the inherent biases in foundation models.
  arXiv  Detail & Related papers  (2023-10-12T08:01:11Z)
• Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain [54.67888148566323]
  We introduce three large-scale time series forecasting datasets from the cloud operations domain. We show it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size. Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method.
  arXiv  Detail & Related papers  (2023-10-08T08:09:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.