Efficient Test-Time Scaling for LLM-based Time Series Forecasting

• URL: http://arxiv.org/abs/2608.08675v1
• Date: Sun, 09 Aug 2026 12:45:08 GMT
• Title: Efficient Test-Time Scaling for LLM-based Time Series Forecasting
• Abstract summary: Long-term time series forecasting benefits from preserving global structure such as trends and seasonality.<n>Recent LLM-based forecasters often improve accuracy through test-time scaling.<n>But these methods are computationally expensive and prone to global-shape mismatch as the prediction horizon extends.<n>We propose a lightweight Transformer tailored to long-term shape modeling to predict a coarse representation of future dynamics.
• Score: 21.66751156586705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-term time series forecasting benefits from preserving global structure such as trends and seasonality. Recent LLM-based forecasters often improve accuracy through test-time scaling (e.g., iterative refinement), but these methods are computationally expensive and increasingly prone to global-shape mismatch as the prediction horizon extends. We propose SCALER, a coarse-to-fine forecasting framework that first employs a lightweight Transformer tailored to long-term shape modeling to predict a coarse representation of future dynamics. This predicted shape then serves as a compact guide for an LLM to perform test-time scaling via iterative coarse-to-fine residual token refinement, while processing substantially fewer tokens at each step. By guiding refinement with an explicit future-shape prediction, SCALER reduces reliance on long description prompts, and its fixed-step refinement avoids costly reward-model-based selection, further lowering computational overhead. Experimental results demonstrate that SCALER outperforms strong forecasting baselines in long-term, short-term and zero-shot forecasting while significantly reducing the inference cost associated with scaled LLM for time series forecasting. Code: https://github.com/xuanmay2701/SCALER.

Related papers

• PRICE: A Systematic Study of LLM Adaptation Choices for Bitcoin Price Forecasting [0.40105987447353786]
  This study introduces PRICE, a structured approach for adapting Large Language Models to short-term Bitcoin price forecasting.<n>Built on a 4-bit quantized LLaMA-3 8B model, PRICE investigates how fine-tuning, numerical representation, prompting, inference, and decoding jointly influence performance.<n>Price achieves the lowest forecasting errors on both validation and test sets while maintaining robust performance across evaluation periods.
  arXiv  Detail & Related papers  (2026-09-04T15:01:38Z)
• CastFlow: Learning Role-Specialized Agentic Workflows for Time Series Forecasting [74.29761960394848]
  We propose CastFlow, a dynamic agentic forecasting framework for large language models (LLMs)<n>CastFlow organizes the forecasting process into planning, action, forecasting, and reflection, establishing an agentic workflow.<n>Third, CastFlow adopts a role-specialized design that combines general-purpose reasoning with specialized numerical forecasting.
  arXiv  Detail & Related papers  (2026-04-30T13:24:42Z)
• Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling [51.78972657142583]
  We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K.<n>To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions.
  arXiv  Detail & Related papers  (2026-03-05T04:13:57Z)
• Predicting LLM Output Length via Entropy-Guided Representations [13.351384070796747]
  We introduce a lightweight framework that reuses the main model's internal hidden states for efficient length prediction.<n>Our framework features two core components: 1) Entropy-Guided Token Pooling (EGTP), which uses on-the-fly activations and token entropy for highly accurate static prediction.
  arXiv  Detail & Related papers  (2026-02-12T10:49:04Z)
• An Optimization Method for Autoregressive Time Series Forecasting [4.467218412104476]
  Current time-series forecasting models are primarily based on transformer-style neural networks.<n>We propose a novel training method for time-series forecasting that enforces two key properties.<n>Our method achieves an MSE reduction of more than 10% compared to iTransformer.
  arXiv  Detail & Related papers  (2026-02-02T16:28:00Z)
• TARFVAE: Efficient One-Step Generative Time Series Forecasting via TARFLOW based VAE [3.7770105485163206]
  This paper presents TARFVAE, a novel generative framework that combines the Transformer-based autoregressive flow (TARFLOW) and variational autoencoder (VAE) for efficient one-step generative time series forecasting.<n>With simple modules, TARFVAE achieves superior performance over state-of-the-art deterministic and generative models across different forecast horizons on datasets.
  arXiv  Detail & Related papers  (2025-11-28T03:19:14Z)
• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts [55.231201692232894]
  $textttSPECS$ is a latency-aware test-time scaling method inspired by speculative decoding.<n>Our results show that $textttSPECS$matches or surpasses beam search accuracy while reducing latency by up to $sim$19.1%.
  arXiv  Detail & Related papers  (2025-06-15T05:50:05Z)
• TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop [79.5773512667468]
  TimeXL is a multi-modal prediction framework that integrates a prototype-based time series encoder with three collaborating Large Language Models.<n>A reflection LLM compares the predicted values against the ground truth, identifying textual inconsistencies or noise.<n>This closed-loop workflow-prediction, critique (reflect), and refinement-continuously boosts the framework's performance and interpretability.
  arXiv  Detail & Related papers  (2025-03-02T20:40:53Z)
• Sundial: A Family of Highly Capable Time Series Foundation Models [47.27032162475962]
  We introduce Sundial, a family of native, flexible, and scalable time series foundation models.<n>Our models are pre-trained without specifying any prior distribution and can generate multiple probable predictions.<n>Sundial achieves state-of-the-art results on both point and probabilistic forecasting benchmarks with a just-in-time inference speed.
  arXiv  Detail & Related papers  (2025-02-02T14:52:50Z)
• The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws [51.608402959163925]
  We present the first systematic exploration of optimal sparse pre-training configurations for large language models.<n>We find that initiating pruning at 25% of total training compute and concluding at 75% achieves near-optimal final evaluation loss.<n>We propose a new scaling law that modifies the Chinchilla scaling law to use the average parameter count over pre-training.
  arXiv  Detail & Related papers  (2025-01-21T20:23:22Z)
• Loss Shaping Constraints for Long-Term Time Series Forecasting [79.3533114027664]
  We present a Constrained Learning approach for long-term time series forecasting that respects a user-defined upper bound on the loss at each time-step. We propose a practical Primal-Dual algorithm to tackle it, and aims to demonstrate that it exhibits competitive average performance in time series benchmarks, while shaping the errors across the predicted window.
  arXiv  Detail & Related papers  (2024-02-14T18:20:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.