TEMPO: Scaling Test-time Training for Large Reasoning Models

• URL: http://arxiv.org/abs/2604.19295v1
• Date: Tue, 21 Apr 2026 10:01:04 GMT
• Title: TEMPO: Scaling Test-time Training for Large Reasoning Models
• Abstract summary: Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time.<n>We propose TEMPO, a TTT framework that interleaves policy refinement on unlabeled questions with periodic critic recalibration on a labeled dataset.
• Score: 87.61789183311856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time training (TTT) adapts model parameters on unlabeled test instances during inference time, which continuously extends capabilities beyond the reach of offline training. Despite initial gains, existing TTT methods for LRMs plateau quickly and do not benefit from additional test-time compute. Without external calibration, the self-generated reward signal increasingly drifts as the policy model evolves, leading to both performance plateaus and diversity collapse. We propose TEMPO, a TTT framework that interleaves policy refinement on unlabeled questions with periodic critic recalibration on a labeled dataset. By formalizing this alternating procedure through the Expectation-Maximization (EM) algorithm, we reveal that prior methods can be interpreted as incomplete variants that omit the crucial recalibration step. Reintroducing this step tightens the evidence lower bound (ELBO) and enables sustained improvement. Across diverse model families (Qwen3 and OLMO3) and reasoning tasks, TEMPO improves OLMO3-7B on AIME 2024 from 33.0% to 51.1% and Qwen3-14B from 42.3% to 65.8%, while maintaining high diversity.

Related papers

• Selective Test-Time Compute Scaling for Click-Through Rate Prediction via Uncertainty-Triggered Feature Path Exploration [9.542597285477683]
  Scaling test-time compute has proven highly effective for language models, yet this opportunity remains largely unexplored for industrial Click-Through Rate (CTR) prediction.<n>We propose UTTSI (Uncertainty-Triggered Test-Time Selective Inference), a training-free model-agnostic framework that scales inference depth proportionally to per-instance uncertainty.
  arXiv  Detail & Related papers  (2026-05-24T10:29:10Z)
• Learning-Zone Energy: Online Data Selection for Efficient RL Post-Training [28.00739954235118]
  We propose a theoretically grounded, fully online data selection framework that concentrates on the model's active learning frontier.<n>A forward pruner with replay further reduces wall-clock time cost by skipping rollout generation for persistently solved prompts.<n>Our method retains only 40% of the training data per step yet matches or surpasses full-data baselines.
  arXiv  Detail & Related papers  (2026-05-16T14:01:12Z)
• Tool Verification for Test-Time Reinforcement Learning [70.09740926883818]
  Test-time reinforcement learning (TTRL) has emerged as a promising paradigm for self-evolving large reasoning models.<n>We present T3RL (Tool-Verification for Test-Time Reinforcement Learning), which introduces test-time tool verification into reward estimation.
  arXiv  Detail & Related papers  (2026-03-02T18:57:52Z)
• Prompt Augmentation Scales up GRPO Training on Mathematical Reasoning [19.22530791401551]
  We introduce prompt augmentation, a training strategy that instructs the model to generate reasoning traces under diverse templates and formats.<n>We show that, without a KL regularization term, prompt augmentation enables stable scaling of training duration under a fixed dataset.<n>A Qwen2.5-Math-1.5B model trained with prompt augmentation on the MATH Level 3-5 dataset achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-03T06:59:42Z)
• Certain Head, Uncertain Tail: Expert-Sample for Test-Time Scaling in Fine-Grained MoE [27.8012190589404]
  Fine-grained MoE features hundreds of well-trained experts per layer and multi-expert activation per token.<n>We empirically characterize fine-grained MoE routing and uncover an informative pattern: router scores exhibit a certain head of high-confidence experts followed by an uncertain tail of low-confidence candidates.<n>We propose Expert-Sample, a training-free method that preserves high-confidence selections while injecting controlledity into the uncertain tail, enabling diverse generation without destabilizing outputs.
  arXiv  Detail & Related papers  (2026-02-02T18:39:33Z)
• Self-Improving LLM Agents at Test-Time [49.9396634315896]
  One paradigm of language model (LM) fine-tuning relies on creating large training datasets.<n>In practice, gathering large sets of data is inefficient, and training on them is prohibitively expensive.<n>We study two variants of this approach: Test-Time Self-Improvement (TT-SI) and Test-Time Distillation (TT-D)
  arXiv  Detail & Related papers  (2025-10-09T06:37:35Z)
• Transition Models: Rethinking the Generative Learning Objective [68.16330673177207]
  We introduce a continuous-time dynamics equation that analytically defines state transitions across any finite time interval.<n>This leads to a novel generative paradigm, Transition Models (TiM), which adapt to arbitrary-step transitions.<n>TiM achieves state-of-the-art performance, surpassing leading models such as SD3.5 (8B parameters) and FLUX.1 (12B parameters) across all evaluated step counts.
  arXiv  Detail & Related papers  (2025-09-04T17:05:59Z)
• The Surprising Effectiveness of Test-Time Training for Few-Shot Learning [59.309477460893916]
  Language models (LMs) have shown impressive performance on tasks within their training distribution, but often struggle with structurally novel tasks.<n>We investigate the effectiveness of test-time training (TTT) as a mechanism for improving LMs' reasoning and few-shot learning capabilities.<n>Our findings highlight the limitations of in-context learning for novel tasks and demonstrate the potential of test-time training to enhance language model adaptability.
  arXiv  Detail & Related papers  (2024-11-11T18:59:45Z)
• RDumb: A simple approach that questions our progress in continual test-time adaptation [12.374649969346441]
  Test-Time Adaptation (TTA) allows to update pre-trained models to changing data distributions at deployment time. Recent work proposed and applied methods for continual adaptation over long timescales. We find that eventually all but one state-of-the-art methods collapse and perform worse than a non-adapting model.
  arXiv  Detail & Related papers  (2023-06-08T17:52:34Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [117.72709110877939]
  Test-time adaptation (TTA) has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions.<n>We categorize TTA into several distinct groups based on the form of test data, namely, test-time domain adaptation, test-time batch adaptation, and online test-time adaptation.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.