TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesis

• URL: http://arxiv.org/abs/2604.08468v1
• Date: Thu, 09 Apr 2026 17:03:49 GMT
• Title: TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesis
• Abstract summary: Test-Time Variational Synthesis (TTVS) is a novel framework that enables Large Reasoning Models to self-evolve.<n>TTVS transforms static test queries into a dynamic stream of diverse, semantically-equivalent variations.<n>We show TTVS yields superior performance across eight model architectures.
• Score: 21.127638319243932
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite significant advances in Large Reasoning Models (LRMs) driven by reinforcement learning with verifiable rewards (RLVR), this paradigm is fundamentally limited in specialized or novel domains where such supervision is prohibitively expensive or unavailable, posing a key challenge for test-time adaptation. While existing test-time methods offer a potential solution, they are constrained by learning from static query sets, risking overfitting to textual patterns. To address this gap, we introduce Test-Time Variational Synthesis (TTVS), a novel framework that enables LRMs to self-evolve by dynamically augmenting the training stream from unlabeled test queries. TTVS comprises two synergistic modules: (1) Online Variational Synthesis, which transforms static test queries into a dynamic stream of diverse, semantically-equivalent variations, enforcing the model to learn underlying problem logic rather than superficial patterns; (2) Test-time Hybrid Exploration, which balances accuracy-driven exploitation with consistency-driven exploration across synthetic variants. Extensive experiments show TTVS yields superior performance across eight model architectures. Notably, using only unlabeled test-time data, TTVS not only surpasses other test-time adaptation methods but also outperforms state-of-the-art supervised RL-based techniques trained on vast, high-quality labeled data.

Related papers

• Boosting ASR Robustness via Test-Time Reinforcement Learning with Audio-Text Semantic Rewards [8.109014000578766]
  We present ASR-TRA, a novel Testtime Reinforcement Adaptation framework inspired by causal intervention.<n>Our method achieves higher accuracy while maintaining lower latency than existing TTA baselines.<n>Our approach provides a practical and robust solution for deploying ASR systems in challenging real-world conditions.
  arXiv  Detail & Related papers  (2026-03-05T14:43:15Z)
• UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
  Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
  arXiv  Detail & Related papers  (2026-02-12T18:59:49Z)
• TTCS: Test-Time Curriculum Synthesis for Self-Evolving [47.826209735956716]
  Test-Time Training offers a promising way to improve the reasoning ability of large language models.<n>We propose TTCS, a co-evolving test-time training framework.<n>We show that TTCS consistently strengthens the reasoning ability on challenging mathematical benchmarks.
  arXiv  Detail & Related papers  (2026-01-30T06:38:02Z)
• Self-Improving LLM Agents at Test-Time [49.9396634315896]
  One paradigm of language model (LM) fine-tuning relies on creating large training datasets.<n>In practice, gathering large sets of data is inefficient, and training on them is prohibitively expensive.<n>We study two variants of this approach: Test-Time Self-Improvement (TT-SI) and Test-Time Distillation (TT-D)
  arXiv  Detail & Related papers  (2025-10-09T06:37:35Z)
• DualReward: A Dynamic Reinforcement Learning Framework for Cloze Tests Distractor Generation [0.4660328753262075]
  DualReward is a novel reinforcement learning framework for automatic distractor generation in cloze tests.<n>We evaluate our approach on both passage-level (CLOTH-F) and sentence-level (MCQ) cloze test datasets.
  arXiv  Detail & Related papers  (2025-07-16T03:39:36Z)
• MINGLE: Mixture of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging [29.58798660724693]
  Continual model merging integrates independently fine-tuned models sequentially without access to the original training data.<n>We propose MINGLE, a novel framework for Test-Time Continual Model Merging.<n> MINGLE achieves robust generalization, significantly reduces forgetting, and consistently surpasses previous state-of-the-art methods by 7-9% on average.
  arXiv  Detail & Related papers  (2025-05-17T07:24:22Z)
• Enhancing Test Time Adaptation with Few-shot Guidance [62.49199492255226]
  Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data.<n>Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data.<n>We develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA.
  arXiv  Detail & Related papers  (2024-09-02T15:50:48Z)
• Enhancing Plasticity for First Session Adaptation Continual Learning [20.62749699589017]
  We introduce Plasticity-Enhanced Test-Time Adaptation in Class-Incremental Learning (PLASTIC)<n>PLASTIC reinstates plasticity in CIL while preserving model stability.<n>It consistently outperforms both conventional and state-of-the-art PTM-based CIL approaches.
  arXiv  Detail & Related papers  (2023-10-17T13:06:39Z)
• AR-TTA: A Simple Method for Real-World Continual Test-Time Adaptation [1.4530711901349282]
  We propose to validate test-time adaptation methods using datasets for autonomous driving, namely CLAD-C and SHIFT. We observe that current test-time adaptation methods struggle to effectively handle varying degrees of domain shift. We enhance the well-established self-training framework by incorporating a small memory buffer to increase model stability.
  arXiv  Detail & Related papers  (2023-09-18T19:34:23Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [117.72709110877939]
  Test-time adaptation (TTA) has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions.<n>We categorize TTA into several distinct groups based on the form of test data, namely, test-time domain adaptation, test-time batch adaptation, and online test-time adaptation.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.