On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning

• URL: http://arxiv.org/abs/2601.06748v2
• Date: Tue, 13 Jan 2026 03:57:18 GMT
• Title: On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning
• Abstract summary: Vision-Language-Action models have emerged as a powerful paradigm for general-purpose robot learning.<n>Current methods remain unsuitable for challenging simulated- or physical-world deployments.<n>We introduce a Test-Time Reinforcement Learning for VLAs framework that enables on-the-fly policy adaptation during inference.
• Score: 61.38273866396522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action models have recently emerged as a powerful paradigm for general-purpose robot learning, enabling agents to map visual observations and natural-language instructions into executable robotic actions. Though popular, they are primarily trained via supervised fine-tuning or training-time reinforcement learning, requiring explicit fine-tuning phases, human interventions, or controlled data collection. Consequently, existing methods remain unsuitable for challenging simulated- or physical-world deployments, where robots must respond autonomously and flexibly to evolving environments. To address this limitation, we introduce a Test-Time Reinforcement Learning for VLAs (TT-VLA), a framework that enables on-the-fly policy adaptation during inference. TT-VLA formulates a dense reward mechanism that leverages step-by-step task-progress signals to refine action policies during test time while preserving the SFT/RL-trained priors, making it an effective supplement to current VLA models. Empirical results show that our approach enhances overall adaptability, stability, and task success in dynamic, previously unseen scenarios under simulated and real-world settings. We believe TT-VLA offers a principled step toward self-improving, deployment-ready VLAs.

Related papers

• TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models [49.463896453707065]
  Vision-Language-Action (VLA) models have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time.<n>Recent VLA models suggest that prompts can serve as an efficient interface for steering policy behavior, but existing prompt-based steering typically relies on external guidance.<n>This raises a natural question: can test-time training (TTT) for VLA be achieved by optimizing a prompt, so that the steering interface itself can be learned and adapted from interaction?<n>We address this question with TTT-VLA, a test-time training framework based on Latent Prompt Optimization (LPO)
  arXiv  Detail & Related papers  (2026-06-02T04:10:39Z)
• EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models [84.73890225707264]
  We present EXPO-FT, a system for stable, sample-efficient RL finetuning of pretrained VLA policies.<n>Our system achieves perfect task performance (30/30 successes) across all evaluated tasks within an average of 19.1 minutes of online robot data.<n>We release an open-source robustness with the aim of facilitating broader adoption of RL finetuning of VLA models in robotics.
  arXiv  Detail & Related papers  (2026-05-25T06:31:03Z)
• Jump-Start Reinforcement Learning with Vision-Language-Action Regularization [1.2599533416395767]
  Reinforcement learning (RL) enables high-frequency, closed-loop control for robotic manipulation.<n>Current limitations hinder their direct use in fast and precise manipulation.<n>We propose Vision-Language-Action Jump-Starting (VLAJS) to improve exploration and learning efficiency.
  arXiv  Detail & Related papers  (2026-04-15T11:17:54Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Models [21.133970394496327]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robotic control.<n>Current test-time scaling (TTS) methods require additional training, verifiers, and multiple forward passes, making them impractical for deployment.<n>We propose a simple inference strategy that jointly modulates visual perception and action based on'self-uncertainty'
  arXiv  Detail & Related papers  (2026-02-04T04:48:16Z)
• VLS: Steering Pretrained Robot Policies via Vision-Language Models [31.189909515514668]
  Vision-Language Steering (VLS) is a training-free framework for inference-time adaptation of frozen generative robot policies.<n>VLS treats adaptation as an inference-time control problem, steering the sampling process of a pretrained diffusion or flow-matching policy.
  arXiv  Detail & Related papers  (2026-02-03T19:50:16Z)
• EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models [57.75717492488268]
  Vision-Language-Action (VLA) models have advanced robotic manipulation by leveraging large language models.<n>Supervised Finetuning (SFT) requires hundreds of demonstrations per task, rigidly memorizing trajectories, and failing to adapt when deployment conditions deviate from training.<n>We introduce EVOLVE-VLA, a test-time training framework enabling VLAs to continuously adapt through environment interaction with minimal or zero task-specific demonstrations.
  arXiv  Detail & Related papers  (2025-12-16T18:26:38Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models [33.503927352666096]
  Vision-Language-Action (VLA) models fail to generalize reliably in out-of-distribution deployments.<n>We introduce RobustVLA, a lightweight online RL post-training method designed to explicitly enhance the resilience of VLA models.<n>Our results highlight the importance of robustness-aware RL post-training as a key step toward improving the principled reliability and robustness of VLA models.
  arXiv  Detail & Related papers  (2025-11-03T08:30:48Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.