Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models

• URL: http://arxiv.org/abs/2606.14375v1
• Date: Fri, 12 Jun 2026 12:06:41 GMT
• Title: Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models
• Abstract summary: Vision-language--action (VLA) models are powerful action generators for robot manipulation.<n>They are typically executed with fixed inference and replanning schedules.<n>We propose Elastic Queries Reinforcement Learning (EQRL) to make each VLA policy query elastic.
• Score: 29.586817761572373
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action (VLA) models are powerful action generators for robot manipulation, but they are typically executed with fixed inference and replanning schedules. This rigidity ignores the uneven difficulty of robot control: contact-rich or uncertain states may need more computation and fresher feedback, while easier states can often be handled with fewer inference steps and longer open-loop execution. We propose Elastic Queries Reinforcement Learning (EQRL), a framework that makes each VLA policy query elastic. A lightweight latent-schedule adaptor jointly selects the latent input, denoising budget, and action chunk length, without fine-tuning the underlying VLA model. To make scheduling difficulty-aware, EQRL trains a critic over the joint latent-schedule action and derives a state difficulty signal from critic ensemble disagreement. This signal guides compute toward difficult states, while a learned residual allows task-driven correction. We formulate variable chunk execution as query-level macro-action RL with chunk-dependent discounting and an amortized number-of-function-evaluations (NFE) budget. Across simulation and real-robot manipulation, EQRL reduces amortized inference cost while preserving or improving task success.

Related papers

• A Hierarchical Approach to Imitation Learning for Manipulation Tasks Requiring Time Varying Forces [14.372073538832803]
  We introduce the Diffusion Policy Augmented by Fast Trajectory Generation (DPA-FTG)<n>Compared to recent visual-tactile approaches that focus on positional correction, DPA-FTG decouples low-frequency planning from high-frequency force regulation.<n>We validate our approach on a bimanual battery disassembly task involving the separation of a compliant sheet.
  arXiv  Detail & Related papers  (2026-08-04T04:21:00Z)
• VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model [54.35791816657227]
  We introduce textbfVLA-ATTC', a framework that endows VLA models with adaptive test-time compute.<n>VLA-ATTC employs an uncertainty-based cognitive clutch'' to dynamically transition from reflexive execution to a TTC deliberation phase.<n>On the LIBERO-LONG benchmark, VLA-ATTC reduces the failure rate of the SOTA model PI0.5 by over 50%.
  arXiv  Detail & Related papers  (2026-05-02T02:13:11Z)
• Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recovery [62.75419724651416]
  We introduce textbfSentinel-VLA, a metacognitive VLA model equipped with an active sentinel'' module to monitor real-time execution status.<n>All training data is automatically generated and annotated through our designed pipeline.<n>Real-world experiments demonstrate that Sentinel-VLA boosts the task success rate by over 30% compared to the SOTA model, PI0.
  arXiv  Detail & Related papers  (2026-05-02T02:10:54Z)
• Beyond Short-Horizon: VQ-Memory for Robust Long-Horizon Manipulation in Non-Markovian Simulation Benchmarks [96.60530830276281]
  RuleSafe is a new articulated manipulation benchmark built upon a scalable LLM-aided simulation framework.<n>VQ-Memory is a compact and structured temporal representation that uses vector-quantized variational autoencoders.
  arXiv  Detail & Related papers  (2026-03-10T11:13:54Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference [7.690958366125321]
  This paper introduces informed routing, a new paradigm that proactively addresses these issues.<n>We propose the Lightweight Feature Forecaster (LFF), a small predictive module that estimates a unit's output before routing decisions are made.<n>Experiments on both language modeling and reasoning tasks show that informed routing achieves state-of-the-art efficiency-performance trade-offs.
  arXiv  Detail & Related papers  (2025-10-10T09:59:36Z)
• VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators [38.880852900641]
  Vision-Language-Action (VLA) models enable embodied decision-making but rely heavily on imitation learning.<n>We introduce VLA-RFT, a reinforcement fine-tuning framework that leverages a data-driven world model as a controllable simulator.<n>With fewer than 400 fine-tuning steps, VLA-RFT surpasses strong supervised baselines and achieves greater efficiency than simulator-based RL.
  arXiv  Detail & Related papers  (2025-10-01T01:33:10Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• Reinforcement Learning Based Temporal Logic Control with Soft Constraints Using Limit-deterministic Generalized Buchi Automata [0.0]
  We study the control synthesis of motion planning subject to uncertainties. The uncertainties are considered in robot motion and environment properties, giving rise to the probabilistic labeled Markov decision process (MDP)
  arXiv  Detail & Related papers  (2021-01-25T18:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.