PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models

• URL: http://arxiv.org/abs/2606.22540v3
• Date: Thu, 25 Jun 2026 01:18:44 GMT
• Title: PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real-world deployment is often bottlenecked by execution efficiency.<n>We propose textbfPolicyTrim, a reinforcement learning-based post-training framework that extends the reliable action chunk length and reduces redundant physical steps.<n>Our framework delivers up to a 5.83$times$ end-to-end deployment speedup without compromising task success rates.
• Score: 50.232333672172395
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real-world deployment is often bottlenecked by execution efficiency. While existing efforts predominantly focus on compute-centric efficiency to reduce per-step inference latency, the intrinsic \textbf{policy efficiency} of these models remains largely unexplored. Policy efficiency is fundamentally affected by two factors, namely the effective executable length of predicted action chunks and the total physical steps required to complete a task. These two factors jointly determine the total number of forward inference calls during execution. We observe that current VLA policies struggle with planning unreliability and action redundancy, suffering from severe prediction degradation at the tail of action chunks and tending to generate unnecessarily redundant physical steps. To address this, we propose \textbf{PolicyTrim}, a reinforcement learning-based post-training framework that extends the reliable action chunk length and reduces redundant physical steps. For reliable chunk extension, we employ a dynamic exploration strategy that explicitly rewards the successful completion of longer executable lengths, progressively pushing the trustworthy prediction horizon to its empirical limit. For step efficiency, we design a redundancy-aware reward that directly favors successful task completions with fewer steps while penalizing unreproducible shortcuts, effectively eliminating redundant physical actions. Extensive experiments across three benchmarks and three VLA models demonstrate that PolicyTrim improves action chunk utilization by 3$\times$ and reduces physical execution steps by 51.4\%. Ultimately, our framework delivers up to a 5.83$\times$ end-to-end deployment speedup without compromising task success rates.

Related papers

• SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning [13.558040481824037]
  Agentic reinforcement learning (RL) has become a critical stage in the post-training of large language models.<n>We propose Single-rollout Autoregressive Policy Optimization (SAPO), a low-memory and compute-efficient framework in which the policy and value functions share a single autoregressive backbone.
  arXiv  Detail & Related papers  (2026-08-20T09:43:47Z)
• Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control? [78.81308700607651]
  Action chunking predicts and executes multiple actions instead of a single action.<n>We show that action chunking benefits from greater non-Markovian expressivity and reduced compounding error compared to Markovian policies.<n>We propose a policy class that amplifies the benefits of action chunking by explicitly instantiating an ensemble.
  arXiv  Detail & Related papers  (2026-08-03T17:32:45Z)
• Pre-VLA: Preemptive Runtime Verification for Reliable Vision-Language-Action and World-Model Rollouts [18.44683570310399]
  Low-quality actions may cause physical failures during execution or lead to misleading world-model rollouts with redundant rendering costs.<n>We propose Pre-VLA, a unified runtime verification architecture that performs preemptive action assessment validity before physical execution or world-model imagination.
  arXiv  Detail & Related papers  (2026-05-21T13:13:31Z)
• Latent Action Reparameterization for Efficient Agent Inference [56.42014061367112]
  We propose a framework that learns a compact latent action space in which each latent action corresponds to a multi-step semantic behavior.<n>Unlike hand-crafted macros or hierarchical controllers, latent actions are learned from agent trajectories and integrated directly into the model.
  arXiv  Detail & Related papers  (2026-05-18T16:07:44Z)
• Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States [4.1864205728857256]
  Vision-Language-Action (VLA) models leverage powerful perceptual priors from web-scale Vision-Language Model (VLM) pre-training.<n>To mitigate this, we propose Contrastive Conceptor Activation Steering (COAST)<n>COAST uses conceptors to identify success-critical subspaces for a target robotic task.
  arXiv  Detail & Related papers  (2026-05-16T20:28:21Z)
• Dynamic Execution Commitment of Vision-Language-Action Models [21.647844049489535]
  We introduce A3, an Adaptive Action Acceptance mechanism that reframes dynamic execution commitment as a self-speculative prefix verification problem.<n>A3 first computes a trajectory-wise consensus score of actions via group sampling, then selects a representative draft and prioritizes downstream verification.<n> Experiments across diverse VLA models and benchmarks demonstrate that A3 eliminates the need for manual horizon tuning while achieving a superior trade-off between execution and inference throughput.
  arXiv  Detail & Related papers  (2026-05-12T05:52:58Z)
• Decoupled Q-Chunking [63.864222078287575]
  We use chunked critics to estimate the value of short action sequences ("chunks") rather than individual actions, speeding up value backup.<n>Our key insight is to decouple the chunk length of the critic from that of the policy, allowing the policy to operate over shorter action chunks.<n>This design retains the benefits of multi-step value propagation while sidestepping both the open-loop sub-optimality and the difficulty of learning action chunking policies for long action chunks.
  arXiv  Detail & Related papers  (2025-12-11T18:52:51Z)
• DEPO: Dual-Efficiency Preference Optimization for LLM Agents [75.6723341304463]
  We propose DEPO, a dual-efficiency preference optimization method that jointly rewards succinct responses and fewer action steps.<n>Experiments on WebShop and BabyAI show that DEPO cuts token usage by up to 60.9% and steps by up to 26.9%, while achieving up to a 29.3% improvement in performance.
  arXiv  Detail & Related papers  (2025-11-19T12:38:43Z)
• ELHPlan: Efficient Long-Horizon Task Planning for Multi-Agent Collaboration [25.45699736192177]
  Large Language Models (LLMs) enable intelligent multi-robot collaboration but face fundamental trade-offs.<n>We propose ELHPlan, a novel framework that introduces Action Chains--sequences of actions explicitly bound to sub-goal intentions.
  arXiv  Detail & Related papers  (2025-09-29T03:15:56Z)
• Dynamic Speculative Agent Planning [57.630218933994534]
  Large language-model-based agents face critical deployment challenges due to prohibitive latency and inference costs.<n>We introduce Dynamic Speculative Planning (DSP), an online reinforcement learning framework that provides lossless acceleration with substantially reduced costs.<n>Experiments on two standard agent benchmarks demonstrate that DSP achieves comparable efficiency to the fastest acceleration method while reducing total cost by 30% and unnecessary cost up to 60%.
  arXiv  Detail & Related papers  (2025-09-02T03:34:36Z)
• EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models [21.42353501209045]
  Vision-Language-Action (VLA) models demonstrate transformative potential for embodied intelligence but are severely hampered by high computational and memory demands.<n>We introduce EfficientVLA, a structured and training-free inference acceleration framework.<n>We apply our method to a standard VLA model CogACT, yielding a 1.93X inference speedup and reduces FLOPs to 28.9%, with only a 0.6% success rate drop in the SIMPLER benchmark.
  arXiv  Detail & Related papers  (2025-06-11T18:34:57Z)
• Devil's Advocate: Anticipatory Reflection for LLM Agents [53.897557605550325]
  Our approach prompts LLM agents to decompose a given task into manageable subtasks. We implement a three-fold introspective intervention:. Anticipatory reflection on potential failures and alternative remedy before action execution. Post-action alignment with subtask objectives and backtracking with remedy to ensure utmost effort in plan execution.
  arXiv  Detail & Related papers  (2024-05-25T19:20:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.