PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models
Abstract Overview
PolicyTrim studies deployment efficiency in vision-language-action models from the policy side rather than only the inference-latency side. The paper argues that total execution time depends on both how much of each predicted action chunk can be executed reliably and how many physical steps the robot needs to finish a task. To address these two factors, it proposes a two-stage reinforcement-learning post-training framework: first extending the reliable executable horizon of action chunks, then reducing redundant physical steps. The method uses dynamic execution-horizon exploration, a step-saving reward, and group-anchored regularization, and is designed to improve efficiency without changing model architectures or requiring additional expert demonstrations.
Novelty
The distinctive contribution is framing intrinsic policy efficiency as a separate optimization target for VLA deployment, complementary to compute-centric acceleration methods. PolicyTrim introduces a two-stage RL post-training recipe that jointly targets reliable chunk extension and redundancy-aware step reduction, including a group-based horizon exploration mechanism and a stability regularizer to avoid collapsing onto fragile shortcuts.
Results
Across LIBERO, ManiSkill, Meta-World, and a real robot setup, PolicyTrim generally preserves or improves success rates while reducing execution steps and inference calls. The paper reports up to 3x higher action chunk utilization, a 51.4% reduction in physical execution steps, and up to a 5.83x end-to-end deployment speedup; on real-world tasks it achieves an average 1.86x speedup. The gains are shown across multiple VLA backbones, though the standard OpenVLA-OFT parallel-decoding setting primarily benefits from the step-reduction stage.
Key Points
- PolicyTrim targets policy efficiency directly by optimizing reliable action-chunk execution length and total physical step count, two factors that combine to determine inference-call frequency during deployment.
- The method combines dynamic horizon exploration for extending trustworthy chunk execution with a step-saving reward and group-anchored regularization for reducing redundant actions without destabilizing training.
- Experiments on multiple benchmarks and backbones, plus real-world deployment, show substantial speedups with maintained task success, and the approach is shown to be orthogonal and complementary to compute-centric methods like VLA-Cache.