PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2605.19580v1
• Date: Tue, 19 May 2026 09:22:49 GMT
• Title: PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
• Abstract summary: Vision-Language-Action (VLA) models show promising ability in language-guided robotic tasks.<n>We argue that improving VLA reliability requires particular attention to planning actions.<n>We propose Planning-Aware Policy Optimization for VLA models (PAPO-VLA)
• Score: 22.797350217163977
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models show promising ability in language-guided robotic tasks. However, making VLA policies reliable remains challenging, because a manipulation task is completed through closed-loop interaction, where each action affects subsequent execution. To analyze this problem, we revisit VLA policy during execution and argue that a VLA policy acts both as a planner, which makes task-oriented decisions that change the direction of execution, and as an executor, which realizes these decisions through dense continuous actions. This view suggests that improving VLA reliability requires particular attention to planning actions. Existing optimization methods can imitate actions or improve complete trajectories, but they usually do not explicitly identify planning actions or measure their importance for task success. To address this issue, we propose Planning-Aware Policy Optimization for VLA models (PAPO-VLA). PAPO-VLA first identifies planning actions by jointly considering action variation and trajectory outcome, then estimates their importance through causal sufficiency and causal necessity, and finally incorporates this importance into GRPO advantage estimation. In this way, more important planning actions receive stronger optimization emphasis, while the whole trajectory is still optimized by trajectory-level feedback. Experiments on multiple benchmarks demonstrate the effectiveness of PAPO-VLA.

Related papers

• PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space [3.4519796338615225]
  We propose PearlVLA, a framework that moves deliberation into the latent space of a vision-language model (VLM)<n>A future-guided RefineNet applies scheduled residual updates to progressively refine a coarse semantic draft into a fine-grained latent action plan.<n>PearlVLA achieves state-of-the-art performance among existing methods.
  arXiv  Detail & Related papers  (2026-06-16T13:38:03Z)
• TTT-VLA: Test-Time Latent Prompt Optimization for Vision-Language-Action Models [49.463896453707065]
  Vision-Language-Action (VLA) models have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time.<n>Recent VLA models suggest that prompts can serve as an efficient interface for steering policy behavior, but existing prompt-based steering typically relies on external guidance.<n>This raises a natural question: can test-time training (TTT) for VLA be achieved by optimizing a prompt, so that the steering interface itself can be learned and adapted from interaction?<n>We address this question with TTT-VLA, a test-time training framework based on Latent Prompt Optimization (LPO)
  arXiv  Detail & Related papers  (2026-06-02T04:10:39Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models [13.30873593845724]
  LoopVLA is a recurrent Vision-Language-Action architecture that learns representation refinement, action prediction, and sufficiency estimation.<n>We show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times.
  arXiv  Detail & Related papers  (2026-05-11T03:51:22Z)
• Anticipation-VLA: Solving Long-Horizon Embodied Tasks via Anticipation-based Subgoal Generation [23.021987362787225]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence.<n>We propose Anticipation-VLA, a hierarchical VLA model that leverages the anticipation model to generate actionable subgoals.<n>We implement Anticipation-VLA with finetuning a Unified Multimodal Model (UMM) for high-level subgoal generation and a goal-conditioned VLA policy for low-level action execution.
  arXiv  Detail & Related papers  (2026-05-03T08:11:35Z)
• AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation [21.23747444669735]
  We propose Action-Context-aware Adaptive Computation for VLA models (AC2-VLA)<n>AC2-VLA adaptively performs cognition reuse across timesteps, token pruning, and selective execution of model components within a unified mechanism.<n>Experiments on robotic manipulation benchmarks show that AC2-VLA achieves up to a 1.79times speedup while reducing FLOPs to 29.4% of the dense baseline.
  arXiv  Detail & Related papers  (2026-01-27T14:10:39Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search [7.9342097024286815]
  We present Vision-Language-Action Planning & Search (VLAPS)<n>It embeds model-based search into the inference procedure of pre-trained VLA policies.<n>VLAPS significantly outperforms VLA-only baselines on language-specified tasks.
  arXiv  Detail & Related papers  (2025-08-17T02:59:42Z)
• LaViPlan : Language-Guided Visual Path Planning with RLVR [0.0]
  Vision-Language Models (VLMs) have shown promise in handling such scenarios by providing high-level scene understanding and user-aligned decisions.<n>ExistingVLMs often exhibit a misalignment between their language-based reasoning and the low-level trajectories required for action-level planning.<n>We propose LaViPlan, a framework that leverages Reinforcement Learning with Verifiable Rewards (RLVR) to fine-tune VLMs using planning-oriented metrics.
  arXiv  Detail & Related papers  (2025-07-17T08:58:24Z)
• PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization [58.465778756331574]
  We propose a pseudocode-style Planning Guided Preference Optimization method called PGPO for effective agent learning.<n>With two planning-oriented rewards, PGPO further enhances LLM agents' ability to generate high-quality P-code Plans.<n>Experiments show that PGPO achieves superior performance on representative agent benchmarks and outperforms the current leading baselines.
  arXiv  Detail & Related papers  (2025-06-02T09:35:07Z)
• Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videos [48.15438373870542]
  VidAssist is an integrated framework designed for zero/few-shot goal-oriented planning in instructional videos. It employs a breadth-first search algorithm for optimal plan generation. Experiments demonstrate that VidAssist offers a unified framework for different goal-oriented planning setups.
  arXiv  Detail & Related papers  (2024-09-30T17:57:28Z)
• Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
  This work lays the foundations for improving planning capabilities of large language models (LLMs) We construct a comprehensive benchmark suite encompassing both classical planning benchmarks and natural language scenarios. We investigate the use of many-shot in-context learning to enhance LLM planning, exploring the relationship between increased context length and improved planning performance.
  arXiv  Detail & Related papers  (2024-06-18T22:57:06Z)
• Theoretically Guaranteed Policy Improvement Distilled from Model-Based Planning [64.10794426777493]
  Model-based reinforcement learning (RL) has demonstrated remarkable successes on a range of continuous control tasks. Recent practices tend to distill optimized action sequences into an RL policy during the training phase. We develop an approach to distill from model-based planning to the policy.
  arXiv  Detail & Related papers  (2023-07-24T16:52:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.