PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

• URL: http://arxiv.org/abs/2606.17924v1
• Date: Tue, 16 Jun 2026 13:38:03 GMT
• Title: PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space
• Abstract summary: We propose PearlVLA, a framework that moves deliberation into the latent space of a vision-language model (VLM)<n>A future-guided RefineNet applies scheduled residual updates to progressively refine a coarse semantic draft into a fine-grained latent action plan.<n>PearlVLA achieves state-of-the-art performance among existing methods.
• Score: 3.4519796338615225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current Vision-Language-Action (VLA) models face a trade-off between efficient action generation and explicit deliberation. Directly decoding actions from vision-language backbone representations enables low-latency control, whereas explicit reasoning through textual chains, pixel-level subgoals, or action search can improve planning but incurs substantial latency and computational cost. We propose PearlVLA, a VLA framework that moves deliberation into the latent space of a vision-language model (VLM). PearlVLA separates VLM meta-query representations into a fixed visual grounding branch and an iterative latent plan branch. At each refinement round, a plan-conditioned world query probes a lightweight frozen latent world model for an action-free future observation latent, which is fed back to guide plan refinement. A future-guided RefineNet then applies scheduled residual updates to progressively refine a coarse semantic draft into a fine-grained latent action plan. The refined plan after K rounds is then decoded in parallel into an action chunk for low-latency execution. We further introduce Causal Refinement-Grouped Process-Reward RL to optimize the latent refinement process with rewards from longer-horizon imagined futures induced by latent plan edits. Empirical evaluations on the LIBERO benchmark demonstrate that PearlVLA achieves state-of-the-art performance among existing methods.

Related papers

• dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models [49.497309561043004]
  We propose textbfdVLA-RL, shifting the learning objective from the marginal action probability to the joint probability of the sampled generation path.<n>Our approach achieves a success rate of textbf99.7% on LIBERO.<n>It also establishes strong VLA-based results on RoboTwin 2.0 by delivering a textbf30.6% improvement over the SFT baseline.
  arXiv  Detail & Related papers  (2026-06-22T17:19:03Z)
• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models [22.797350217163977]
  Vision-Language-Action (VLA) models show promising ability in language-guided robotic tasks.<n>We argue that improving VLA reliability requires particular attention to planning actions.<n>We propose Planning-Aware Policy Optimization for VLA models (PAPO-VLA)
  arXiv  Detail & Related papers  (2026-05-19T09:22:49Z)
• LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models [13.30873593845724]
  LoopVLA is a recurrent Vision-Language-Action architecture that learns representation refinement, action prediction, and sufficiency estimation.<n>We show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times.
  arXiv  Detail & Related papers  (2026-05-11T03:51:22Z)
• Long-Horizon Manipulation via Trace-Conditioned VLA Planning [55.80061850746898]
  LoHo-Manip is a framework that scales short-horizon VLA execution to long-horizon instruction following via a dedicated task-management VLM.<n>LoHo-Manip predicts a progress-aware remaining plan that combines a subtask sequence with an explicit done + remaining split as lightweight language memory.<n>The executor VLA is adapted to condition on the rendered trace, thereby turning long-horizon decision-making into repeated local control.
  arXiv  Detail & Related papers  (2026-04-23T17:59:04Z)
• World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems [32.4918638412746]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for building embodied agents that ground perception and language into action.<n>We introduce World-Value-Action (WAV) model, a unified framework that enables implicit planning in VLA systems.
  arXiv  Detail & Related papers  (2026-04-16T07:46:05Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• SPIRAL: A Closed-Loop Framework for Self-Improving Action World Models via Reflective Planning Agents [135.00390535239129]
  We introduce SPIRAL, a self-improving planning and iterative reflective action world modeling framework.<n> SPIRAL formulates ActWM as a closed-loop think-act-reflect process, where generation proceeds step by step under explicit planning and feedback.<n> Experiments across multiple TI2V backbones demonstrate consistent gains on ActWM-Bench and mainstream video generation benchmarks.
  arXiv  Detail & Related papers  (2026-03-09T14:00:36Z)
• LaViPlan : Language-Guided Visual Path Planning with RLVR [0.0]
  Vision-Language Models (VLMs) have shown promise in handling such scenarios by providing high-level scene understanding and user-aligned decisions.<n>ExistingVLMs often exhibit a misalignment between their language-based reasoning and the low-level trajectories required for action-level planning.<n>We propose LaViPlan, a framework that leverages Reinforcement Learning with Verifiable Rewards (RLVR) to fine-tune VLMs using planning-oriented metrics.
  arXiv  Detail & Related papers  (2025-07-17T08:58:24Z)
• From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems [59.40480894948944]
  Large language model (LLM) empowered agents are able to solve decision-making problems in the physical world. Under this model, the LLM Planner navigates a partially observable Markov decision process (POMDP) by iteratively generating language-based subgoals via prompting. We prove that the pretrained LLM Planner effectively performs Bayesian aggregated imitation learning (BAIL) through in-context learning.
  arXiv  Detail & Related papers  (2024-05-30T09:42:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.