PAVE: Predictive Alignment and Value-Guided Evolution for World-Action Policies

• URL: http://arxiv.org/abs/2608.30378v2
• Date: Wed, 02 Sep 2026 09:43:34 GMT
• Title: PAVE: Predictive Alignment and Value-Guided Evolution for World-Action Policies
• Abstract summary: Direct vision-language-action policies generate continuous robot actions efficiently, but standard behavior cloning leaves two complementary gaps.<n>We introduce method, a direct world-action policy that combines outcome-agnostic predictive learning with outcome-aware policy improvement.
• Score: 17.87859739145508
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Direct vision-language-action policies generate continuous robot actions efficiently, but standard behavior cloning leaves two complementary gaps: their representations are not explicitly required to describe how the scene evolves over multiple time scales, and deployment trajectories of unequal quality are often reused without separating useful dynamics from undesirable behavior. We introduce \method, a direct world-action policy that combines outcome-agnostic predictive learning with outcome-aware policy improvement. \method first retains a local fixed-offset JEPA objective and adds trajectory-relative multi-horizon transition alignment at 25%, 50%, 75%, and 100% of the remaining episode. These training-only targets require the current policy representation to preserve both local physical changes and longer-range task progress, without supplying explicit future tokens to the action head. \method then trains an independent distributional value critic on cumulative deployment trajectories, computes action-chunk-aligned $N$-step advantages, and converts them into positive, negative, or null text conditions for a flow-matching actor. Thus, every valid trajectory can teach what physically happened, while the actor is deployed only under the condition associated with relatively better actions. The multi-horizon predictor and critic are removed from online execution, preserving direct action generation from the current observation, language instruction, and proprioception. \redclaim{Across the three simulation benchmarks, \method achieves the strongest overall performance while preserving the direct actor's online execution path.}

Related papers

• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning [53.77164209208635]
  We introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and trajectory alignment.<n>We propose WorldSync, which strengthens action following along three axes: distributional coverage, representational grounding, and intervention-effect alignment.<n>Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement.
  arXiv  Detail & Related papers  (2026-08-25T17:59:49Z)
• Max-Q Selective Imitation for Human-in-the-Loop Online Robot Learning [1.96243636752331]
  Human-in-the-loop (HIL) online reinforcement learning for real robots must absorb human interventions quickly while continuing to improve beyond the human prior.<n>We present a training method for this setting based on two components.<n>emphMC Q-chunk critic regresses chunk-level action values onto Monte Carlo returns from a replay buffer.<n>emphmax-Q selective imitation updates the actor by imitating, at each state, the higher-$Q$ action between the current policy action and a buffer sample.
  arXiv  Detail & Related papers  (2026-08-15T07:13:59Z)
• PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulation [17.66354304197193]
  Policy improvement requires step-level credit signals to distinguish behaviors that advance the task from those that stall or regress.<n>We present PACE, a credit-assignment framework for post-training on long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-08-15T04:25:16Z)
• Potential-Guided Flow Matching for Vision-Language-Action Policy Improvement [27.695600755960736]
  We introduce ForesightFlow, a self-guided flow-matching policy.<n>It augments each generated action chunk with a learned success-potential trajectory.<n>It scores candidate actions, enabling best-of-$K$ inference without an external critic.
  arXiv  Detail & Related papers  (2026-06-03T14:49:35Z)
• Implicit Drifting Policy: One-Step Action Generation via Conditional Expert Geometry [46.0185525503119]
  Implicit Drifting Policy (IDP) is a one-step imitation learning framework for robot control.<n>IDP brings the training-time correction of Drifting into policy learning without explicit vector field estimation.
  arXiv  Detail & Related papers  (2026-05-31T08:39:57Z)
• Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation [91.20850436220267]
  We introduce the Referring-Aware Visuomotor Policy (ReV)<n>ReV incorporates sparse referring points provided by a human or a high-level reasoning planner.<n>It is trained only by applying targeted perturbations to expert demonstrations.
  arXiv  Detail & Related papers  (2026-04-07T07:41:11Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning [10.037416068775853]
  We introduce Guided Flow Policy, which couples a multi-step flow-matching policy with a distilled one-step actor.<n>The actor directs the flow policy through weighted behavior cloning to focus on cloning high-value actions from the dataset.<n>This mutual guidance enables GFP to achieve state-of-the-art performance across 144 state and pixel-based tasks.
  arXiv  Detail & Related papers  (2025-12-03T17:05:58Z)
• Dense Policy: Bidirectional Autoregressive Learning of Actions [51.60428100831717]
  This paper introduces a bidirectionally expanded learning approach, termed Dense Policy, to establish a new paradigm for autoregressive policies in action prediction.<n>It employs a lightweight encoder-only architecture to iteratively unfold the action sequence from an initial single frame into the target sequence in a coarse-to-fine manner.<n>Experiments validate that our dense policy has superior autoregressive learning capabilities and can surpass existing holistic generative policies.
  arXiv  Detail & Related papers  (2025-03-17T14:28:08Z)
• Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows [58.762959061522736]
  offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. We build upon recent works on learning policies in latent action spaces and use a special form of Normalizing Flows for constructing a generative model. We evaluate our method on various locomotion and navigation tasks, demonstrating that our approach outperforms recently proposed algorithms.
  arXiv  Detail & Related papers  (2022-11-20T21:57:10Z)
• Offline Reinforcement Learning with Implicit Q-Learning [85.62618088890787]
  Current offline reinforcement learning methods need to query the value of unseen actions during training to improve the policy. We propose an offline RL method that never needs to evaluate actions outside of the dataset. This method enables the learned policy to improve substantially over the best behavior in the data through generalization.
  arXiv  Detail & Related papers  (2021-10-12T17:05:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.