Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution

• URL: http://arxiv.org/abs/2608.03483v1
• Date: Tue, 04 Aug 2026 11:21:08 GMT
• Title: Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution
• Abstract summary: Bernoulli-Continuation Policy (BCP) is a lightweight, plug-and-play framework for adaptive horizon execution.<n>On RoboTwin 2.0 with LingBot-VLA as the base policy, BCP improves the average success rate by +11.08% on 13 low-success tasks.<n>On a real robot, BCP lifts success from 74% to 92% and from 44% to 84% on two manipulation tasks.
• Score: 38.8233224696416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing chunk-based Vision-Language-Action (VLA) models execute a fixed number of actions (i.e., execution horizon) before replanning, turning replanning into a task-agnostic periodic schedule that is independent of task progress. As a result, when no replanning boundary falls before a critical manipulation stage, it is executed from a stale chunk rather than a freshly replanned one. To address this limitation, we propose Bernoulli-Continuation Policy (BCP), a lightweight, plug-and-play framework for adaptive horizon execution that keeps the base VLA frozen. Given a fixed-length action chunk, its continuation head decomposes execution-horizon selection into a sequence of continue-or-replan decisions, which imposes an ordinal, prefix-sharing inductive bias over candidate horizons rather than treating them as independent classes. Since the optimal horizon for each chunk is not observable, we train this head with reinforcement learning from trajectory-level outcomes and introduce a Replanning-Efficiency Reward that jointly rewards task success and efficient VLA usage, discouraging the policy from collapsing to unnecessarily short horizons. On RoboTwin 2.0 with LingBot-VLA as the base policy, BCP improves the average success rate by +11.08% on 13 low-success tasks and from 89.88% to 93.94% (+4.06%) across all 50 tasks. Although trained only under the Clean setting, BCP generalizes to the Randomized setting, raising the average success rate by +4.06%. It also transfers to a different base policy $π_{0.5}$, achieving a better result on LIBERO (+1.7%) and, notably, on the harder LIBERO-PRO (+6.8%). On a real robot, BCP lifts success from 74% to 92% and from 44% to 84% on two manipulation tasks. Meanwhile, its negligible overhead, combined with higher success, makes BCP's overall runtime even lower than the fixed-horizon baselines.

Related papers

• Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents [50.85250898062218]
  Large language model (LLM) agents for long-horizon interactive tasks typically follow a ReAct-style protocol.<n>A natural alternative is to let the agent emit variable-length action chunks.<n>But naively training such policies with standard reinforcement learning fails.
  arXiv  Detail & Related papers  (2026-09-02T03:17:11Z)
• EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents [56.25120535601186]
  Vision-language-action (VLA) models map visual observations and language instructions directly to robot actions.<n>We propose EmbodiedSkills, a framework that treats each skill decision as an execution proposal.<n>A shared executable-skill interface connects high-level skill selection, bounded low-level VLA execution, and post-action verification.
  arXiv  Detail & Related papers  (2026-09-01T14:14:47Z)
• Foresight Residual RL for Long-Horizon Robot Manipulation with Vision-Language-Action Models [32.14222486099161]
  Vision-Language-Action (VLA) policies offer strong general-purpose manipulation priors, but often fail on tight-tolerance, contact-rich assembly.<n>We show this failure mode in residual reinforcement learning over a frozen VLA base policy.<n>We propose Residual RL, which optimize handoff quality by augmenting each subtask's sparse success reward with an offline-estimated foresight value.
  arXiv  Detail & Related papers  (2026-07-17T21:00:28Z)
• Dynamic Execution Horizon Prediction for Chunk-based Robot Policies [29.482987292744568]
  Action chunking has become a standard design in modern robot policies, where the policy predicts a sequence of actions and executes a fixed number of them instead of acting one step at a time.<n>In practice, the execution horizon is typically chosen through empirical tuning and is highly task-dependent.<n>We propose Dynamic Execution Horizon Prediction (DEHP), an effective method that trains a lightweight execution-horizon prediction branch using online reinforcement learning.
  arXiv  Detail & Related papers  (2026-06-09T19:58:31Z)
• PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking [37.9894332388921]
  PACE is a training-free test-time execution method that selects the execution horizon online from the predicted chunk itself.<n>Because PACE uses only the predicted action chunk, it is plug-and-play and requires no retraining or access to policy internals.<n>We validate PACE through large-scale evaluations in both simulation and real-robot settings.
  arXiv  Detail & Related papers  (2026-05-30T05:11:06Z)
• ESPO: Early-Stopping Proximal Policy Optimization [78.79610718910628]
  ESPO (Early-Stopping Proximal Policy Optimization) detects trajectory failure on-the-fly and terminates rollouts early.<n>On DeepSeek-R1-Distill-Qwen-7B trained for mathematical reasoning, ESPO surpasses PPO on AIME2024 (46.28% vs. 45.25%), AMC2023 (85.83% vs. 82.94%), and MATH-500 (87.42% vs. 85.43%)
  arXiv  Detail & Related papers  (2026-05-28T12:40:22Z)
• What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies [36.91260665881213]
  Vision-language--action (VLA) policies are trained to imitate actions; their loss never asks them to estimate reward, progress, or future success.<n>We recover Monte-Carlo outcome targets using lightweight linear probes on frozen features.<n>The gains are not universal and require additional inference compute, but the underlying finding is clean.
  arXiv  Detail & Related papers  (2026-05-27T14:23:29Z)
• RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models [90.39703013636868]
  RePO-VLA is a recovery-driven policy optimization framework.<n>It assigns distinct roles to success, recovery, and failure trajectories.<n>It raises adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.
  arXiv  Detail & Related papers  (2026-05-10T08:24:05Z)
• HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents [36.77027704958893]
  HiPER is a novel Hierarchical Plan-Execute RL framework that separates high-level planning from low-level execution.<n>HiPER achieves state-of-the-art performance on challenging interactive benchmarks, reaching 97.4% success on ALFWorld and 83.3% on WebShop with Qwen2.5-7B-Instruct.
  arXiv  Detail & Related papers  (2026-02-18T03:31:34Z)
• Acting in Delayed Environments with Non-Stationary Markov Policies [57.52103323209643]
  We introduce a framework for learning and planning in MDPs where the decision-maker commits actions that are executed with a delay of $m$ steps. We prove that with execution delay, deterministic Markov policies in the original state-space are sufficient for attaining maximal reward, but need to be non-stationary. We devise a non-stationary Q-learning style model-based algorithm that solves delayed execution tasks without resorting to state-augmentation.
  arXiv  Detail & Related papers  (2021-01-28T13:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.