HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation

• URL: http://arxiv.org/abs/2604.18791v1
• Date: Mon, 20 Apr 2026 19:57:35 GMT
• Title: HELM: Harness-Enhanced Long-horizon Memory for Vision-Language-Action Manipulation
• Abstract summary: Vision-Language-Action (VLA) models fail systematically on long-horizon manipulation tasks despite strong short-horizon performance.<n>We show that this failure is not resolved by extending context length alone in the current reactive execution setting.<n>We present HELM, a model-agnostic framework that addresses these deficiencies with three components.
• Score: 2.9231828959903474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models fail systematically on long-horizon manipulation tasks despite strong short-horizon performance. We show that this failure is not resolved by extending context length alone in the current reactive execution setting; instead, it stems from three recurring execution-loop deficiencies: the memory gap, the verification gap, and the recovery gap. We present HELM, a model-agnostic framework that addresses these deficiencies with three components: an Episodic Memory Module (EMM) that retrieves key task history via CLIP-indexed keyframes, a learned State Verifier (SV) that predicts action failure before execution from observation, action, subgoal, and memory-conditioned context, and a Harness Controller (HC) that performs rollback and replanning. The SV is the core learning contribution: it consistently outperforms rule-based feasibility checks and ensemble uncertainty baselines, and its effectiveness depends critically on access to episodic memory. On LIBERO-LONG, HELM improves task success rate by 23.1 percentage points over OpenVLA (58.4% to 81.5%), while extending the context window to H=32 yields only a 5.4-point gain and same-budget LoRA adaptation remains 12.2 points below HELM. HELM also improves long-horizon performance on CALVIN and substantially boosts recovery success under controlled perturbations. Ablations and mechanism analyses isolate the contribution of each component, and we release LIBERO-Recovery as a perturbation-injection protocol for evaluating failure recovery in long-horizon manipulation.

Related papers

• ActFovea: Runtime Safeguarding for VLA Policies via Spatiotemporal Visual-Action Consistency [23.6841289206823]
  Vision-language-action (VLA) policies achieve strong performance in robotic manipulation.<n>They are vulnerable to runtime disturbances that break the temporal alignment among visual observations, robot states, and executed actions.<n>We introduce ActFovea, a plug-and-play safeguarding framework that detects and mitigates such failures without retraining or modifying the underlying VLA policy.
  arXiv  Detail & Related papers  (2026-07-31T08:47:57Z)
• TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors [54.12833308568015]
  Vision-Language-Action (VLA) models are deployed through pipelines that end users cannot audit.<n>A small visual trigger redirects a long-horizon robot policy before any failure becomes observable.<n>We study this gap through two independently proposed VLA attacks from groups with distinct injection strategies.
  arXiv  Detail & Related papers  (2026-07-14T09:43:52Z)
• Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models [60.17417174639865]
  Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining.<n>The prevailing remedy, post-training, improves task-specific performance but narrows the generalist capability that makes pretraining valuable.<n>We propose SVA, a framework that equips frozen VLA policies with long-term consequence awareness.
  arXiv  Detail & Related papers  (2026-07-04T07:53:10Z)
• PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models [34.954808072081676]
  We introduce PHASER, an architecture-agnostic continual learning framework.<n>PhaseR employs a phase-centric capacity allocation to guarantee equal memory support for all sub-skills.<n>We also integrate Auto-PC, a lightweight pipeline combining unsupervised action-signal change-point detection with VLM-based semantic verification.
  arXiv  Detail & Related papers  (2026-06-02T13:04:15Z)
• Pre-VLA: Preemptive Runtime Verification for Reliable Vision-Language-Action and World-Model Rollouts [18.44683570310399]
  Low-quality actions may cause physical failures during execution or lead to misleading world-model rollouts with redundant rendering costs.<n>We propose Pre-VLA, a unified runtime verification architecture that performs preemptive action assessment validity before physical execution or world-model imagination.
  arXiv  Detail & Related papers  (2026-05-21T13:13:31Z)
• RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models [90.39703013636868]
  RePO-VLA is a recovery-driven policy optimization framework.<n>It assigns distinct roles to success, recovery, and failure trajectories.<n>It raises adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.
  arXiv  Detail & Related papers  (2026-05-10T08:24:05Z)
• Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflection [15.30405243180468]
  Long-horizon embodied manipulation tasks require persistent memory, adaptive task decomposition, and explicit recovery from execution failures.<n>Our framework explicitly separates high-level semantic reasoning from low-level motor execution.<n>A high-level planner, implemented as a VLM-based agentic module, maintains structured task memory.<n>A low-level executor, instantiated as a VLA-based visuomotor controller, carries out each sub-task.
  arXiv  Detail & Related papers  (2026-04-15T14:53:09Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Recursive Belief Vision Language Action Models [0.0]
  Long-horizon manipulation requires persistent, action-conditioned state representations.<n>Current vision-language models exhibit limited temporal and physical reasoning.<n>This paper introduces RB-VLA, a belief-centric architecture trained with self-supervised world-model objectives.
  arXiv  Detail & Related papers  (2026-02-24T08:02:16Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution [84.41751286055909]
  We develop a training-based KV cache eviction framework that learns to predict which KV pairs to evict during longtext generations.<n>We formulate cache eviction as a Markov Decision Process and apply the GRPO algorithm to mitigate the significant language modeling loss increase on low-entropy tokens.
  arXiv  Detail & Related papers  (2026-02-03T07:16:51Z)
• Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction [50.99402504483692]
  We propose a novel gating-based KV cache eviction method for frozen-weight language models.<n>Our approach integrates seamlessly into both the prefill and decoding stages.<n>Experiments show that our method maintains near-lossless performance while evicting up to 70% of the KV cache.
  arXiv  Detail & Related papers  (2026-01-25T03:07:54Z)
• CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding [60.06899554269808]
  CycleVLA is a system that equips Vision-Language-Action models (VLAs) with proactive self-correction.<n>CycleVLA achieves this by integrating a progress-aware VLA that flags critical subtask transition points.<n>Extensive experiments show that CycleVLA improves performance for both well-trained and under-trained VLAs.
  arXiv  Detail & Related papers  (2026-01-05T17:31:01Z)
• On GRPO Collapse in Search-R1: The Lazy Likelihood-Displacement Death Spiral [59.14787085809595]
  We identify Lazy Likelihood Displacement (LLD) as the core mechanism driving this failure.<n>LLD emerges early and triggers a self-reinforcing LLD Death Spiral, where declining likelihood leads to low-confidence responses.<n>We propose a lightweight likelihood-preserving regularization LLDS for GRPO that activates only when a trajectory's likelihood decreases.
  arXiv  Detail & Related papers  (2025-12-03T19:41:15Z)
• Leave No Observation Behind: Real-time Correction for VLA Action Chunks [36.13271200613596]
  Asynchronous Action Chunk Correction (A2C2) is a lightweight real-time chunk correction head that runs every control step.<n>We show that A2C2 is an effective plug-in mechanism for deploying high-capacity chunking policies in real-time control.
  arXiv  Detail & Related papers  (2025-09-27T10:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.