LM-X: Explainable Action Modeling with Progress, Event, and Uncertainty Prediction for Generalist Robot Manipulation

• URL: http://arxiv.org/abs/2608.25757v2
• Date: Thu, 27 Aug 2026 12:24:23 GMT
• Title: LM-X: Explainable Action Modeling with Progress, Event, and Uncertainty Prediction for Generalist Robot Manipulation
• Abstract summary: Generalist vision--language--action (VLA) policies learn long-horizon behavior mainly through short-horizon action prediction.<n>We introduce LM-X, which organizes prediction across task, event, and motor scales without claiming anatomical correspondence.
• Score: 28.431160101245837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generalist vision--language--action (VLA) policies learn long-horizon behavior mainly through short-horizon action prediction and reveal little beyond sampled commands. This creates two coupled bottlenecks: a single action target must implicitly absorb task progress, intermediate intent, and local reliability, while these control states remain hidden during execution. Inspired by functional principles of biological sensorimotor control, we introduce LM-X , which organizes prediction across task, event, and motor scales without claiming anatomical correspondence. Three explicitly supervised signals are emitted online and directly condition action generation: return-to-go (RTG) measures visible task progress, event-to-go (ETG) identifies the next semantic transition, and heteroscedastic action flow estimates local reliability through propagated variance. Explanation is therefore intrinsic to control rather than generated post hoc. Before a costly 20-day pretraining run on 64 NVIDIA B200 GPUs, a controlled five-task pretraining gate verifies the design: the complete model improves success by 16.0 points over the action-only backbone and by 10.8 points over the strongest single-head variant. We then train LM-X on more than 20,000 hours of real-robot trajectories, including over 1,000 hours of failed policy rollouts. LM-X achieves 74.1\% across 50 randomized-hard RoboTwin2.0 tasks versus 55.4\% for GR00T N1.7, and 68.6\% versus 50.7\% across seven real-robot tasks. RTG tracks semantic progress and visible regression, while variance rises during hesitation and oscillatory control. These results show that explicit multi-timescale predictive state can strengthen control while exposing interpretable internal estimates.

Related papers

• GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation [63.853929983870955]
  We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all from scratch on manipulation data.<n>It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM)<n>The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action.
  arXiv  Detail & Related papers  (2026-09-04T17:16:48Z)
• Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models [120.24139942108405]
  Scaling robot data is crucial for building generalist Vision-Language-Action (VLA) models.<n>Under a fixed robot-data budget, continued pre-training must turn limited trajectories into transferable visual-action knowledge.<n>We propose VLAct, a VLA-oriented VLM backbone trained on broad, heterogeneous, multi-embodiment robot data.
  arXiv  Detail & Related papers  (2026-08-27T17:59:40Z)
• G0.5: One Autoregressive Stream for Robot Reasoning and Action [43.43377463558725]
  We introduce G0.5, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective.<n>Because reasoning and action share a single set of weights, the pretrained VLM's capabilities carry over to physical behavior.<n>G0.5 surpasses state-of-the-art models across 7 independent regimes.
  arXiv  Detail & Related papers  (2026-08-12T07:26:47Z)
• RynnValue: Scaling Robotic Value Foundation Models with Temporal Distance [84.26294415726723]
  General-purpose reward models are increasingly the bottleneck for scaling robot learning.<n>We introduce RynnValue, an open-source value foundation model for robotic manipulation.<n>RynnValue scales to over 7,000 hours and roughly 3M instruction-conditioned clips without preference or progress annotations.
  arXiv  Detail & Related papers  (2026-08-10T17:09:37Z)
• Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision [26.793499191581315]
  Cross-embodiment transfer in vision-language-action (VLA) models remains challenging.<n>We present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Reason (ECoT) supervision to align cross-embodiment representations.
  arXiv  Detail & Related papers  (2026-06-29T16:48:48Z)
• ActProbe: Action-Space Probe for Early Failure Detection of Generative Robot Policies [21.31011797815255]
  We introduce ActProbe, a pure action-space detector for generative robot policies.<n>It uses two compact signals available from a single forward pass: Temporal Consistency Error (TCE) between consecutive action chunks and Action Chunk Magnitude (ACM) of the current chunk.<n>It maps these signals to per-step failure probabilities with a task-conditioned LSTM-MLP architecture.
  arXiv  Detail & Related papers  (2026-06-07T08:18:11Z)
• Wall-OSS-0.5 Technical Report [13.983843529533113]
  We present Wall-OSS-0.5, an open-source 4B VLA built upon a 3B VLM backbone augmented with action-generation components.<n>The model is pretrained across more than 20 embodiments, processing over one million robot trajectories per epoch.<n>It achieves non-trivial zero-shot real-robot behavior, completing several tasks, including a held-out deformable manipulation task, at high task progress on a 17-task suite.
  arXiv  Detail & Related papers  (2026-05-29T06:04:03Z)
• RLDX-1 Technical Report [74.70437517338186]
  We introduce RLDX-1, a general-purpose robotic policy for dexterous manipulation built on the Multi-Stream Action Transformer (MSAT)<n>Through empirical evaluation, we show that RLDX-1 consistently outperforms recent frontier Vision-Language-Action models.
  arXiv  Detail & Related papers  (2026-05-05T01:40:15Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation [32.41847293364159]
  PRIMO R1 is a 7B framework that transforms video MLLMs into active "Critics"<n>We leverage outcome-based Reinforcement Learning to incentivize explicit Chain-of-Thought generation for progress estimation.<n>We establish state-of-the-art performance on RoboFail benchmark with 67.0% accuracy, surpassing closed-source models like OpenAI o1 by 6.2%.
  arXiv  Detail & Related papers  (2026-03-16T17:53:28Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.