Causally Debiased Latent Action Model for Embodied Action Conditioned World Models

• URL: http://arxiv.org/abs/2607.09185v1
• Date: Fri, 10 Jul 2026 08:20:27 GMT
• Title: Causally Debiased Latent Action Model for Embodied Action Conditioned World Models
• Abstract summary: Action-conditioned world models (ACWMs) aim to simulate future observations conditioned on embodied actions.<n>Existing LAMs are typically trained with reconstruction-only objectives.<n>We propose CD-LAM, a causally debiased framework for LAM-based ACWMs.
• Score: 47.281442841934876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action-conditioned world models (ACWMs) aim to simulate future observations conditioned on embodied actions, offering a promising foundation for robot planning, policy evaluation, and data augmentation. However, learning controllable ACWMs requires large-scale action-labeled data, which remains costly to collect in the real world. Latent action models (LAMs) mitigate this bottleneck by inferring latent actions from unlabeled videos, but existing LAMs are typically trained with reconstruction-only objectives and therefore entangle action-relevant dynamics with action-irrelevant visual factors such as backgrounds and untouched objects. In this work, we identify this action-irrelevant bias as a key obstacle to controllable ACWMs and introduce evaluation metrics to measure latent-action bias, action following, and robustness. We propose CD-LAM, a causally debiased framework for LAM-based ACWMs. CD-LAM introduces three efficient fine-tuning objectives: embodiment-centric reconstruction, action-centric contrastive learning, and latent space calibration, which together encourage embodiment-focused, action-aware, and calibrated non-collapsed latent action representations. Experiments on 2B and 14B ACWM backbones show that CD-LAM substantially improves latent-action controllability, downstream robot-action following, visual fidelity, and adaptation efficiency, requiring only 6k fine-tuning steps and more than 12$\times$ fewer robot-action adaptation updates than the baseline.

Related papers

• Reconstructing Is Not Acting: Action-Centric Latent Dynamics Modeling [21.941208569455657]
  Latent action models learn action representations from unlabeled videos by inferring latent actions from visual transitions and reconstructing future states.<n>We propose $textbfACT-LAM, a lightweight action-centric framework that strengthens both action extraction and action utilization.<n>Experiments on several robotic datasets and the VP$2$ benchmark demonstrate stronger latent action consistency, forward dynamics, and downstream visual planning performance.
  arXiv  Detail & Related papers  (2026-09-14T08:10:46Z)
• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• Overcoming Statistical Bias in Action-Controllable World Models [18.890337378174678]
  Action-conditioned world models aim to predict how visual environments evolve under an agent's actions.<n>Future frames are often highly predictable from visual inertia and recurring motion patterns alone.<n>We introduce CoCo, a Counterfactual Consistency framework to enhance action controllability.
  arXiv  Detail & Related papers  (2026-08-05T10:10:36Z)
• ABot-M0.5: Unified Mobility-and-Manipulation World Action Model [27.262271208923995]
  ABot-M0.5 is built on the insight that mobile manipulation requires alignment at three levels: temporal granularity, action space, and train-test consistency.<n> Experiments on challenging mobile and fine-grained manipulation benchmarks demonstrate that ABot-M0.5 achieves state-of-the-art performance in both long-horizon task success and finegrained control accuracy.
  arXiv  Detail & Related papers  (2026-07-01T09:21:20Z)
• Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding [25.77502065015808]
  Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions.<n>We propose Delta-JEPA, an end-to-end reconstruction-free world model that augments latent forward prediction with a Latent Difference Action Decoder (LDAD)<n>LDAD reconstructs the executed action from the latent displacement between consecutive observations.
  arXiv  Detail & Related papers  (2026-06-30T07:08:24Z)
• Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model [15.646017114869169]
  Action conditioning remains underexplored in action-conditioned world models.<n>We propose DexAC, which treats action conditioning as a structured process rather than global compression.<n>We show that combining the semantic branch with DexAC significantly improves FID, FVD, and PCK, demonstrating gains in visual-temporal realism and action-following consistency.
  arXiv  Detail & Related papers  (2026-06-25T17:36:35Z)
• RotVLA: Rotational Latent Action for Vision-Language-Action Model [54.22746299071677]
  We introduce RotVLA, a VLA framework built on a continuous rotational latent action representation.<n>Latent actions are modeled as elements of SO(n), providing continuity, compositionality, and structured geometry aligned with real-world action dynamics.<n>RotVLA consists of a VLM backbone and a flow-matching action head, pretrained on large-scale cross-embodiment robotic datasets and human videos.
  arXiv  Detail & Related papers  (2026-05-13T11:58:02Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• ResWM: Residual-Action World Model for Visual RL [0.06999740786886534]
  Learning predictive world models from raw visual observations is a central challenge in reinforcement learning (RL)<n>We introduce the Residual-Action World Model (ResWM), a new framework that reformulates the control variable from absolute actions to residual actions.<n>ResWM produces more stable and energy-efficient action trajectories, a property critical for robotic systems deployed in real-world environments.
  arXiv  Detail & Related papers  (2026-03-11T11:27:08Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• ActionSink: Toward Precise Robot Manipulation with Dynamic Integration of Action Flow [93.00917887667234]
  This paper introduces a novel robot manipulation framework, i.e., ActionSink, to pave the way toward precise action estimations.<n>As the name suggests, ActionSink reformulates the actions of robots as action-caused optical flows from videos, called "action flow"<n>Our framework outperformed prior SOTA on the LIBERO benchmark by a 7.9% success rate, and obtained nearly an 8% accuracy gain on the challenging long-horizon visual task LIBERO-Long.
  arXiv  Detail & Related papers  (2025-08-05T08:46:17Z)
• Object-Centric Latent Action Learning [70.3173534658611]
  We propose a novel object-centric latent action learning framework that centers on objects rather than pixels.<n>We leverage self-supervised object-centric pretraining to disentangle action-related and distracting dynamics.<n>Our results show that object-centric pretraining mitigates the negative effects of distractors by 50%.
  arXiv  Detail & Related papers  (2025-02-13T11:27:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.