Rethinking Visual Embodiment Dependence in Visuomotor Policies

• URL: http://arxiv.org/abs/2609.16815v1
• Date: Tue, 15 Sep 2026 08:20:56 GMT
• Title: Rethinking Visual Embodiment Dependence in Visuomotor Policies
• Abstract summary: Visuomotor policies observe both the task scene and the acting embodiment, allowing embodiment-specific visual cues to influence action prediction.<n>We show, through cue-conflict interventions across representative policies, that visible robot configuration can become a shortcut to task progress.<n>We realize this through embodiment canonicalization in 3D point clouds, replacing the original embodiment with a canonical end-effector representation.
• Score: 51.032312792468126
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Visuomotor policies observe both the task scene and the acting embodiment, allowing embodiment-specific visual cues to influence action prediction. We study this phenomenon as visual embodiment dependence (VED) and show, through cue-conflict interventions across representative policies, that visible robot configuration can become a shortcut to task progress. Rather than eliminating VED, we argue that it should be structured around embodiment information that supports control and generalization. We realize this through embodiment canonicalization in 3D point clouds, replacing the original embodiment with a canonical end-effector representation (CER) that preserves control-relevant geometry while abstracting embodiment-specific morphology. Its editable form further enables configuration-decorrelation augmentation for unfamiliar robot configurations. Experiments show that embodiment canonicalization substantially improves human-to-robot policy transfer without robot demonstrations, while simply removing the embodiment is insufficient without preserving control-relevant geometry. We further find that CER itself can become a configuration shortcut when robot configuration becomes decoupled from task progress; configuration-decorrelation augmentation mitigates this failure mode and restores robust recovery without sacrificing performance on seen configurations. Together, these results show that robust visuomotor learning benefits from structuring, rather than removing, visual embodiment information. Project website: https://tonyfang.net/ved

Related papers

• Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models [28.23835152537988]
  We propose Artificial Foveated Perception (AFP), a lightweight, policy-agnostic module for robotic foundation models.<n>AFP predicts task-conditioned masks over relevant objects, the robot, and other action-critical regions.<n>We show AFP reduces fine-tuning time, suppresses overfitting, and improves generalization under environmental perturbations.
  arXiv  Detail & Related papers  (2026-07-12T08:42:36Z)
• GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation [54.305781492241785]
  We present a framework for learning visuomotor manipulation policies that generalize beyond the training distribution.<n>GHOST factorizes control into (i) a high-level policy that predicts the next sub-goal as a distribution over 3D end-effector poses from multi-view RGB-D observations.<n>Across a suite of tasks, this hierarchical factorization consistently improves performance and robustness compared to a flat Diffusion Policy.
  arXiv  Detail & Related papers  (2026-06-08T18:08:01Z)
• SCAR: Self-Supervised Continuous Action Representation Learning [36.917304453471864]
  We propose SCAR, a joint inverse-forward dynamics framework for learning unified action representations across embodiments from visual transitions.<n>Built on a pretrained generative backbone, SCAR uses an inverse dynamics model (IDM) to infer latent actions from latent observation pairs and a forward dynamics model (FDM) to predict future dynamics conditioned on them.<n>Experiments on the Procgen and Robotwin dataset show that the learned unified latent action representation serves as a stronger conditioning interface for world modeling than embodiment-specific raw actions.
  arXiv  Detail & Related papers  (2026-05-13T16:23:11Z)
• LIDEA: Human-to-Robot Imitation Learning via Implicit Feature Distillation and Explicit Geometry Alignment [48.809726040947226]
  We introduce LIDEA, an imitation learning framework in which policy learning benefits from human demonstrations.<n>In the 2D visual domain, LIDEA employs a dual-stage transitive distillation pipeline that aligns human and robot representations in a shared latent space.<n>In the 3D geometric domain, we propose an embodiment-agnostic alignment strategy that explicitly decouples embodiment from interaction geometry.<n>Results show that human data substitutes up to 80% of costly robot demonstrations, and the framework successfully transfers unseen patterns from human videos for out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-04-12T15:02:34Z)
• Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning [1.6641413441634143]
  We propose a new Verbal Reinforcement Learning framework for interpretable task-level planning in mobile robotic systems.<n>The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment.<n>We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty.
  arXiv  Detail & Related papers  (2026-03-23T16:28:36Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• \textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation [50.027425808733994]
  textscNaVIDA is a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution.<n>textscNaVIDA augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions.<n>Experiments show that textscNaVIDA achieves superior navigation performance compared to state-of-the-art methods with fewer parameters.
  arXiv  Detail & Related papers  (2026-01-26T06:16:17Z)
• RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
  Existing solutions require massive training data or lack generalizability to unknown rendering configurations. We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem. Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
  arXiv  Detail & Related papers  (2022-05-11T17:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.