Demystifying Action Space Design for Robotic Manipulation Policies

• URL: http://arxiv.org/abs/2602.23408v1
• Date: Thu, 26 Feb 2026 13:32:41 GMT
• Title: Demystifying Action Space Design for Robotic Manipulation Policies
• Abstract summary: The action space plays a pivotal role in imitation-based robotic policy learning.<n>We show that the choice of action space does have significant and complex impacts on robotic policy learning.<n>Based on 13,000+ rollouts on a bimanual robot, we show that properly designing the policy to predict delta actions consistently improves performance.
• Score: 44.19773878335221
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The specification of the action space plays a pivotal role in imitation-based robotic manipulation policy learning, fundamentally shaping the optimization landscape of policy learning. While recent advances have focused heavily on scaling training data and model capacity, the choice of action space remains guided by ad-hoc heuristics or legacy designs, leading to an ambiguous understanding of robotic policy design philosophies. To address this ambiguity, we conducted a large-scale and systematic empirical study, confirming that the action space does have significant and complex impacts on robotic policy learning. We dissect the action design space along temporal and spatial axes, facilitating a structured analysis of how these choices govern both policy learnability and control stability. Based on 13,000+ real-world rollouts on a bimanual robot and evaluation on 500+ trained models over four scenarios, we examine the trade-offs between absolute vs. delta representations, and joint-space vs. task-space parameterizations. Our large-scale results suggest that properly designing the policy to predict delta actions consistently improves performance, while joint-space and task-space representations offer complementary strengths, favoring control stability and generalization, respectively.

Related papers

• Affordance-Based Hierarchical Reinforcement Learning for Quadruped Pedipulation [0.0]
  This study proposes a three-level hierarchical reinforcement learning (RL) framework that utilizes pose affordances to guide the navigation policy.<n>In addition, the pedipulation policy is guided by interaction-point affordances, enabling object-centric pose alignment of the quadruped robot.<n>We train the proposed framework in the IsaacSim ecosystem and evaluate it in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2026-06-05T17:56:42Z)
• Kinematics-Aware Diffusion Policy with Consistent 3D Observation and Action Space for Whole-Arm Robotic Manipulation [9.547155394196158]
  Whole-body control of robotic manipulators with awareness of full-arm kinematics is crucial for many manipulation scenarios.<n>The typical approach for whole-arm manipulation is to learn actions in the robot's joint space.<n>This letter proposes a kinematics-aware imitation learning framework with consistent task, observation, and action spaces.
  arXiv  Detail & Related papers  (2025-12-19T13:34:29Z)
• Spatial Policy: Guiding Visuomotor Robotic Manipulation with Spatial-Aware Modeling and Reasoning [43.35947581429867]
  Vision-centric hierarchical embodied models have demonstrated strong potential for long-horizon robotic control.<n>Existing methods lack spatial awareness capabilities, limiting their effectiveness in bridging visual plans to actionable control in complex environments.<n>We propose Spatial Policy (SP), a unified spatial-aware visuomotor robotic manipulation framework via explicit spatial modeling and reasoning.
  arXiv  Detail & Related papers  (2025-08-21T10:24:18Z)
• Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving [0.0]
  Reinforcement Learning (RL) offers a promising framework for autonomous driving.<n>Large and high-dimensional action spaces often used to support fine-grained control can impede training efficiency and increase exploration costs.<n>We introduce and evaluate two novel structured action space modification strategies for RL in autonomous driving.
  arXiv  Detail & Related papers  (2025-07-07T17:58:08Z)
• Situationally-Aware Dynamics Learning [57.698553219660376]
  We propose a novel framework for online learning of hidden state representations.<n>Our approach explicitly models the influence of unobserved parameters on both transition dynamics and reward structures.<n>Experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies.
  arXiv  Detail & Related papers  (2025-05-26T06:40:11Z)
• SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model [45.03115608632622]
  spatial understanding is the keypoint in robot manipulation.<n>We propose SpatialVLA to explore effective spatial representations for the robot foundation model.<n>We show the proposed Adaptive Action Grids offer a new and effective way to fine-tune the pre-trained SpatialVLA model for new simulation and real-world setups.
  arXiv  Detail & Related papers  (2025-01-27T07:34:33Z)
• On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer [6.1622717998840395]
  We study the choice of action space in robot manipulation learning and sim-to-real transfer. We train over 250 reinforcement learning(RL) agents in simulated reaching and pushing tasks. The choice of spaces spans combinations of common action space design characteristics.
  arXiv  Detail & Related papers  (2023-12-06T18:38:05Z)
• Co-learning Planning and Control Policies Constrained by Differentiable Logic Specifications [4.12484724941528]
  This paper presents a novel reinforcement learning approach to solving high-dimensional robot navigation tasks. We train high-quality policies with much fewer samples compared to existing reinforcement learning algorithms. Our approach also demonstrates capabilities for high-dimensional control and avoiding suboptimal policies via policy alignment.
  arXiv  Detail & Related papers  (2023-03-02T15:24:24Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees [63.31965375413414]
  We propose to solve multi-task problems through learning structured policies from human demonstrations. Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces. We derive an end-to-end learning objective function that is suitable for the multi-task problem.
  arXiv  Detail & Related papers  (2020-12-24T22:46:22Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.