Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control

• URL: http://arxiv.org/abs/2605.03363v1
• Date: Tue, 05 May 2026 04:49:38 GMT
• Title: Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control
• Abstract summary: We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
• Score: 50.28263951510334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose a hybrid hierarchical control framework for reactive dexterous grasping that explicitly decouples high-level spatial intent from low-level joint execution. We introduce a multi-agent reinforcement learning architecture, specialized into distinct arm and hand agents, that acts as a high-level planner by generating desired task-space velocity commands. These commands are then processed by a GPU-parallelized quadratic programming controller, which translates them into feasible joint velocities while strictly enforcing kinematic limits and collision avoidance. This structural isolation not only accelerates training convergence but also strictly enforces hardware safety. Furthermore, the architecture unlocks zero-shot steerability, allowing system operators to dynamically adjust safety margins and avoid dynamic obstacles without retraining the policy. We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline. Real-world hardware experiments on a 7-DoF arm equipped with a 20-DoF anthropomorphic hand demonstrate highly robust zero-shot transferability for dexterous grasping to a diverse set of unseen objects, highlighting the system's ability to reactively recover from unexpected physical disturbances in unstructured environments.

Related papers

• Graph-Based Safe Reinforcement Learning for Multi-Agent Systems with Time-Varying Topology [0.0]
  This paper presents a graph-based safe multi-agent reinforcement learning (MARL) framework for cooperative navigation with time-varying topology.<n>To address the critical challenge of ensuring safety in environments with sensing constraints, a safety-decoupled mechanism is introduced.<n>Building upon this safety foundation, a unified structural architecture is proposed, integrating a attention-based actor and a Graph Attention Network (GAT) centralized critic.
  arXiv  Detail & Related papers  (2026-09-08T14:30:06Z)
• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy [79.09714980186278]
  Building persistent embodied agents in unstructured environments requires unified orchestration of heterogeneous tools.<n>We present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces.<n>It achieves consistent improvements in end-to-end success across all complexity levels.
  arXiv  Detail & Related papers  (2026-06-25T16:36:35Z)
• Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking [78.30338456206984]
  We introduce a novel framework for training general-purpose 3D foothold-tracking policies.<n>By dynamically providing footstep support through a goal sampler, this method enables the learned policy to be agnostic to specific terrains.<n>Our new target representation effectively mitigates challenges arising in the real world, such as noisy and inaccurate pose estimation and foot contact estimation.
  arXiv  Detail & Related papers  (2026-06-06T16:44:56Z)
• HCLM: A Hierarchical Framework for Cooperative Loco-Manipulation with Dual Quadrupeds [24.47863400717969]
  We introduce HCLM, a hierarchical framework for general-purpose cooperative loco-manipulation with dual quadrupedal systems.<n>Our architecture systematically decouples high-level collaborative reasoning from low-level robust motion execution.<n>Results demonstrate reliable task execution, strict configurationism, and exceptional resilience against severe physical perturbations.
  arXiv  Detail & Related papers  (2026-05-17T07:23:03Z)
• Navigating the Clutter: Waypoint-Based Bi-Level Planning for Multi-Robot Systems [54.887871365121775]
  Multi-robot control in cluttered environments is a challenging problem that involves complex physical constraints.<n>We propose a hybrid multi-robot control framework that jointly optimize task and motion planning.<n>Our approach consistently improves task success over motion-agnostic and VLA-based baselines.
  arXiv  Detail & Related papers  (2026-04-22T22:58:47Z)
• UEREBot: Learning Safe Quadrupedal Locomotion under Unstructured Environments and High-Speed Dynamic Obstacles [16.149422197272937]
  UEREBot is a hierarchical framework that separates slow planning from instantaneous reflexive evasion.<n>We evaluate UEREBot in Isaac Lab simulation and deploy it on a Unitree Go2 quadruped equipped with onboard perception.
  arXiv  Detail & Related papers  (2026-02-07T05:09:48Z)
• Towards Space-Based Environmentally-Adaptive Grasping [45.88028371034407]
  We study limitations through the example of grasping in space environments.<n>We learn control policies directly in a learned latent manifold that fuses (grammarizes) multiple modalities into a structured representation for policy decision-making.
  arXiv  Detail & Related papers  (2026-01-29T08:31:03Z)
• Learning to Move in Rhythm: Task-Conditioned Motion Policies with Orbital Stability Guarantees [45.137864140049814]
  We introduce Orbitally Stable Motion Primitives (OSMPs) - a framework that combines a learned diffeomorphic encoder with a supercritical Hopf bifurcation in latent space.<n>We validate the proposed approach through extensive simulation and real-world experiments across a diverse range of robotic platforms.
  arXiv  Detail & Related papers  (2025-07-12T17:10:03Z)
• A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment [48.90852123901697]
  We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors.<n>Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
  arXiv  Detail & Related papers  (2025-04-21T14:25:23Z)
• PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion [1.123472110161393]
  A core strength of Model Predictive Control (MPC) for quadrupedal locomotion has been its ability to enforce constraints.<n>We propose a framework that integrates proprioceptive planning with Reinforcement Learning (RL)<n>During deployment, the Dreamer module solves an infinite-horizon MPC problem, adapting actions and velocity commands to respect the constraints.
  arXiv  Detail & Related papers  (2024-09-14T13:51:37Z)
• Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
  In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability. We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
  arXiv  Detail & Related papers  (2024-02-04T15:54:03Z)
• First Steps: Latent-Space Control with Semantic Constraints for Quadruped Locomotion [73.37945453998134]
  Traditional approaches to quadruped control employ simplified, hand-derived models. This significantly reduces the capability of the robot since its effective kinematic range is curtailed. In this work, these challenges are addressed by framing quadruped control as optimisation in a structured latent space. A deep generative model captures a statistical representation of feasible joint configurations, whilst complex dynamic and terminal constraints are expressed via high-level, semantic indicators. We validate the feasibility of locomotion trajectories optimised using our approach both in simulation and on a real-worldmal quadruped.
  arXiv  Detail & Related papers  (2020-07-03T07:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.