Mana: Dexterous Manipulation of Articulated Tools

• URL: http://arxiv.org/abs/2606.13677v1
• Date: Thu, 11 Jun 2026 17:59:49 GMT
• Title: Mana: Dexterous Manipulation of Articulated Tools
• Abstract summary: Mana is a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem.<n>Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasps into manipulation trajectories.<n>Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.
• Score: 64.16038120159688
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Articulated tool manipulation remains a major challenge in dexterous robotics due to the need to coordinate internal degrees of freedom and contact-rich interactions. While prior work has largely focused on rigid objects, articulated tool use remains underexplored because of its physical complexity and the difficulty of learning functional grasping and manipulation policies. We present Mana (Manipulation Animator), a general sim-to-real framework that reinterprets dexterous manipulation as an animation problem. Inspired by computer animation, Mana employs a coarse-to-fine pipeline that transforms procedurally-generated grasp keyframes into manipulation trajectories through motion planning and reinforcement learning. The data generation process is largely automatic, requiring only a few mouse clicks to specify functional affordances (<1 minute per tool). Across four articulated tools spanning different scales and joint types, Mana achieves zero-shot sim-to-real transfer for both grasping and in-hand manipulation, demonstrating a scalable approach to dexterous articulated tool use.

Related papers

• CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements [63.40365657068892]
  Transferring human hand demonstrations to robotic grippers has recently emerged as a cost-effective solution for robot learning.<n>Motivated by this gap, we adopt an implicit, data-driven approach guided by fine-grained hand-pose motions.<n>In both simulation and real-robot experiments, our framework enables stable and precise hand-to-gripper transfer of complex spatial manipulations.
  arXiv  Detail & Related papers  (2026-09-07T13:47:22Z)
• UniHM: Unified Dexterous Hand Manipulation with Vision Language Model [39.2419824041854]
  Planning physically feasible dexterous hand manipulation is a central challenge in robotic manipulation and Embodied AI.<n>We introduce UniHM, the first framework for unified dexterous hand manipulation guided by free-form language commands.
  arXiv  Detail & Related papers  (2026-02-28T16:37:11Z)
• Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos [56.510263910611684]
  We tackle prehensile manipulation, in which tasks involve grasping an object before performing various post-grasp motions.<n>Human videos offer strong signals for learning the post-grasp motions, but they are less useful for learning the prerequisite grasping behaviors.<n>We present Perceive-Simulate-Imitate (PSI), a framework for training a modular manipulation policy using human video motion data.
  arXiv  Detail & Related papers  (2026-02-13T18:59:10Z)
• DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videos [56.64773686434068]
  We propose DexImit, an automated framework that converts human manipulation videos into physically plausible robot data.<n>DexImit can generate large-scale robot data based on human videos, either from the Internet or video generation models.<n>It is capable of handling diverse manipulation tasks, including tool use, long-horizon tasks, and fine-grained manipulations.
  arXiv  Detail & Related papers  (2026-02-10T18:59:02Z)
• MimicFunc: Imitating Tool Manipulation from a Single Human Video via Functional Correspondence [18.953496415412335]
  Imitating tool manipulation from human videos offers an intuitive approach to teaching robots.<n>We propose MimicFunc, a framework that establishes functional correspondences with function frame.<n>MimicFunc effectively enables the robot to generalize the skill from a single RGB-D human video to manipulating novel tools.
  arXiv  Detail & Related papers  (2025-08-19T05:49:47Z)
• MAPLE: Encoding Dexterous Robotic Manipulation Priors Learned From Egocentric Videos [43.836197294180316]
  We present MAPLE, a novel method for dexterous robotic manipulation that exploits rich manipulation priors to enable efficient policy learning.<n>Specifically, we predict hand-object contact points and detailed hand poses at the moment of hand-object contact and use the learned features to train policies for downstream manipulation tasks.
  arXiv  Detail & Related papers  (2025-04-08T14:25:25Z)
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning [24.675197489823898]
  We introduce ManipTrans, a novel method for transferring human bimanual skills to dexterous robotic hands in simulation.<n>Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency.<n>We also create DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing.
  arXiv  Detail & Related papers  (2025-03-27T17:50:30Z)
• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids [56.892520712892804]
  We introduce a practical sim-to-real RL recipe that trains a humanoid robot to perform three dexterous manipulation tasks.<n>We demonstrate high success rates on unseen objects and robust, adaptive policy behaviors.
  arXiv  Detail & Related papers  (2025-02-27T18:59:52Z)
• Learning Granularity-Aware Affordances from Human-Object Interaction for Tool-Based Functional Dexterous Grasping [27.124273762587848]
  Affordance features of objects serve as a bridge in the functional interaction between agents and objects.<n>We propose a granularity-aware affordance feature extraction method for locating functional affordance areas.<n>We use highly activated coarse-grained affordance features in hand-object interaction regions to predict grasp gestures.<n>This forms GAAF-Dex, a complete framework that learns Granularity-Aware Affordances from human-object interaction.
  arXiv  Detail & Related papers  (2024-06-30T07:42:57Z)
• DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools [96.38972082580294]
  DiffSkill is a novel framework that uses a differentiable physics simulator for skill abstraction to solve deformable object manipulation tasks. In particular, we first obtain short-horizon skills using individual tools from a gradient-based simulator. We then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input.
  arXiv  Detail & Related papers  (2022-03-31T17:59:38Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.