Concurrent Prehensile and Nonprehensile Manipulation: A Practical Approach to Multi-Stage Dexterous Tasks

• URL: http://arxiv.org/abs/2603.11655v2
• Date: Fri, 13 Mar 2026 22:10:34 GMT
• Title: Concurrent Prehensile and Nonprehensile Manipulation: A Practical Approach to Multi-Stage Dexterous Tasks
• Abstract summary: We present DexMulti, a sample-efficient approach for real-world dexterous multi-task manipulation.<n>Rather than learning monolithic policies, our method retrieves demonstrated skills based on current object geometry.<n>Our approach achieves an average success rate of 66% on training objects with only 3-4 demonstrations per object, outperforming diffusion policy baselines by 2-3x.
• Score: 23.52755440099898
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dexterous hands enable concurrent prehensile and nonprehensile manipulation, such as holding one object while interacting with another, a capability essential for everyday tasks yet underexplored in robotics. Learning such long-horizon, contact-rich multi-stage behaviors is challenging because demonstrations are expensive to collect and end-to-end policies require substantial data to generalize across varied object geometries and placements. We present DexMulti, a sample-efficient approach for real-world dexterous multi-task manipulation that decomposes demonstrations into object-centric skills with well-defined temporal boundaries. Rather than learning monolithic policies, our method retrieves demonstrated skills based on current object geometry, aligns them to the observed object state using an uncertainty-aware estimator that tracks centroid and yaw, and executes them via a retrieve-align-execute paradigm. We evaluate on three multi-stage tasks requiring concurrent manipulation (Grasp + Pull, Grasp + Open, and Grasp + Grasp) across two dexterous hands (Allegro and LEAP) in over 1,000 real-world trials. Our approach achieves an average success rate of 66% on training objects with only 3-4 demonstrations per object, outperforming diffusion policy baselines by 2-3x while requiring far fewer demonstrations. Results demonstrate robust generalization to held-out objects and spatial variations up to +/-25 cm.

Related papers

• DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation [66.68529423691461]
  We present DexVerse, a large-scale and modular benchmark for dexterous manipulation.<n>We benchmark representative methods, including Diffusion Policy, DP3, OpenVLA, and $_0.5$, across 19 tasks.<n>Results reveal substantial challenges in task generalization and visuomotor robustness.
  arXiv  Detail & Related papers  (2026-07-09T17:50:47Z)
• DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo [82.73355932907565]
  DexJoCo is a benchmark and toolkit for task-oriented dexterous manipulation.<n>We present 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning.<n>We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation.
  arXiv  Detail & Related papers  (2026-05-15T17:59:51Z)
• Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warping [44.348686148716894]
  We introduce Tether, a method for autonomous functional play involving structured, task-directed interactions.<n>First, we design a novel open-loop policy that warps actions from a small set of source demonstrations.<n>Second, we deploy this policy for autonomous functional play in the real world via a continuous cycle of task selection, execution, evaluation, and improvement.
  arXiv  Detail & Related papers  (2026-03-03T18:59:07Z)
• DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations [30.95930862562584]
  We propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects.<n>Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and experiments are conducted.<n>For grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories.<n>For the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%
  arXiv  Detail & Related papers  (2026-02-25T11:38:07Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Learning from 10 Demos: Generalisable and Sample-Efficient Policy Learning with Oriented Affordance Frames [10.738838923944876]
  Existing methods require a substantial number of demonstrations to cover possible task variations.<n>We introduce oriented affordance frames, a structured representation for state and action spaces.<n>We show how this abstraction allows for compositional generalisation of independently trained sub-policies.<n>We validate our method across three real-world tasks, each requiring multi-step, multi-object interactions.
  arXiv  Detail & Related papers  (2024-10-15T23:57:35Z)
• The Art of Imitation: Learning Long-Horizon Manipulation Tasks from Few Demonstrations [13.747258771184372]
  There are several open challenges to applying TP-GMMs in the wild. We factorize the robot's end-effector velocity into its direction and magnitude. We then segment and sequence skills from complex demonstration trajectories. Our approach enables learning complex manipulation tasks from just five demonstrations.
  arXiv  Detail & Related papers  (2024-07-18T12:01:09Z)
• Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations [60.241144377865716]
  We consider the imitation of sub-optimal demonstrations, with both a small clean demonstration set and a large noisy set. We propose method by evaluating and imitating at the sub-demonstration level, encoding action primitives of varying quality into different skills.
  arXiv  Detail & Related papers  (2023-06-13T17:24:37Z)
• Eliciting Compatible Demonstrations for Multi-Human Imitation Learning [16.11830547863391]
  Imitation learning from human-provided demonstrations is a strong approach for learning policies for robot manipulation. Natural human behavior has a great deal of heterogeneity, with several optimal ways to demonstrate a task. This mismatch presents a problem for interactive imitation learning, where sequences of users improve on a policy by iteratively collecting new, possibly conflicting demonstrations. We show that we can both identify incompatible demonstrations via post-hoc filtering, and apply our compatibility measure to actively elicit compatible demonstrations from new users.
  arXiv  Detail & Related papers  (2022-10-14T19:37:55Z)
• Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning [108.08083976908195]
  We show that policies learned by existing reinforcement learning algorithms can in fact be generalist. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms single-object specialist policies.
  arXiv  Detail & Related papers  (2021-11-04T17:59:56Z)
• Learning to Shift Attention for Motion Generation [55.61994201686024]
  One challenge of motion generation using robot learning from demonstration techniques is that human demonstrations follow a distribution with multiple modes for one task query. Previous approaches fail to capture all modes or tend to average modes of the demonstrations and thus generate invalid trajectories. We propose a motion generation model with extrapolation ability to overcome this problem.
  arXiv  Detail & Related papers  (2021-02-24T09:07:52Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.