DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands

• URL: http://arxiv.org/abs/2310.08809v2
• Date: Sun, 18 Aug 2024 12:22:38 GMT
• Title: DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands
• Abstract summary: We propose a Learning-based framework for Throwing-Catching tasks using dexterous hands. Our method achieves a 73% success rate across 45 scenarios (diverse hand poses and objects) In tasks where the object in hand faces sideways, an extremely unstable scenario due to the lack of support from the palm, our method still achieves a success rate of over 60%.
• Score: 14.712280514097912
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving human-like dexterous manipulation remains a crucial area of research in robotics. Current research focuses on improving the success rate of pick-and-place tasks. Compared with pick-and-place, throwing-catching behavior has the potential to increase the speed of transporting objects to their destination. However, dynamic dexterous manipulation poses a major challenge for stable control due to a large number of dynamic contacts. In this paper, we propose a Learning-based framework for Throwing-Catching tasks using dexterous hands (LTC). Our method, LTC, achieves a 73\% success rate across 45 scenarios (diverse hand poses and objects), and the learned policies demonstrate strong zero-shot transfer performance on unseen objects. Additionally, in tasks where the object in hand faces sideways, an extremely unstable scenario due to the lack of support from the palm, all baselines fail, while our method still achieves a success rate of over 60\%.

Related papers

• Learning In-Hand Object Reaching to General 6D Poses [20.489992320820324]
  In-hand manipulation allows multi-fingered dexterous hands to reconfigure grasped objects without releasing and regrasping them.<n>We formulate this capability as in-hand 6D object pose reaching: starting from an existing grasp, coordinated finger motions move the object to a palm-relative target pose.<n>We present POISE, a sim-to-real reinforcement learning framework for this task.
  arXiv  Detail & Related papers  (2026-09-12T07:11:37Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo [82.73355932907565]
  DexJoCo is a benchmark and toolkit for task-oriented dexterous manipulation.<n>We present 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning.<n>We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation.
  arXiv  Detail & Related papers  (2026-05-15T17:59:51Z)
• Concurrent Prehensile and Nonprehensile Manipulation: A Practical Approach to Multi-Stage Dexterous Tasks [23.52755440099898]
  We present DexMulti, a sample-efficient approach for real-world dexterous multi-task manipulation.<n>Rather than learning monolithic policies, our method retrieves demonstrated skills based on current object geometry.<n>Our approach achieves an average success rate of 66% on training objects with only 3-4 demonstrations per object, outperforming diffusion policy baselines by 2-3x.
  arXiv  Detail & Related papers  (2026-03-12T08:25:21Z)
• DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations [30.95930862562584]
  We propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects.<n>Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and experiments are conducted.<n>For grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories.<n>For the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%
  arXiv  Detail & Related papers  (2026-02-25T11:38:07Z)
• Cross-Embodiment Dexterous Hand Articulation Generation via Morphology-Aware Learning [82.63833405368159]
  Existing end-to-end methods require training on large-scale datasets for specific hands.<n>We propose an eigengrasp-based, end-to-end framework for cross-embodiment grasp generation.
  arXiv  Detail & Related papers  (2025-10-07T15:57:00Z)
• Learning Dexterous Object Handover [4.351636062759616]
  In this work, we demonstrate the use of Reinforcement Learning (RL) for dexterous object handover between two multi-finger hands.<n>Key to this task is the use of a novel reward function based on dual quaternions to minimize the rotation distance.<n>The results demonstrate that the trained policy successfully perform this task, achieving a total success rate of 94% in the best-case scenario after 100 experiments.
  arXiv  Detail & Related papers  (2025-06-20T08:22:46Z)
• DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation [48.68321200585559]
  We study the problem of functional discoterous manipulation policies to track object states from human hand-object demonstrations.<n>We propose a novel curriculum-based algorithm: the key idea is to use virtual object controllers with strength.<n>We release a simulation benchmark with a diverse set of tasks and dexterous hands, and show that DexMachina significantly outperforms baseline methods.
  arXiv  Detail & Related papers  (2025-05-30T17:50:23Z)
• Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning [9.305146484955296]
  We develop a learning-based controller for a mobile manipulator to move an unknown object to a desired position and yaw orientation through a sequence of pushing actions. The proposed controller for the robotic arm and the mobile base motion is trained using a constrained Reinforcement Learning (RL) formulation. The learned policy achieves a success rate of 91.35% in simulation and at least 80% on hardware in challenging scenarios.
  arXiv  Detail & Related papers  (2025-02-03T17:28:35Z)
• Local Policies Enable Zero-shot Long-horizon Manipulation [80.1161776000682]
  We introduce ManipGen, which leverages a new class of policies for sim2real transfer: local policies. ManipGen outperforms SOTA approaches such as SayCan, OpenVLA, LLMTrajGen and VoxPoser across 50 real-world manipulation tasks by 36%, 76%, 62% and 60% respectively.
  arXiv  Detail & Related papers  (2024-10-29T17:59:55Z)
• Single-Shot Learning of Stable Dynamical Systems for Long-Horizon Manipulation Tasks [48.54757719504994]
  This paper focuses on improving task success rates while reducing the amount of training data needed. Our approach introduces a novel method that segments long-horizon demonstrations into discrete steps defined by waypoints and subgoals. We validate our approach through both simulation and real-world experiments, demonstrating effective transfer from simulation to physical robotic platforms.
  arXiv  Detail & Related papers  (2024-10-01T19:49:56Z)
• Hand-Object Interaction Pretraining from Videos [77.92637809322231]
  We learn general robot manipulation priors from 3D hand-object interaction trajectories. We do so by sharing both the human hand and the manipulated object in 3D space and human motions to robot actions. We empirically demonstrate that finetuning this policy, with both reinforcement learning (RL) and behavior cloning (BC), enables sample-efficient adaptation to downstream tasks and simultaneously improves robustness and generalizability compared to prior approaches.
  arXiv  Detail & Related papers  (2024-09-12T17:59:07Z)
• GraspXL: Generating Grasping Motions for Diverse Objects at Scale [30.104108863264706]
  We unify the generation of hand-object grasping motions across multiple motion objectives in a policy learning framework GraspXL. Our policy trained with 58 objects can robustly synthesize diverse grasping motions for more than 500k unseen objects with a success rate of 82.2%. Our framework can be deployed to different dexterous hands and work with reconstructed or generated objects.
  arXiv  Detail & Related papers  (2024-03-28T17:57:27Z)
• Physically Plausible Full-Body Hand-Object Interaction Synthesis [32.83908152822006]
  We propose a physics-based method for synthesizing dexterous hand-object interactions in a full-body setting. Existing methods often focus on isolated segments of the interaction process and rely on data-driven techniques that may result in artifacts.
  arXiv  Detail & Related papers  (2023-09-14T17:55:18Z)
• HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation [29.01984677695523]
  We introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline.
  arXiv  Detail & Related papers  (2023-05-06T05:55:27Z)
• Interacting Hand-Object Pose Estimation via Dense Mutual Attention [97.26400229871888]
  3D hand-object pose estimation is the key to the success of many computer vision applications. We propose a novel dense mutual attention mechanism that is able to model fine-grained dependencies between the hand and the object. Our method is able to produce physically plausible poses with high quality and real-time inference speed.
  arXiv  Detail & Related papers  (2022-11-16T10:01:33Z)
• Reactive Human-to-Robot Handovers of Arbitrary Objects [57.845894608577495]
  We present a vision-based system that enables human-to-robot handovers of unknown objects. Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation. We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects.
  arXiv  Detail & Related papers  (2020-11-17T21:52:22Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.