Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping

• URL: http://arxiv.org/abs/2503.00466v1
• Date: Sat, 01 Mar 2025 12:35:05 GMT
• Title: Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping
• Authors: Giuseppe Stracquadanio, Federico Vasile, Elisa Maiettini, Nicolò Boccardo, Lorenzo Natale,
• Abstract summary: We present a novel eye-in-hand prosthetic grasping system that follows shared-autonomy principles.<n>Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand.<n>We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness.
• Score: 4.476245767508223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One of the most important research challenges in upper-limb prosthetics is enhancing the user-prosthesis communication to closely resemble the experience of a natural limb. As prosthetic devices become more complex, users often struggle to control the additional degrees of freedom. In this context, leveraging shared-autonomy principles can significantly improve the usability of these systems. In this paper, we present a novel eye-in-hand prosthetic grasping system that follows these principles. Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand. First, it reconstructs the 3D geometry of the target object without the need of a depth camera. Then, it tracks the hand motion during the approach-to-grasp action and finally selects a candidate grasp configuration according to user's intentions. We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness. We compare it with a multi-DoF prosthetic control baseline and find that our method enables faster grasps, while simplifying the user experience. Code and demo videos are available online at https://hsp-iit.github.io/byogg/.

Related papers

• Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework [5.428117915362002]
  Most control techniques for prosthetic grasping focus on dexterous fingers control, but overlook the wrist motion.<n>This forces the user to perform compensatory movements with the elbow, shoulder and hip to adapt the wrist for grasping.<n>We propose a computer vision-based system that leverages the collaboration between the user and an automatic system in a shared autonomy framework.<n>Our pipeline allows to seamlessly control the prosthetic wrist to follow the target object and finally orient it for grasping according to the user intent.
  arXiv  Detail & Related papers  (2025-02-24T15:48:25Z)
• HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit [52.12750762494588]
  Current humanoid teleoperation systems either lack reliable low-level control policies, or struggle to acquire accurate whole-body control commands.<n>We propose a novel humanoid teleoperation cockpit integrates a humanoid loco-manipulation policy and a low-cost exoskeleton-based hardware system.
  arXiv  Detail & Related papers  (2025-02-18T16:33:38Z)
• Hand-Object Interaction Pretraining from Videos [77.92637809322231]
  We learn general robot manipulation priors from 3D hand-object interaction trajectories. We do so by sharing both the human hand and the manipulated object in 3D space and human motions to robot actions. We empirically demonstrate that finetuning this policy, with both reinforcement learning (RL) and behavior cloning (BC), enables sample-efficient adaptation to downstream tasks and simultaneously improves robustness and generalizability compared to prior approaches.
  arXiv  Detail & Related papers  (2024-09-12T17:59:07Z)
• Privileged Sensing Scaffolds Reinforcement Learning [28.100745092661587]
  We consider sensory scaffolding setups for training artificial agents. "Scaffolder" is a reinforcement learning approach which effectively exploits privileged sensing in critics. Agents must use privileged camera sensing to train blind hurdlers.
  arXiv  Detail & Related papers  (2024-05-23T17:57:14Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
  Planning motions that take human comfort into account is not a part of the human-robot handover process. We propose to generate smooth motions via an efficient model-predictive control framework. We conduct human-to-robot handover experiments on a diverse set of objects with several users.
  arXiv  Detail & Related papers  (2022-03-31T23:08:20Z)
• Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis [6.517935794312337]
  We present an eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences. We tackle the peculiarity of the eye-in-hand setting by means of a model for the human arm trajectories.
  arXiv  Detail & Related papers  (2022-03-18T09:16:48Z)
• Towards Predicting Fine Finger Motions from Ultrasound Images via Kinematic Representation [12.49914980193329]
  We study the inference problem of identifying the activation of specific fingers from a sequence of US images. We consider this task as an important step towards higher adoption rates of robotic prostheses among arm amputees.
  arXiv  Detail & Related papers  (2022-02-10T18:05:09Z)
• ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning [91.58711082348293]
  Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem. This approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse. We propose a hierarchical solution that learns efficiently from sparse user feedback.
  arXiv  Detail & Related papers  (2022-02-05T02:01:19Z)
• Design of an Affordable Prosthetic Arm Equipped with Deep Learning Vision-Based Manipulation [0.0]
  This paper lays the complete outline of the design process of an affordable and easily accessible novel prosthetic arm. The 3D printed prosthetic arm is equipped with a depth camera and closed-loop off-policy deep learning algorithm to help form grasps to the object in view. We were able to achieve a 78% grasp success rate on previously unseen objects and generalize across multiple objects for manipulation tasks.
  arXiv  Detail & Related papers  (2021-03-03T00:35:06Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.