HannesImitation: Grasping with the Hannes Prosthetic Hand via Imitation Learning

• URL: http://arxiv.org/abs/2508.00491v1
• Date: Fri, 01 Aug 2025 10:09:38 GMT
• Title: HannesImitation: Grasping with the Hannes Prosthetic Hand via Imitation Learning
• Authors: Carlo Alessi, Federico Vasile, Federico Ceola, Giulia Pasquale, Nicolò Boccardo, Lorenzo Natale,
• Abstract summary: In robotics, imitation learning has emerged as a promising approach for learning grasping and complex manipulation tasks.<n>We present HannesImitationPolicy, an imitation learning-based method to control the Hannes prosthetic hand.<n>We leverage such data to train a single diffusion policy and deploy it on the prosthetic hand to predict the wrist orientation and hand closure for grasping.
• Score: 5.122722600158078
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in control of prosthetic hands have focused on increasing autonomy through the use of cameras and other sensory inputs. These systems aim to reduce the cognitive load on the user by automatically controlling certain degrees of freedom. In robotics, imitation learning has emerged as a promising approach for learning grasping and complex manipulation tasks while simplifying data collection. Its application to the control of prosthetic hands remains, however, largely unexplored. Bridging this gap could enhance dexterity restoration and enable prosthetic devices to operate in more unconstrained scenarios, where tasks are learned from demonstrations rather than relying on manually annotated sequences. To this end, we present HannesImitationPolicy, an imitation learning-based method to control the Hannes prosthetic hand, enabling object grasping in unstructured environments. Moreover, we introduce the HannesImitationDataset comprising grasping demonstrations in table, shelf, and human-to-prosthesis handover scenarios. We leverage such data to train a single diffusion policy and deploy it on the prosthetic hand to predict the wrist orientation and hand closure for grasping. Experimental evaluation demonstrates successful grasps across diverse objects and conditions. Finally, we show that the policy outperforms a segmentation-based visual servo controller in unstructured scenarios. Additional material is provided on our project page: https://hsp-iit.github.io/HannesImitation

Related papers

• Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper [7.618517580705364]
  We present a portable, lightweight gripper with integrated tactile sensors.<n>We propose a cross-modal representation learning framework that integrates visual and tactile signals.<n>We validate our approach on fine-grained tasks such as test tube insertion and pipette-based fluid transfer.
  arXiv  Detail & Related papers  (2025-07-20T17:53:59Z)
• Towards Biosignals-Free Autonomous Prosthetic Hand Control via Imitation Learning [1.072044330361478]
  This study aims to develop a fully autonomous control system for a prosthetic hand.<n>By placing the hand near an object, the system will automatically execute grasping actions with a proper grip force.<n>To release the object being grasped, just naturally place the object close to the table and the system will automatically open the hand.
  arXiv  Detail & Related papers  (2025-06-10T13:44:08Z)
• Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping [4.476245767508223]
  We present a novel eye-in-hand prosthetic grasping system that follows shared-autonomy principles.<n>Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand.<n>We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness.
  arXiv  Detail & Related papers  (2025-03-01T12:35:05Z)
• Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework [5.428117915362002]
  Most control techniques for prosthetic grasping focus on dexterous fingers control, but overlook the wrist motion.<n>This forces the user to perform compensatory movements with the elbow, shoulder and hip to adapt the wrist for grasping.<n>We propose a computer vision-based system that leverages the collaboration between the user and an automatic system in a shared autonomy framework.<n>Our pipeline allows to seamlessly control the prosthetic wrist to follow the target object and finally orient it for grasping according to the user intent.
  arXiv  Detail & Related papers  (2025-02-24T15:48:25Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• Context-Aware Sequence Alignment using 4D Skeletal Augmentation [67.05537307224525]
  Temporal alignment of fine-grained human actions in videos is important for numerous applications in computer vision, robotics, and mixed reality. We propose a novel context-aware self-supervised learning architecture to align sequences of actions. Specifically, CASA employs self-attention and cross-attention mechanisms to incorporate the spatial and temporal context of human actions.
  arXiv  Detail & Related papers  (2022-04-26T10:59:29Z)
• Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis [6.517935794312337]
  We present an eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences. We tackle the peculiarity of the eye-in-hand setting by means of a model for the human arm trajectories.
  arXiv  Detail & Related papers  (2022-03-18T09:16:48Z)
• Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control [67.23580984118479]
  Imitation Learning (IL) is an effective framework to learn visuomotor skills from offline demonstration data. Hand-eye Action Networks (HAN) can approximate human's hand-eye coordination behaviors by learning from human teleoperated demonstrations.
  arXiv  Detail & Related papers  (2021-02-28T01:49:13Z)
• Human-in-the-Loop Imitation Learning using Remote Teleoperation [72.2847988686463]
  We build a data collection system tailored to 6-DoF manipulation settings. We develop an algorithm to train the policy iteratively on new data collected by the system. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators.
  arXiv  Detail & Related papers  (2020-12-12T05:30:35Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.