Prosthetic Hand Manipulation System Based on EMG and Eye Tracking Powered by the Neuromorphic Processor AltAi

• URL: http://arxiv.org/abs/2601.17991v2
• Date: Fri, 20 Feb 2026 13:21:33 GMT
• Title: Prosthetic Hand Manipulation System Based on EMG and Eye Tracking Powered by the Neuromorphic Processor AltAi
• Abstract summary: This paper presents a novel neuromorphic control architecture for upper-limb prostheses that combines surface electromyography (sEMG) with gaze-guided computer vision.<n>The system uses a spiking neural network deployed on the neuromorphic processor AltAi to classify EMG patterns in real time while an eye-tracking headset and scene camera identify the object within the user's focus.
• Score: 1.9044180551673717
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents a novel neuromorphic control architecture for upper-limb prostheses that combines surface electromyography (sEMG) with gaze-guided computer vision. The system uses a spiking neural network deployed on the neuromorphic processor AltAi to classify EMG patterns in real time while an eye-tracking headset and scene camera identify the object within the user's focus. In our prototype, the same EMG recognition model that was originally developed for a conventional GPU is deployed as a spiking network on AltAi, achieving comparable accuracy while operating in a sub-watt power regime, which enables a lightweight, wearable implementation. For six distinct functional gestures recorded from upper-limb amputees, the system achieves robust recognition performance comparable to state-of-the-art myoelectric interfaces. When the vision pipeline restricts the decision space to three context-appropriate gestures for the currently viewed object, recognition accuracy increases to roughly 95% while excluding unsafe, object-inappropriate grasps. These results indicate that the proposed neuromorphic, context-aware controller can provide energy-efficient and reliable prosthesis control and has the potential to improve safety and usability in everyday activities for people with upper-limb amputation.

Related papers

• Real-Time sEMG-Based Telecontrol of an Assistive Robotic Arm Using a 1D Convolutional Neural Network [0.30786914102688595]
  This work proposes a real-time sEMG-based interface for the teleoperation of an assistive robotic arm.<n>The system relies on four-channel sEMG acquisition, signal preprocessing, segmentation into sliding windows, and classification using a one-dimensional convolutional neural network (CNN)<n>The system exhibits stable real-time behavior, with an average latency of approximately 0.32 s consistent with the chosen windowing strategy.
  arXiv  Detail & Related papers  (2026-07-14T15:34:28Z)
• Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality [40.023882327049684]
  Surgical navigation provides real-time guidance by estimating the pose of patient anatomy and surgical instruments.<n>Most existing approaches rely on a clear line of sight, which is difficult to maintain in dynamic operating room environments.<n>This work introduces a framework for tracking surgical instruments under occlusion by fusing multiple sensing modalities within a dynamic scene graph representation.
  arXiv  Detail & Related papers  (2026-03-09T05:37:49Z)
• Decoding Listeners Identity: Person Identification from EEG Signals Using a Lightweight Spiking Transformer [31.66487449656124]
  We propose a novel EEG person identification approach using spiking neural networks (SNNs) with a lightweight spiking transformer for efficiency and effectiveness.<n>The proposed SNN model is capable of handling the temporal complexities inherent in EEG signals.<n>On the EEG-Music Emotion Recognition Challenge dataset, the proposed model achieves 100% classification accuracy with less than 10% energy consumption.
  arXiv  Detail & Related papers  (2025-10-17T08:20:01Z)
• Finger Force Decoding from Motor Units Activity on Neuromorphic Hardware [1.8754256211583082]
  We propose a novel approach to perform finger force regression using spike trains from individual motor neurons.<n>These biologically grounded signals drive a spiking neural network implemented on a mixed-signal neuromorphic processor.<n>This is the first demonstration of motor neuron-based continuous regression computed directly on neuromorphic hardware.
  arXiv  Detail & Related papers  (2025-07-31T11:55:02Z)
• BrainOmni: A Brain Foundation Model for Unified EEG and MEG Signals [46.121056431476156]
  This paper proposes Brain Omni, the first brain foundation model that generalises across heterogeneous EEG and MEG recordings.<n>Existing approaches typically rely on separate, modality- and dataset-specific models, which limits performance and cross-domain scalability.<n>A total of 1,997 hours of EEG and 656 hours of MEG data are curated and standardised from publicly available sources for pretraining.
  arXiv  Detail & Related papers  (2025-05-18T14:07:14Z)
• Efficient and Robust Multidimensional Attention in Remote Physiological Sensing through Target Signal Constrained Factorization [7.947387272047604]
  We present MMRPhys, an efficient dual-branch 3D-CNN architecture designed for simultaneous estimation of photoplethysmography (rRSP) and respiratory (rRSP) signals from multimodal video inputs.<n>We demonstrate that MMRPhys with TSFM significantly outperforms state-of-the-art methods in generalization across domain shifts for rRSP estimation, while maintaining a minimal inference latency suitable for real-time applications.
  arXiv  Detail & Related papers  (2025-05-11T15:20:45Z)
• Event-Driven Implementation of a Physical Reservoir Computing Framework for superficial EMG-based Gesture Recognition [2.222098162797332]
  This paper explores a novel neuromorphic implementation approach for gesture recognition by extracting spiking information from surface electromyography (sEMG) data in an event-driven manner.<n>The network was designed by implementing a simple-structured and hardware-friendly Physical Reservoir Computing framework called Rotating Neuron Reservoir (RNR) within the domain of Spiking neural network (SNN)<n>The proposed system was validated by an open-access large-scale sEMG database and achieved an average classification accuracy of 74.6% and 80.3%.
  arXiv  Detail & Related papers  (2025-03-10T17:18:14Z)
• REST: Efficient and Accelerated EEG Seizure Analysis through Residual State Updates [54.96885726053036]
  This paper introduces a novel graph-based residual state update mechanism (REST) for real-time EEG signal analysis. By leveraging a combination of graph neural networks and recurrent structures, REST efficiently captures both non-Euclidean geometry and temporal dependencies within EEG data. Our model demonstrates high accuracy in both seizure detection and classification tasks.
  arXiv  Detail & Related papers  (2024-06-03T16:30:19Z)
• Agile gesture recognition for capacitive sensing devices: adapting on-the-job [55.40855017016652]
  We demonstrate a hand gesture recognition system that uses signals from capacitive sensors embedded into the etee hand controller. The controller generates real-time signals from each of the wearer five fingers. We use a machine learning technique to analyse the time series signals and identify three features that can represent 5 fingers within 500 ms.
  arXiv  Detail & Related papers  (2023-05-12T17:24:02Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection [122.4894940892536]
  We present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss.
  arXiv  Detail & Related papers  (2022-09-25T04:56:10Z)
• Low Power Neuromorphic EMG Gesture Classification [3.8761525368152725]
  Spiking Neural Networks (SNNs) are promising for low-power, real-time EMG gesture recognition. We present low-power, high accuracy demonstration of EMG-signal based gesture recognition using neuromorphic Recurrent Spiking Neural Networks (RSNN) Our network achieves state-of-the-art accuracy classification (90%) while using 53% than best reported art on Roshambo EMG dataset.
  arXiv  Detail & Related papers  (2022-06-04T22:09:34Z)
• Towards Creating a Deployable Grasp Type Probability Estimator for a Prosthetic Hand [11.008123712007402]
  InceptionV3 achieves highest accuracy with 0.95 angular similarity followed by 1.4 MobileNetV2 with 0.93 at 20% the amount of operations. Our work enables augmenting EMG intent inference with physical state probability through machine learning and computer vision method.
  arXiv  Detail & Related papers  (2021-01-13T21:39:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.