ALVI Interface: Towards Full Hand Motion Decoding for Amputees Using sEMG

• URL: http://arxiv.org/abs/2502.21256v1
• Date: Fri, 28 Feb 2025 17:29:35 GMT
• Title: ALVI Interface: Towards Full Hand Motion Decoding for Amputees Using sEMG
• Abstract summary: We present a system for decoding hand movements using surface EMG signals.<n>The interface provides real-time reconstruction of finger joint angles across 20 degrees of freedom, designed for upper limb amputees.
• Score: 32.472478101675655
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present a system for decoding hand movements using surface EMG signals. The interface provides real-time (25 Hz) reconstruction of finger joint angles across 20 degrees of freedom, designed for upper limb amputees. Our offline analysis shows 0.8 correlation between predicted and actual hand movements. The system functions as an integrated pipeline with three key components: (1) a VR-based data collection platform, (2) a transformer-based model for EMG-to-motion transformation, and (3) a real-time calibration and feedback module called ALVI Interface. Using eight sEMG sensors and a VR training environment, users can control their virtual hand down to finger joint movement precision, as demonstrated in our video: youtube link.

Related papers

• Sign in the Air to Unlock: An Interface for authentication in Virtual and Augmented Reality Powered by Point-Voxel Cross-Attention Network [0.0]
  Sign in the Air to Unlock is an in-air signature interface that enables users to authenticate by signing naturally in 3D space.<n>PV-Net achieves an Equal Error Rate of 2.5% on DeepAirSig and 76% classification accuracy on ImmAirsig.
  arXiv  Detail & Related papers  (2026-07-01T19:56:55Z)
• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• AVI-HT: Adaptive Vision-IMU Fusion for 3D Hand Tracking [7.3378748076111036]
  We present an adaptive visual-IMU fusion approach for tracking 3D hand poses by jointly modeling the egocentric image with on-glove 6-DoF IMU signals.<n>Avi-HT achieves significantly improved accuracy and availability, particularly in hand-object interaction scenarios.<n>We conduct extensive experiments on our DexGloveHOI dataset that consists of 100K+ pairwise vision-IMU samples with synchronized 3D annotated poses.
  arXiv  Detail & Related papers  (2026-05-20T20:19:24Z)
• MOSAIC: Bridging the Sim-to-Real Gap in Generalist Humanoid Motion Tracking and Teleoperation with Rapid Residual Adaptation [50.643488406438514]
  Generalist humanoid motion trackers have recently achieved strong simulation metrics by scaling data and training, yet often remain brittle on hardware during sustained teleoperation.<n>We present MOSAIC, an open-source, full-stack system for humanoid motion tracking and whole-body teleoperation across multiple interfaces.
  arXiv  Detail & Related papers  (2026-02-09T12:35:01Z)
• Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors [25.67875816218477]
  Full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range.<n>Previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints.<n>To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists.
  arXiv  Detail & Related papers  (2025-05-08T15:28:09Z)
• mmEgoHand: Egocentric Hand Pose Estimation and Gesture Recognition with Head-mounted Millimeter-wave Radar and IMU [15.086294532079908]
  mmEgoHand is a head-mounted egocentric system for hand pose estimation.<n>It supports applications such as gesture recognition, VR interaction, skill digitization and assessment, and robotic teleoperation.<n>mmEgoHand integrates mmWave radar with inertial measurement units (IMUs) to enable dynamic perception.
  arXiv  Detail & Related papers  (2025-01-23T16:25:08Z)
• Towards Consumer-Grade Cybersickness Prediction: Multi-Model Alignment for Real-Time Vision-Only Inference [3.4667973471411853]
  Cybersickness is a major obstacle to the widespread adoption of immersive virtual reality (VR)<n>We propose a scalable, deployable framework for personalized cybersickness prediction.<n>Our framework supports real-time applications, ideal for integration into consumer-grade VR platforms.
  arXiv  Detail & Related papers  (2025-01-02T11:41:43Z)
• emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation [12.566524562446467]
  Reliable and always-available hand pose inference could yield new and intuitive control schemes for human-computer interactions.<n>Wearable wrist-based surface electromyography (sEMG) presents a promising alternative.<n>emg2pose is the largest publicly available dataset of high-quality hand pose labels and wrist sEMG recordings.
  arXiv  Detail & Related papers  (2024-12-02T23:39:37Z)
• Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps [100.72245315180433]
  We present a reconfigurable data glove design to capture different modes of human hand-object interactions. The glove operates in three modes for various downstream tasks with distinct features. We evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses.
  arXiv  Detail & Related papers  (2023-01-14T05:35:50Z)
• Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergy [52.03068246508119]
  We present IMAS, a method that segments the primary objects in videos without manual annotation in training or inference. IMAS achieves Improved UVOS with Motion-Appearance Synergy. We demonstrate its effectiveness in tuning critical hyperparams previously tuned with human annotation or hand-crafted hyperparam-specific metrics.
  arXiv  Detail & Related papers  (2022-12-17T06:47:30Z)
• Simultaneous Estimation of Hand Configurations and Finger Joint Angles using Forearm Ultrasound [8.753262480814493]
  Forearm ultrasound images provide a musculoskeletal visualization that can be used to understand hand motion. We propose a CNN based deep learning pipeline for predicting the MCP joint angles. A low latency pipeline has been proposed for estimating both MCP joint angles and hand configuration aimed at real-time control of human-machine interfaces.
  arXiv  Detail & Related papers  (2022-11-29T02:06:19Z)
• Force-Aware Interface via Electromyography for Natural VR/AR Interaction [69.1332992637271]
  We design a learning-based neural interface for natural and intuitive force inputs in VR/AR. We show that our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. We envision our findings to push forward research towards more realistic physicality in future VR/AR.
  arXiv  Detail & Related papers  (2022-10-03T20:51:25Z)
• AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing [24.053096294334694]
  We present AvatarPoser, the first learning-based method that predicts full-body poses in world coordinates using only motion input from the user's head and hands. Our method builds on a Transformer encoder to extract deep features from the input signals and decouples global motion from the learned local joint orientations. In our evaluation, AvatarPoser achieved new state-of-the-art results in evaluations on large motion capture datasets.
  arXiv  Detail & Related papers  (2022-07-27T20:52:39Z)
• Forward and Inverse models in HCI:Physical simulation and deep learning for inferring 3D finger pose [2.8952292379640636]
  We use machine learning to develop data-driven models to infer position, pose and sensor readings. We combine a Conditional Variational Autoencoder with domain expertise/models experimentally collected data.
  arXiv  Detail & Related papers  (2021-09-07T23:11:21Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Physics-Based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning [52.37106940303246]
  We learn a model that maps noisy input hand poses to target virtual poses. The agent is trained in a residual setting by using a model-free hybrid RL+IL approach. We test our framework in two applications that use hand pose estimates for dexterous manipulations: hand-object interactions in VR and hand-object motion reconstruction in-the-wild.
  arXiv  Detail & Related papers  (2020-08-07T17:34:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.