AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Gloves

• URL: http://arxiv.org/abs/2602.05159v1
• Date: Thu, 05 Feb 2026 00:24:02 GMT
• Title: AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Gloves
• Abstract summary: We present the first systematic evaluation of vision-based hand tracking models on gloved hands.<n>Existing bare-hand models suffer from substantial performance degradation due to large appearance gap between bare-hand and glove designs.<n>We propose AirGlove, which leverages existing gloves to generalize the learned glove representations towards new gloves with limited data.
• Score: 10.234494805527943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sensing gloves have become important tools for teleoperation and robotic policy learning as they are able to provide rich signals like speed, acceleration and tactile feedback. A common approach to track gloved hands is to directly use the sensor signals (e.g., angular velocity, gravity orientation) to estimate 3D hand poses. However, sensor-based tracking can be restrictive in practice as the accuracy is often impacted by sensor signal and calibration quality. Recent advances in vision-based approaches have achieved strong performance on human hands via large-scale pre-training, but their performance on gloved hands with distinct visual appearances remains underexplored. In this work, we present the first systematic evaluation of vision-based hand tracking models on gloved hands under both zero-shot and fine-tuning setups. Our analysis shows that existing bare-hand models suffer from substantial performance degradation on sensing gloves due to large appearance gap between bare-hand and glove designs. We therefore propose AirGlove, which leverages existing gloves to generalize the learned glove representations towards new gloves with limited data. Experiments with multiple sensing gloves show that AirGlove effectively generalizes the hand pose models to new glove designs and achieves a significant performance boost over the compared schemes.

Related papers

• Teleopit: A Full-Embodiment Humanoid Teleoperation System [3.436474641224862]
  We introduce Teleopit, a full-embodiment teleoperation system that maps body, hand, and signals from VR to a humanoid body.<n>A history encoder and failure-aware rewind sampling improve the motion tracker on both motion-capture and live VR references.<n>ACT and GRT N1.7 policies trained on 96 successful demonstrations collected with Teleopit achieve task success rates of 90.0% and 95.0%, respectively.
  arXiv  Detail & Related papers  (2026-08-03T07:45:13Z)
• Pose-Aware Modeling to Mitigate Pose-Related Artifacts in Tactile Gloves [12.960256702707596]
  Tactile gloves digitize contact and force during hand-object interactions.<n> flexible sensors are sensitive not only to contact forces but also to hand pose changes, resulting in pose-related artifacts (PRAs)<n>We introduce a glove-agnostic algorithmic framework that leverages hand pose information, which is increasingly available, to mitigate PRAs without glove modifications.
  arXiv  Detail & Related papers  (2026-07-25T00:12:31Z)
• AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance [52.973947477815614]
  AnyDexRT is a calibration-free method for intuitive dexterous teleoperation across human-like dexterous hands.<n>Experiments show that AnyDexRT improves quality, reduces manual tuning, and provides more intuitive and efficient control than prior methods.
  arXiv  Detail & Related papers  (2026-07-09T10:29:53Z)
• Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves [29.4768155708928]
  Glove2Hand is a framework that translates multi-modal sensing glove HOI videos into bare hands.<n>HandSense is the first multi-modal HOI dataset featuring glove-to-hand videos with synchronized tactile and IMU signals.
  arXiv  Detail & Related papers  (2026-03-21T15:12:57Z)
• HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit [52.12750762494588]
  This paper introduces HOMIE, a semi-autonomous teleoperation system.<n>It combines a reinforcement learning policy for body control mapped to a pedal, an isomorphic exoskeleton arm for arm control, and motion-sensing gloves for hand control.<n>The system is fully open-source, demos and code can be found in our https://homietele.org/.
  arXiv  Detail & Related papers  (2025-02-18T16:33:38Z)
• mmEgoHand: Egocentric Hand Pose Estimation and Gesture Recognition with Head-mounted Millimeter-wave Radar and IMU [15.086294532079908]
  mmEgoHand is a head-mounted egocentric system for hand pose estimation.<n>It supports applications such as gesture recognition, VR interaction, skill digitization and assessment, and robotic teleoperation.<n>mmEgoHand integrates mmWave radar with inertial measurement units (IMUs) to enable dynamic perception.
  arXiv  Detail & Related papers  (2025-01-23T16:25:08Z)
• Vision Controlled Sensorized Prosthetic Hand [0.31666540219908274]
  This paper presents a sensorized vision-enabled prosthetic hand aimed at replicating a natural hand's performance, functionality, appearance, and comfort. The design goal was to create an accessible substitution with a user-friendly interface requiring little to no training.
  arXiv  Detail & Related papers  (2024-06-25T14:44:04Z)
• UltraGlove: Hand Pose Estimation with Mems-Ultrasonic Sensors [14.257535961674021]
  We propose a novel and low-cost hand-tracking glove that utilizes several MEMS-ultrasonic sensors attached to the fingers. Our experimental results demonstrate that this approach is both accurate, size-agnostic, and robust to external interference.
  arXiv  Detail & Related papers  (2023-06-22T03:41:47Z)
• Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimation [59.3035531612715]
  Existing methods often struggle to generate plausible hand poses when the hand is heavily occluded or blurred. In videos, the movements of the hand allow us to observe various parts of the hand that may be occluded or blurred in a single frame. We propose the Deformer: a framework that implicitly reasons about the relationship between hand parts within the same image.
  arXiv  Detail & Related papers  (2023-03-09T02:24:30Z)
• Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps [100.72245315180433]
  We present a reconfigurable data glove design to capture different modes of human hand-object interactions. The glove operates in three modes for various downstream tasks with distinct features. We evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses.
  arXiv  Detail & Related papers  (2023-01-14T05:35:50Z)
• MC-hands-1M: A glove-wearing hand dataset for pose estimation [16.83787646285556]
  We present a synthetic dataset for 3D pose estimation of glove-wearing hands. The dataset is used to fine-tune a public hand joint detection model.
  arXiv  Detail & Related papers  (2022-10-19T09:50:04Z)
• HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances [84.7225785061814]
  HiFECap simultaneously captures human pose, clothing, facial expression, and hands just from a single RGB video. Our method also captures high-frequency details, such as deforming wrinkles on the clothes, better than the previous works.
  arXiv  Detail & Related papers  (2022-10-11T17:57:45Z)
• 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
  Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
  arXiv  Detail & Related papers  (2022-07-22T13:04:06Z)
• A Skeleton-Driven Neural Occupancy Representation for Articulated Hands [49.956892429789775]
  Hand ArticuLated Occupancy (HALO) is a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces. We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects.
  arXiv  Detail & Related papers  (2021-09-23T14:35:19Z)
• Egocentric View Hand Action Recognition by Leveraging Hand Surface and Hand Grasp Type [15.878905144552204]
  The framework synthesizes the mean curvature of the hand mesh model to encode the hand surface geometry in 3D space. Using hand grasp type and mean curvature of hand increases the performance of the hand action recognition.
  arXiv  Detail & Related papers  (2021-09-08T17:12:02Z)
• Tracking Hand Hygiene Gestures with Leap Motion Controller [0.0]
  The process of hand washing, according to the WHO, is divided into stages with clearly defined two handed dynamic gestures. In this paper, videos of hand washing experts are segmented and analyzed with the goal of extracting their corresponding features. A 3D gesture tracker, the Leap Motion Controller (LEAP), was used to track and detect the hand features associated with these stages.
  arXiv  Detail & Related papers  (2021-08-11T08:48:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.