BiFingerPose: Bimodal Finger Pose Estimation for Touch Devices

• URL: http://arxiv.org/abs/2511.17306v1
• Date: Fri, 21 Nov 2025 15:21:28 GMT
• Title: BiFingerPose: Bimodal Finger Pose Estimation for Touch Devices
• Authors: Xiongjun Guan, Zhiyu Pan, Jianjiang Feng, Jie Zhou,
• Abstract summary: We propose BiFingerPose, a novel bimodal based finger pose estimation algorithm.<n>A capacitive image and a fingerprint patch obtained from the touchscreen with an under-screen fingerprint sensor are explored.<n>Our approach leads to reliable estimation of roll angle, which is not achievable using only a single modality.
• Score: 31.932777820189827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Finger pose offers promising opportunities to expand human computer interaction capability of touchscreen devices. Existing finger pose estimation algorithms that can be implemented in portable devices predominantly rely on capacitive images, which are currently limited to estimating pitch and yaw angles and exhibit reduced accuracy when processing large-angle inputs (especially when it is greater than 45 degrees). In this paper, we propose BiFingerPose, a novel bimodal based finger pose estimation algorithm capable of simultaneously and accurately predicting comprehensive finger pose information. A bimodal input is explored, including a capacitive image and a fingerprint patch obtained from the touchscreen with an under-screen fingerprint sensor. Our approach leads to reliable estimation of roll angle, which is not achievable using only a single modality. In addition, the prediction performance of other pose parameters has also been greatly improved. The evaluation of a 12-person user study on continuous and discrete interaction tasks further validated the advantages of our approach. Specifically, BiFingerPose outperforms previous SOTA methods with over 21% improvement in prediction performance, 2.5 times higher task completion efficiency, and 23% better user operation accuracy, demonstrating its practical superiority. Finally, we delineate the application space of finger pose with respect to enhancing authentication security and improving interactive experiences, and develop corresponding prototypes to showcase the interaction potential. Our code will be available at https://github.com/XiongjunGuan/DualFingerPose.

Related papers

• OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction [93.88239833545623]
  We present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset.<n>We show that tactile signals provide a compact yet powerful cue for grasp understanding.<n>We aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation.
  arXiv  Detail & Related papers  (2025-12-18T18:18:17Z)
• Finger Pose Estimation for Under-screen Fingerprint Sensor [33.05877729161858]
  We present a novel dual-modal input based network for under-screen fingerprint pose estimation.<n>Our approach effectively integrates texture details extracted from ridge patches through the under-screen fingerprint sensor, and rough contours derived from capacitive images obtained via the touch screen.<n>Our method is significantly superior to previous state-of-the-art (SOTA) methods and remarkably boosts the recognition ability of fingerprint recognition algorithms.
  arXiv  Detail & Related papers  (2025-05-05T09:05:47Z)
• Joint Identity Verification and Pose Alignment for Partial Fingerprints [33.05877729161858]
  We propose a novel framework for joint identity verification and pose alignment of partial fingerprint pairs.<n>Our method achieves state-of-the-art performance in both partial fingerprint verification and relative pose estimation.
  arXiv  Detail & Related papers  (2024-05-07T02:45:50Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Finger-UNet: A U-Net based Multi-Task Architecture for Deep Fingerprint Enhancement [0.0]
  fingerprint enhancement plays a vital role in the early stages of the fingerprint recognition/verification pipeline. We suggest intuitive modifications to U-Net to enhance low-quality fingerprints effectively. We replace regular convolutions with depthwise separable convolutions, which significantly reduces the memory footprint of the model.
  arXiv  Detail & Related papers  (2023-10-01T09:49:10Z)
• Accelerated Fingerprint Enhancement: A GPU-Optimized Mixed Architecture Approach [47.87570819350573]
  This document presents a preliminary approach to latent fingerprint enhancement, fundamentally designed around a mixed Unet architecture. It combines the capabilities of the Resnet-101 network and Unet encoder, aiming to form a potentially powerful composite. One innovative element of this approach includes a novel Fingerprint Enhancement Gabor layer, specifically designed for GPU computations.
  arXiv  Detail & Related papers  (2023-06-01T01:19:32Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• FIGO: Enhanced Fingerprint Identification Approach Using GAN and One Shot Learning Techniques [0.0]
  We propose a Fingerprint Identification approach based on Generative adversarial network and One-shot learning techniques. First, we propose a Pix2Pix model to transform low-quality fingerprint images to a higher level of fingerprint images pixel by pixel directly in the fingerprint enhancement tier. Second, we construct a fully automated fingerprint feature extraction model using a one-shot learning approach to differentiate each fingerprint from the others in the fingerprint identification process.
  arXiv  Detail & Related papers  (2022-08-11T02:45:42Z)
• Mobile Behavioral Biometrics for Passive Authentication [65.94403066225384]
  This work carries out a comparative analysis of unimodal and multimodal behavioral biometric traits. Experiments are performed over HuMIdb, one of the largest and most comprehensive freely available mobile user interaction databases. In our experiments, the most discriminative background sensor is the magnetometer, whereas among touch tasks the best results are achieved with keystroke.
  arXiv  Detail & Related papers  (2022-03-14T17:05:59Z)
• A Contactless Fingerprint Recognition System [5.565364597145569]
  We propose an approach for developing a contactless fingerprint recognition system that captures finger photo from a distance. The captured finger photos are then processed further to obtain global and local (minutiae-based) features. The proposed system is developed using the Nvidia Jetson Nano development kit, which allows us to perform contactless fingerprint recognition in real-time.
  arXiv  Detail & Related papers  (2021-08-20T08:21:55Z)
• The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes? [57.366931129764815]
  We collect more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger.<n>Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
  arXiv  Detail & Related papers  (2017-10-16T05:32:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.