PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images

• URL: http://arxiv.org/abs/2301.02310v3
• Date: Wed, 3 Jan 2024 18:59:57 GMT
• Title: PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images
• Authors: Patrick Grady, Jeremy A. Collins, Chengcheng Tang, Christopher D. Twigg, Kunal Aneja, James Hays, Charles C. Kemp
• Abstract summary: Deep models can estimate hand pressure based on a single RGB image. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces.
• Score: 23.877554759345607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Touch plays a fundamental role in manipulation for humans; however, machine perception of contact and pressure typically requires invasive sensors. Recent research has shown that deep models can estimate hand pressure based on a single RGB image. However, evaluations have been limited to controlled settings since collecting diverse data with ground-truth pressure measurements is difficult. We present a novel approach that enables diverse data to be captured with only an RGB camera and a cooperative participant. Our key insight is that people can be prompted to apply pressure in a certain way, and this prompt can serve as a weak label to supervise models to perform well under varied conditions. We collect a novel dataset with 51 participants making fingertip contact with diverse objects. Our network, PressureVision++, outperforms human annotators and prior work. We also demonstrate an application of PressureVision++ to mixed reality where pressure estimation allows everyday surfaces to be used as arbitrary touch-sensitive interfaces. Code, data, and models are available online.

Related papers

• Digitizing Touch with an Artificial Multimodal Fingertip [51.7029315337739]
  Humans and robots both benefit from using touch to perceive and interact with the surrounding environment. Here, we describe several conceptual and technological innovations to improve the digitization of touch. These advances are embodied in an artificial finger-shaped sensor with advanced sensing capabilities.
  arXiv  Detail & Related papers  (2024-11-04T18:38:50Z)
• EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision [69.1005706608681]
  We introduce EgoPressure, a novel dataset of touch contact and pressure interaction from an egocentric perspective. EgoPressure comprises 5.0 hours of touch contact and pressure interaction from 21 participants captured by a moving egocentric camera and 7 stationary Kinect cameras.
  arXiv  Detail & Related papers  (2024-09-03T18:53:32Z)
• Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation [57.60490773016364]
  We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation.
  arXiv  Detail & Related papers  (2023-12-20T22:36:37Z)
• PressureTransferNet: Human Attribute Guided Dynamic Ground Pressure Profile Transfer using 3D simulated Pressure Maps [7.421780713537146]
  PressureTransferNet is an encoder-decoder model taking a source pressure map and a target human attribute vector as inputs. We use a sensor simulation to create a diverse dataset with various human attributes and pressure profiles. We visually confirm the fidelity of the synthesized pressure shapes using a physics-based deep learning model and achieve a binary R-square value of 0.79 on areas with ground contact.
  arXiv  Detail & Related papers  (2023-08-01T13:31:25Z)
• RidgeBase: A Cross-Sensor Multi-Finger Contactless Fingerprint Dataset [10.219621548854343]
  RidgeBase consists of more than 15,000 contactless and contact-based fingerprint image pairs acquired from 88 individuals. Unlike existing datasets, RidgeBase is designed to promote research under different matching scenarios. We propose a set-based matching protocol inspired by the advances in facial recognition datasets.
  arXiv  Detail & Related papers  (2023-07-09T22:09:15Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• PressureVision: Estimating Hand Pressure from a Single RGB Image [27.449311565446443]
  We explore the possibility of using a conventional RGB camera to infer hand pressure. We collected videos of 36 participants with diverse skin tone applying pressure to an instrumented planar surface. We trained a deep model (PressureVisionNet) to infer a pressure image from a single RGB image.
  arXiv  Detail & Related papers  (2022-03-19T19:54:56Z)
• Mobile Behavioral Biometrics for Passive Authentication [65.94403066225384]
  This work carries out a comparative analysis of unimodal and multimodal behavioral biometric traits. Experiments are performed over HuMIdb, one of the largest and most comprehensive freely available mobile user interaction databases. In our experiments, the most discriminative background sensor is the magnetometer, whereas among touch tasks the best results are achieved with keystroke.
  arXiv  Detail & Related papers  (2022-03-14T17:05:59Z)
• Pressure Eye: In-bed Contact Pressure Estimation via Contact-less Imaging [18.35652911833834]
  We present our pressure eye (PEye) approach to estimate contact pressure between a human body and the surface she is lying on. PEye could ultimately enable the prediction and early detection of pressure ulcers in bed-bound patients.
  arXiv  Detail & Related papers  (2022-01-27T22:22:17Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.