PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis

• URL: http://arxiv.org/abs/2209.05022v1
• Date: Mon, 12 Sep 2022 04:49:41 GMT
• Title: PoseIt: A Visual-Tactile Dataset of Holding Poses for Grasp Stability Analysis
• Abstract summary: We present PoseIt, a novel dataset that contains visual and tactile data collected from a full cycle of grasping an object. Using data from PoseIt, we can formulate and tackle the task of predicting whether a grasped object is stable in a particular held pose. Our experimental results show that multi-modal models trained on PoseIt achieve higher accuracy than using solely vision or tactile data.
• Score: 7.644487678804541
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When humans grasp objects in the real world, we often move our arms to hold the object in a different pose where we can use it. In contrast, typical lab settings only study the stability of the grasp immediately after lifting, without any subsequent re-positioning of the arm. However, the grasp stability could vary widely based on the object's holding pose, as the gravitational torque and gripper contact forces could change completely. To facilitate the study of how holding poses affect grasp stability, we present PoseIt, a novel multi-modal dataset that contains visual and tactile data collected from a full cycle of grasping an object, re-positioning the arm to one of the sampled poses, and shaking the object. Using data from PoseIt, we can formulate and tackle the task of predicting whether a grasped object is stable in a particular held pose. We train an LSTM classifier that achieves 85% accuracy on the proposed task. Our experimental results show that multi-modal models trained on PoseIt achieve higher accuracy than using solely vision or tactile data and that our classifiers can also generalize to unseen objects and poses.

Related papers

• UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation [82.93208597526503]
  Existing methods are specialized, focusing on either bare-hand or hand interacting with object. No method can flexibly handle both scenarios and their performance degrades when applied to the other scenario. We propose UniHOPE, a unified approach for general 3D hand-object pose estimation.
  arXiv  Detail & Related papers  (2025-03-17T15:46:43Z)
• Learning Gentle Grasping Using Vision, Sound, and Touch [4.482495915731277]
  This paper proposes using visual, tactile, and auditory signals to learn to grasp and regrasp objects stably and gently.<n>We use audio signals as an indicator of gentleness during the grasping, and then train an action-conditional model from raw visuo-tactile inputs.<n> Experimental results on a multi-fingered hand over 1,500 grasping trials demonstrated that our model is useful for gentle grasping.
  arXiv  Detail & Related papers  (2025-03-11T00:12:25Z)
• Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model [25.00532805042292]
  We propose a simple yet effective framework that jointly models the relationship between the body, hands, and the given object motion sequences. We introduce novel contact-aware losses and incorporate a data-driven, carefully designed guidance. Experimental results demonstrate that our approach outperforms the state-of-the-art method and generates plausible whole-body motion sequences.
  arXiv  Detail & Related papers  (2024-12-30T02:21:43Z)
• AugInsert: Learning Robust Visual-Force Policies via Data Augmentation for Object Assembly Tasks [7.631503105866245]
  This paper primarily focuses on learning robust visual-force policies in the context of high-precision object assembly tasks. We aim to learn contact-rich manipulation policies with multisensory inputs on limited expert data by expanding human demonstrations via online data augmentation.
  arXiv  Detail & Related papers  (2024-10-19T04:19:52Z)
• VINECS: Video-based Neural Character Skinning [82.39776643541383]
  We propose a fully automated approach for creating a fully rigged character with pose-dependent skinning weights. We show that our approach outperforms state-of-the-art while not relying on dense 4D scans.
  arXiv  Detail & Related papers  (2023-07-03T08:35:53Z)
• PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimation [40.50255017107963]
  We propose Pose Transformation (PoseTrans) to create new training samples that have diverse poses. We also propose Pose Clustering Module (PCM) to measure the pose rarity and select the "rarest" poses to help balance the long-tailed distribution. Our method is efficient and simple to implement, which can be easily integrated into the training pipeline of existing pose estimation models.
  arXiv  Detail & Related papers  (2022-08-16T14:03:01Z)
• SporeAgent: Reinforced Scene-level Plausibility for Object Pose Refinement [28.244027792644097]
  While depth- and RGB-based pose refinement approaches increase the accuracy of the resulting pose estimates, they are susceptible to ambiguity as they consider visual alignment. We show that considering plausibility reduces ambiguity and, in consequence, allows poses to be more accurately predicted in cluttered environments. Experiments on the LINEMOD and YCB-VIDEO datasets demonstrate the state-of-the-art performance of our depth-based refinement approach.
  arXiv  Detail & Related papers  (2022-01-01T20:26:19Z)
• Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking [98.91894395941766]
  We propose a novel online approach to learning the pose dynamics, which are independent of pose detections in current fame. Specifically, we derive this prediction of dynamics through a graph neural network(GNN) that explicitly accounts for both spatial-temporal and visual information. Experiments on PoseTrack 2017 and PoseTrack 2018 datasets demonstrate that the proposed method achieves results superior to the state of the art on both human pose estimation and tracking tasks.
  arXiv  Detail & Related papers  (2021-06-07T16:36:50Z)
• Physically Plausible Pose Refinement using Fully Differentiable Forces [68.8204255655161]
  We propose an end-to-end differentiable model that refines pose estimates by learning the forces experienced by the object. By matching the learned net force to an estimate of net force based on finite differences of position, this model is able to find forces that accurately describe the movement of the object. We show this model successfully corrects poses and finds contact maps that better match the ground truth, despite not using any RGB or depth image data.
  arXiv  Detail & Related papers  (2021-05-17T23:33:04Z)
• Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation [58.72192168935338]
  Generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable. We propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our proposed model employs three consecutive differentiable transformations named as forward-kinematics, camera-projection and spatial-map transformation.
  arXiv  Detail & Related papers  (2020-06-24T23:56:33Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction [137.28465645405655]
  HANDS'19 is a challenge to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set.
  arXiv  Detail & Related papers  (2020-03-30T19:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.