ContactPose: A Dataset of Grasps with Object Contact and Hand Pose

• URL: http://arxiv.org/abs/2007.09545v1
• Date: Sun, 19 Jul 2020 01:01:14 GMT
• Title: ContactPose: A Dataset of Grasps with Object Contact and Hand Pose
• Authors: Samarth Brahmbhatt, Chengcheng Tang, Christopher D. Twigg, Charles C. Kemp, James Hays
• Abstract summary: We introduce ContactPose, the first dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images.
• Score: 27.24450178180785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Grasping is natural for humans. However, it involves complex hand configurations and soft tissue deformation that can result in complicated regions of contact between the hand and the object. Understanding and modeling this contact can potentially improve hand models, AR/VR experiences, and robotic grasping. Yet, we currently lack datasets of hand-object contact paired with other data modalities, which is crucial for developing and evaluating contact modeling techniques. We introduce ContactPose, the first dataset of hand-object contact paired with hand pose, object pose, and RGB-D images. ContactPose has 2306 unique grasps of 25 household objects grasped with 2 functional intents by 50 participants, and more than 2.9 M RGB-D grasp images. Analysis of ContactPose data reveals interesting relationships between hand pose and contact. We use this data to rigorously evaluate various data representations, heuristics from the literature, and learning methods for contact modeling. Data, code, and trained models are available at https://contactpose.cc.gatech.edu.

Related papers

• NL2Contact: Natural Language Guided 3D Hand-Object Contact Modeling with Diffusion Model [45.00669505173757]
  NL2Contact is a model that generates controllable contacts by leveraging staged diffusion models. Given a language description of the hand and contact, NL2Contact generates realistic and faithful 3D hand-object contacts. We show applications of our model to grasp pose optimization and novel human grasp generation.
  arXiv  Detail & Related papers  (2024-07-17T16:46:40Z)
• ContactGen: Generative Contact Modeling for Grasp Generation [37.56729700157981]
  This paper presents a novel object-centric contact representation ContactGen for hand-object interaction. We propose a conditional generative model to predict ContactGen and adopt model-based optimization to predict diverse and geometrically feasible grasps.
  arXiv  Detail & Related papers  (2023-10-05T17:59:45Z)
• DECO: Dense Estimation of 3D Human-Scene Contact In The Wild [54.44345845842109]
  We train a novel 3D contact detector that uses both body-part-driven and scene-context-driven attention to estimate contact on the SMPL body. We significantly outperform existing SOTA methods across all benchmarks. We also show qualitatively that DECO generalizes well to diverse and challenging real-world human interactions in natural images.
  arXiv  Detail & Related papers  (2023-09-26T21:21:07Z)
• Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images [59.49985837246644]
  We show how to model contacts in an explicit way to benefit the implicit reconstruction of hand-held objects. In the first part, we propose a new subtask of directly estimating 3D hand-object contacts from a single image. In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space.
  arXiv  Detail & Related papers  (2023-05-31T17:59:26Z)
• ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation [46.815231896011284]
  We propose a new dataset and a novel approach to learning hand-object interaction priors for hand and articulated object pose estimation. We first collect a dataset using visual teleoperation, where the human operator can directly play within a physical simulator to manipulate the articulated objects. Our system only requires an iPhone to record human hand motion, which can be easily scaled up and largely lower the costs of data and annotation collection.
  arXiv  Detail & Related papers  (2023-05-02T17:24:08Z)
• HMDO: Markerless Multi-view Hand Manipulation Capture with Deformable Objects [8.711239906965893]
  HMDO is the first markerless deformable interaction dataset recording interactive motions of the hands and deformable objects. The proposed method can reconstruct interactive motions of hands and deformable objects with high quality.
  arXiv  Detail & Related papers  (2023-01-18T16:55:15Z)
• Full-Body Articulated Human-Object Interaction [61.01135739641217]
  CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions. CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process. By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
  arXiv  Detail & Related papers  (2022-12-20T19:50:54Z)
• BEHAVE: Dataset and Method for Tracking Human Object Interactions [105.77368488612704]
  We present the first full body human- object interaction dataset with multi-view RGBD frames and corresponding 3D SMPL and object fits along with the annotated contacts between them. We use this data to learn a model that can jointly track humans and objects in natural environments with an easy-to-use portable multi-camera setup.
  arXiv  Detail & Related papers  (2022-04-14T13:21:19Z)
• Estimating 3D Motion and Forces of Human-Object Interactions from Internet Videos [49.52070710518688]
  We introduce a method to reconstruct the 3D motion of a person interacting with an object from a single RGB video. Our method estimates the 3D poses of the person together with the object pose, the contact positions and the contact forces on the human body.
  arXiv  Detail & Related papers  (2021-11-02T13:40:18Z)
• ContactOpt: Optimizing Contact to Improve Grasps [17.518463627346897]
  Physical contact between hands and objects plays a critical role in human grasps. We show that optimizing the pose of a hand to achieve expected contact with an object can improve hand poses inferred via image-based methods.
  arXiv  Detail & Related papers  (2021-04-15T06:40:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.