Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects

• URL: http://arxiv.org/abs/2306.15858v1
• Date: Wed, 28 Jun 2023 01:18:53 GMT
• Title: Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects
• Authors: Alireza Rezazadeh, Snehal Dikhale, Soshi Iba and Nawid Jamali
• Abstract summary: We introduce a hierarchical graph neural network architecture for combining multimodal (vision and touch) data. We also introduce a hierarchical message passing operation that flows the information within and across modalities to learn a graph-based object representation.
• Score: 1.8263882169310044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic manipulation, in particular in-hand object manipulation, often requires an accurate estimate of the object's 6D pose. To improve the accuracy of the estimated pose, state-of-the-art approaches in 6D object pose estimation use observational data from one or more modalities, e.g., RGB images, depth, and tactile readings. However, existing approaches make limited use of the underlying geometric structure of the object captured by these modalities, thereby, increasing their reliance on visual features. This results in poor performance when presented with objects that lack such visual features or when visual features are simply occluded. Furthermore, current approaches do not take advantage of the proprioceptive information embedded in the position of the fingers. To address these limitations, in this paper: (1) we introduce a hierarchical graph neural network architecture for combining multimodal (vision and touch) data that allows for a geometrically informed 6D object pose estimation, (2) we introduce a hierarchical message passing operation that flows the information within and across modalities to learn a graph-based object representation, and (3) we introduce a method that accounts for the proprioceptive information for in-hand object representation. We evaluate our model on a diverse subset of objects from the YCB Object and Model Set, and show that our method substantially outperforms existing state-of-the-art work in accuracy and robustness to occlusion. We also deploy our proposed framework on a real robot and qualitatively demonstrate successful transfer to real settings.

Related papers

• GenFlow: Generalizable Recurrent Flow for 6D Pose Refinement of Novel Objects [14.598853174946656]
  We present GenFlow, an approach that enables both accuracy and generalization to novel objects. Our method predicts optical flow between the rendered image and the observed image and refines the 6D pose iteratively. It boosts the performance by a constraint of the 3D shape and the generalizable geometric knowledge learned from an end-to-end differentiable system.
  arXiv  Detail & Related papers  (2024-03-18T06:32:23Z)
• MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [84.80956484848505]
  MegaPose is a method to estimate the 6D pose of novel objects, that is, objects unseen during training. We present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner.
  arXiv  Detail & Related papers  (2022-12-13T19:30:03Z)
• Category-Agnostic 6D Pose Estimation with Conditional Neural Processes [19.387280883044482]
  We present a novel meta-learning approach for 6D pose estimation on unknown objects. Our algorithm learns object representation in a category-agnostic way, which endows it with strong generalization capabilities across object categories.
  arXiv  Detail & Related papers  (2022-06-14T20:46:09Z)
• What's in your hands? 3D Reconstruction of Generic Objects in Hands [49.12461675219253]
  Our work aims to reconstruct hand-held objects given a single RGB image. In contrast to prior works that typically assume known 3D templates and reduce the problem to 3D pose estimation, our work reconstructs generic hand-held object without knowing their 3D templates.
  arXiv  Detail & Related papers  (2022-04-14T17:59:02Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• VIPose: Real-time Visual-Inertial 6D Object Pose Tracking [3.44942675405441]
  We introduce a novel Deep Neural Network (DNN) called VIPose to address the object pose tracking problem in real-time. The key contribution is the design of a novel DNN architecture which fuses visual and inertial features to predict the objects' relative 6D pose. The approach presents accuracy performances comparable to state-of-the-art techniques, but with additional benefit to be real-time.
  arXiv  Detail & Related papers  (2021-07-27T06:10:23Z)
• DONet: Learning Category-Level 6D Object Pose and Size Estimation from Depth Observation [53.55300278592281]
  We propose a method of Category-level 6D Object Pose and Size Estimation (COPSE) from a single depth image. Our framework makes inferences based on the rich geometric information of the object in the depth channel alone. Our framework competes with state-of-the-art approaches that require labeled real-world images.
  arXiv  Detail & Related papers  (2021-06-27T10:41:50Z)
• Spatial Attention Improves Iterative 6D Object Pose Estimation [52.365075652976735]
  We propose a new method for 6D pose estimation refinement from RGB images. Our main insight is that after the initial pose estimate, it is important to pay attention to distinct spatial features of the object. We experimentally show that this approach learns to attend to salient spatial features and learns to ignore occluded parts of the object, leading to better pose estimation across datasets.
  arXiv  Detail & Related papers  (2021-01-05T17:18:52Z)
• Self6D: Self-Supervised Monocular 6D Object Pose Estimation [114.18496727590481]
  We propose the idea of monocular 6D pose estimation by means of self-supervised learning. We leverage recent advances in neural rendering to further self-supervise the model on unannotated real RGB-D data.
  arXiv  Detail & Related papers  (2020-04-14T13:16:36Z)
• Object 6D Pose Estimation with Non-local Attention [29.929911622127502]
  We propose a network that integrate 6D object pose parameter estimation into the object detection framework. The proposed method reaches the state-of-the-art performance on the YCB-video and the Linemod datasets.
  arXiv  Detail & Related papers  (2020-02-20T14:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.