Challenges for Monocular 6D Object Pose Estimation in Robotics

• URL: http://arxiv.org/abs/2307.12172v2
• Date: Sat, 27 Jul 2024 17:02:51 GMT
• Title: Challenges for Monocular 6D Object Pose Estimation in Robotics
• Authors: Stefan Thalhammer, Dominik Bauer, Peter Hönig, Jean-Baptiste Weibel, José García-Rodríguez, Markus Vincze,
• Abstract summary: We provide a unified view on recent publications from both robotics and computer vision. We find that occlusion handling, novel pose representations, and formalizing and improving category-level pose estimation are still fundamental challenges. In order to address them, ontological reasoning, deformability handling, scene-level reasoning, realistic datasets, and the ecological footprint of algorithms need to be improved.
• Score: 12.037567673872662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object pose estimation is a core perception task that enables, for example, object grasping and scene understanding. The widely available, inexpensive and high-resolution RGB sensors and CNNs that allow for fast inference based on this modality make monocular approaches especially well suited for robotics applications. We observe that previous surveys on object pose estimation establish the state of the art for varying modalities, single- and multi-view settings, and datasets and metrics that consider a multitude of applications. We argue, however, that those works' broad scope hinders the identification of open challenges that are specific to monocular approaches and the derivation of promising future challenges for their application in robotics. By providing a unified view on recent publications from both robotics and computer vision, we find that occlusion handling, novel pose representations, and formalizing and improving category-level pose estimation are still fundamental challenges that are highly relevant for robotics. Moreover, to further improve robotic performance, large object sets, novel objects, refractive materials, and uncertainty estimates are central, largely unsolved open challenges. In order to address them, ontological reasoning, deformability handling, scene-level reasoning, realistic datasets, and the ecological footprint of algorithms need to be improved.

Related papers

• Deep Learning-Based Object Pose Estimation: A Comprehensive Survey [73.74933379151419]
  We discuss the recent advances in deep learning-based object pose estimation. Our survey also covers multiple input data modalities, degrees-of-freedom of output poses, object properties, and downstream tasks.
  arXiv  Detail & Related papers  (2024-05-13T14:44:22Z)
• Object Detectors in the Open Environment: Challenges, Solutions, and Outlook [95.3317059617271]
  The dynamic and intricate nature of the open environment poses novel and formidable challenges to object detectors. This paper aims to conduct a comprehensive review and analysis of object detectors in open environments. We propose a framework that includes four quadrants (i.e., out-of-domain, out-of-category, robust learning, and incremental learning) based on the dimensions of the data / target changes.
  arXiv  Detail & Related papers  (2024-03-24T19:32:39Z)
• Viewpoint Generation using Feature-Based Constrained Spaces for Robot Vision Systems [63.942632088208505]
  This publication outlines the generation of viewpoints as a geometrical problem and introduces a generalized theoretical framework for solving it. A $mathcalC$-space can be understood as the topological space that a viewpoint constraint spans, where the sensor can be positioned for acquiring a feature while fulfilling the regarded constraint. The introduced $mathcalC$-spaces are characterized based on generic domain and viewpoint constraints models to ease the transferability of the present framework to different applications and robot vision systems.
  arXiv  Detail & Related papers  (2023-06-12T08:57:15Z)
• Open Challenges for Monocular Single-shot 6D Object Pose Estimation [15.01623452269803]
  Object pose estimation is a non-trivial task that enables robotic manipulation, bin picking, augmented reality, and scene understanding. Monocular object pose estimation gained considerable momentum with the rise of high-performing deep learning-based solutions. We identify promising research directions in order to help researchers to formulate relevant research ideas and effectively advance the state of the art.
  arXiv  Detail & Related papers  (2023-02-23T07:26:50Z)
• Universal Object Detection with Large Vision Model [79.06618136217142]
  This study focuses on the large-scale, multi-domain universal object detection problem. To address these challenges, we introduce our approach to label handling, hierarchy-aware design, and resource-efficient model training. Our method has demonstrated remarkable performance, securing a prestigious second-place ranking in the object detection track of the Robust Vision Challenge 2022.
  arXiv  Detail & Related papers  (2022-12-19T12:40:13Z)
• Multi-Robot Collaborative Perception with Graph Neural Networks [6.383576104583731]
  We propose a general-purpose Graph Neural Network (GNN) with the main goal to increase, in multi-robot perception tasks. We show that the proposed framework can address multi-view visual perception problems such as monocular depth estimation and semantic segmentation.
  arXiv  Detail & Related papers  (2022-01-05T18:47:07Z)
• MetaGraspNet: A Large-Scale Benchmark Dataset for Vision-driven Robotic Grasping via Physics-based Metaverse Synthesis [78.26022688167133]
  We present a large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis. The proposed dataset contains 100,000 images and 25 different object types. We also propose a new layout-weighted performance metric alongside the dataset for evaluating object detection and segmentation performance.
  arXiv  Detail & Related papers  (2021-12-29T17:23:24Z)
• Object Detection and Pose Estimation from RGB and Depth Data for Real-time, Adaptive Robotic Grasping [0.0]
  We propose a system that performs real-time object detection and pose estimation, for the purpose of dynamic robot grasping. The proposed approach allows the robot to detect the object identity and its actual pose, and then adapt a canonical grasp in order to be used with the new pose. For training, the system defines a canonical grasp by capturing the relative pose of an object with respect to the gripper attached to the robot's wrist. During testing, once a new pose is detected, a canonical grasp for the object is identified and then dynamically adapted by adjusting the robot arm's joint angles.
  arXiv  Detail & Related papers  (2021-01-18T22:22:47Z)
• Reactive Human-to-Robot Handovers of Arbitrary Objects [57.845894608577495]
  We present a vision-based system that enables human-to-robot handovers of unknown objects. Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation. We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects.
  arXiv  Detail & Related papers  (2020-11-17T21:52:22Z)
• Fit to Measure: Reasoning about Sizes for Robust Object Recognition [0.5352699766206808]
  We present an approach to integrating knowledge about object sizes in a ML based architecture. Our experiments in a real world robotic scenario show that this combined approach ensures a significant performance increase over state of the art Machine Learning methods.
  arXiv  Detail & Related papers  (2020-10-27T13:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.