Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data

• URL: http://arxiv.org/abs/2308.10627v1
• Date: Mon, 21 Aug 2023 10:56:00 GMT
• Title: Polarimetric Information for Multi-Modal 6D Pose Estimation of Photometrically Challenging Objects with Limited Data
• Authors: Patrick Ruhkamp, Daoyi Gao, HyunJun Jung, Nassir Navab, Benjamin Busam
• Abstract summary: 6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects. A supervised learning-based method utilising complementary polarisation information is proposed to overcome such limitations.
• Score: 51.95347650131366
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects with e.g. textureless surfaces, reflections or transparency. A supervised learning-based method utilising complementary polarisation information as input modality is proposed to overcome such limitations. This supervised approach is then extended to a self-supervised paradigm by leveraging physical characteristics of polarised light, thus eliminating the need for annotated real data. The methods achieve significant advancements in pose estimation by leveraging geometric information from polarised light and incorporating shape priors and invertible physical constraints.

Related papers

• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• SS-SfP:Neural Inverse Rendering for Self Supervised Shape from (Mixed) Polarization [21.377923666134116]
  Shape from Polarization (SfP) is the problem popularly known as Shape from Polarization (SfP) We present a novel inverse rendering-based framework to estimate the 3D shape (per-pixel surface normals and depth) of objects and scenes from single-view polarization images.
  arXiv  Detail & Related papers  (2024-07-12T14:29:00Z)
• S2P3: Self-Supervised Polarimetric Pose Prediction [55.43547228561919]
  This paper proposes the first self-supervised 6D object pose prediction from multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable constraints.
  arXiv  Detail & Related papers  (2023-12-02T10:46:40Z)
• NeISF: Neural Incident Stokes Field for Geometry and Material Estimation [50.588983686271284]
  Multi-view inverse rendering is the problem of estimating the scene parameters such as shapes, materials, or illuminations from a sequence of images captured under different viewpoints. We propose Neural Incident Stokes Fields (NeISF), a multi-view inverse framework that reduces ambiguities using polarization cues.
  arXiv  Detail & Related papers  (2023-11-22T06:28:30Z)
• Geo6D: Geometric Constraints Learning for 6D Pose Estimation [21.080439293774464]
  We propose a novel geometric constraints learning approach called Geo6D for direct regression 6D pose estimation methods. We show that when equipped with Geo6D, the direct 6D methods achieve state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2022-10-20T02:00:58Z)
• Polarimetric Pose Prediction [42.47531308682873]
  Colour-band separated wavelength and intensity are arguably the most commonly used ones for monocular 6D object pose estimation. This paper explores how complementary polarisation information, i.e. the orientation of light wave oscillations, can influence the accuracy of pose predictions.
  arXiv  Detail & Related papers  (2021-12-07T16:38:10Z)
• Spatial Attention Improves Iterative 6D Object Pose Estimation [52.365075652976735]
  We propose a new method for 6D pose estimation refinement from RGB images. Our main insight is that after the initial pose estimate, it is important to pay attention to distinct spatial features of the object. We experimentally show that this approach learns to attend to salient spatial features and learns to ignore occluded parts of the object, leading to better pose estimation across datasets.
  arXiv  Detail & Related papers  (2021-01-05T17:18:52Z)
• Robust RGB-based 6-DoF Pose Estimation without Real Pose Annotations [92.5075742765229]
  We introduce an approach to robustly and accurately estimate the 6-DoF pose in challenging conditions without using any real pose annotations. We achieve state of the art performance on LINEMOD, and OccludedLINEMOD in without real-pose setting, even outperforming methods that rely on real annotations during training on Occluded-LINEMOD.
  arXiv  Detail & Related papers  (2020-08-19T12:07:01Z)
• P2D: a self-supervised method for depth estimation from polarimetry [0.7046417074932255]
  We propose exploiting polarization cues to encourage accurate reconstruction of scenes. Our method is evaluated both qualitatively and quantitatively demonstrating that the contribution of this new information as well as an enhanced loss function improves depth estimation results.
  arXiv  Detail & Related papers  (2020-07-15T09:32:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.