Polarimetric Pose Prediction
- URL: http://arxiv.org/abs/2112.03810v1
- Date: Tue, 7 Dec 2021 16:38:10 GMT
- Title: Polarimetric Pose Prediction
- Authors: Daoyi Gao, Yitong Li, Patrick Ruhkamp, Iuliia Skobleva, Magdalena
Wysock, HyunJun Jung, Pengyuan Wang, Arturo Guridi, Nassir Navab, Benjamin
Busam
- Abstract summary: Colour-band separated wavelength and intensity are arguably the most commonly used ones for monocular 6D object pose estimation.
This paper explores how complementary polarisation information, i.e. the orientation of light wave oscillations, can influence the accuracy of pose predictions.
- Score: 42.47531308682873
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Light has many properties that can be passively measured by vision sensors.
Colour-band separated wavelength and intensity are arguably the most commonly
used ones for monocular 6D object pose estimation. This paper explores how
complementary polarisation information, i.e. the orientation of light wave
oscillations, can influence the accuracy of pose predictions. A hybrid model
that leverages physical priors jointly with a data-driven learning strategy is
designed and carefully tested on objects with different amount of photometric
complexity. Our design not only significantly improves the pose accuracy in
relation to photometric state-of-the-art approaches, but also enables object
pose estimation for highly reflective and transparent objects.
Related papers
- Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors.
Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors.
To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
arXiv Detail & Related papers (2024-04-05T17:55:33Z) - S2P3: Self-Supervised Polarimetric Pose Prediction [55.43547228561919]
This paper proposes the first self-supervised 6D object pose prediction from multimodal RGB+polarimetric images.
The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable constraints.
arXiv Detail & Related papers (2023-12-02T10:46:40Z) - Polarimetric Information for Multi-Modal 6D Pose Estimation of
Photometrically Challenging Objects with Limited Data [51.95347650131366]
6D pose estimation pipelines that rely on RGB-only or RGB-D data show limitations for photometrically challenging objects.
A supervised learning-based method utilising complementary polarisation information is proposed to overcome such limitations.
arXiv Detail & Related papers (2023-08-21T10:56:00Z) - A Geometric Model for Polarization Imaging on Projective Cameras [5.381004207943598]
We present a geometric model describing how a general projective camera captures the light polarization state.
Our model is implemented as a pre-processing operation acting on raw images, followed by a per-pixel rotation of the reconstructed normal field.
Experiments on existing and new datasets demonstrate the accuracy of the model when applied to commercially available polarimetric cameras.
arXiv Detail & Related papers (2022-11-29T17:12:26Z) - Sparse Ellipsometry: Portable Acquisition of Polarimetric SVBRDF and
Shape with Unstructured Flash Photography [32.68190169944569]
We present a portable polarimetric acquisition method that captures both polarimetric SVBRDF and 3D shape simultaneously.
Instead of days, the total acquisition time varies between twenty and thirty minutes per object.
Our results show a strong agreement with a recent ground-truth dataset of captured polarimetric BRDFs of real-world objects.
arXiv Detail & Related papers (2022-07-09T09:42:59Z) - DeepRM: Deep Recurrent Matching for 6D Pose Refinement [77.34726150561087]
DeepRM is a novel recurrent network architecture for 6D pose refinement.
The architecture incorporates LSTM units to propagate information through each refinement step.
DeepRM achieves state-of-the-art performance on two widely accepted challenging datasets.
arXiv Detail & Related papers (2022-05-28T16:18:08Z) - Pose Estimation of Specular and Symmetrical Objects [0.719973338079758]
In the robotic industry, specular and textureless metallic components are ubiquitous.
The 6D pose estimation of such objects with only a monocular RGB camera is difficult because of the absence of rich texture features.
This paper proposes a data-driven solution to estimate the 6D pose of specular objects for grasping them.
arXiv Detail & Related papers (2020-10-31T22:08:46Z) - Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset
for Spatially Varying Isotropic Materials [65.95928593628128]
We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique.
Our algorithm is suitable for perspective cameras and nearby point light sources.
arXiv Detail & Related papers (2020-01-18T12:26:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.