Transparent Shape from a Single View Polarization Image

• URL: http://arxiv.org/abs/2204.06331v6
• Date: Sat, 12 Aug 2023 11:52:29 GMT
• Title: Transparent Shape from a Single View Polarization Image
• Authors: Mingqi Shao, Chongkun Xia, Zhendong Yang, Junnan Huang, Xueqian Wang
• Abstract summary: This paper presents a learning-based method for transparent surface estimation from a single view polarization image. Existing shape from polarization(SfP) methods have the difficulty in estimating transparent shape since the inherent transmission interference heavily reduces the reliability of physics-based prior.
• Score: 6.18278691318801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a learning-based method for transparent surface estimation from a single view polarization image. Existing shape from polarization(SfP) methods have the difficulty in estimating transparent shape since the inherent transmission interference heavily reduces the reliability of physics-based prior. To address this challenge, we propose the concept of physics-based prior, which is inspired by the characteristic that the transmission component in the polarization image has more noise than reflection. The confidence is used to determine the contribution of the interfered physics-based prior. Then, we build a network(TransSfP) with multi-branch architecture to avoid the destruction of relationships between different hierarchical inputs. To train and test our method, we construct a dataset for transparent shape from polarization with paired polarization images and ground-truth normal maps. Extensive experiments and comparisons demonstrate the superior accuracy of our method.

Related papers

• SS-SfP:Neural Inverse Rendering for Self Supervised Shape from (Mixed) Polarization [21.377923666134116]
  Shape from Polarization (SfP) is the problem popularly known as Shape from Polarization (SfP) We present a novel inverse rendering-based framework to estimate the 3D shape (per-pixel surface normals and depth) of objects and scenes from single-view polarization images.
  arXiv  Detail & Related papers  (2024-07-12T14:29:00Z)
• Video Frame Interpolation for Polarization via Swin-Transformer [9.10220649654041]
  Video Frame Interpolation (VFI) has been extensively explored and demonstrated, yet its application to polarization remains largely unexplored. This study proposes a multi-stage and multi-scale network called Swin-VFI based on the Swin-Transformer. Experimental results demonstrate our approach's superior reconstruction accuracy across all tasks.
  arXiv  Detail & Related papers  (2024-06-17T09:48:54Z)
• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• S2P3: Self-Supervised Polarimetric Pose Prediction [55.43547228561919]
  This paper proposes the first self-supervised 6D object pose prediction from multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable constraints.
  arXiv  Detail & Related papers  (2023-12-02T10:46:40Z)
• NeISF: Neural Incident Stokes Field for Geometry and Material Estimation [50.588983686271284]
  Multi-view inverse rendering is the problem of estimating the scene parameters such as shapes, materials, or illuminations from a sequence of images captured under different viewpoints. We propose Neural Incident Stokes Fields (NeISF), a multi-view inverse framework that reduces ambiguities using polarization cues.
  arXiv  Detail & Related papers  (2023-11-22T06:28:30Z)
• PARTNER: Level up the Polar Representation for LiDAR 3D Object Detection [81.16859686137435]
  We present PARTNER, a novel 3D object detector in the polar coordinate. Our method outperforms the previous polar-based works with remarkable margins of 3.68% and 9.15% on and ONCE validation set.
  arXiv  Detail & Related papers  (2023-08-08T01:59:20Z)
• Polarimetric Inverse Rendering for Transparent Shapes Reconstruction [1.807492010338763]
  We propose a novel method for the detailed reconstruction of transparent objects by exploiting polarimetric cues. We implicitly represent the object's geometry as a neural network, while the polarization render is capable of rendering the object's polarization images. We build a polarization dataset for multi-view transparent shapes reconstruction to verify our method.
  arXiv  Detail & Related papers  (2022-08-25T02:52:31Z)
• Deep Polarization Imaging for 3D shape and SVBRDF Acquisition [7.86578678811226]
  We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints, we lift such restrictions by coupling polarization imaging with deep learning. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.
  arXiv  Detail & Related papers  (2021-05-06T17:58:43Z)
• Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces [103.08512487830669]
  This paper presents an uncalibrated deep neural network framework for the photometric stereo problem. Existing neural network-based methods either require exact light directions or ground-truth surface normals of the object or both. We propose an uncalibrated neural inverse rendering approach to this problem.
  arXiv  Detail & Related papers  (2020-12-12T10:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.