Polarimetric Inverse Rendering for Transparent Shapes Reconstruction

• URL: http://arxiv.org/abs/2208.11836v1
• Date: Thu, 25 Aug 2022 02:52:31 GMT
• Title: Polarimetric Inverse Rendering for Transparent Shapes Reconstruction
• Authors: Mingqi Shao, Chongkun Xia, Dongxu Duan, Xueqian Wang
• Abstract summary: We propose a novel method for the detailed reconstruction of transparent objects by exploiting polarimetric cues. We implicitly represent the object's geometry as a neural network, while the polarization render is capable of rendering the object's polarization images. We build a polarization dataset for multi-view transparent shapes reconstruction to verify our method.
• Score: 1.807492010338763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we propose a novel method for the detailed reconstruction of transparent objects by exploiting polarimetric cues. Most of the existing methods usually lack sufficient constraints and suffer from the over-smooth problem. Hence, we introduce polarization information as a complementary cue. We implicitly represent the object's geometry as a neural network, while the polarization render is capable of rendering the object's polarization images from the given shape and illumination configuration. Direct comparison of the rendered polarization images to the real-world captured images will have additional errors due to the transmission in the transparent object. To address this issue, the concept of reflection percentage which represents the proportion of the reflection component is introduced. The reflection percentage is calculated by a ray tracer and then used for weighting the polarization loss. We build a polarization dataset for multi-view transparent shapes reconstruction to verify our method. The experimental results show that our method is capable of recovering detailed shapes and improving the reconstruction quality of transparent objects. Our dataset and code will be publicly available at https://github.com/shaomq2187/TransPIR.

Related papers

• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• SPIDeRS: Structured Polarization for Invisible Depth and Reflectance Sensing [31.605927493154656]
  We introduce structured polarization for invisible depth and reflectance sensing (SPIDeRS) The key idea is to modulate the angle of linear polarization (AoLP) of projected light at each pixel. The use of polarization makes it invisible and lets us recover not only depth but also directly surface normals and even reflectance.
  arXiv  Detail & Related papers  (2023-12-07T18:59:21Z)
• Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container [61.50401406132946]
  Transparent enclosures pose challenges of multiple light reflections and refractions at the interface between different propagation media. We use an existing neural reconstruction method (NeuS) that implicitly represents the geometry and appearance of the inner subspace. In order to account for complex light interactions, we develop a hybrid rendering strategy that combines volume rendering with ray tracing.
  arXiv  Detail & Related papers  (2023-03-24T04:58:27Z)
• Self-calibrating Photometric Stereo by Neural Inverse Rendering [88.67603644930466]
  This paper tackles the task of uncalibrated photometric stereo for 3D object reconstruction. We propose a new method that jointly optimize object shape, light directions, and light intensities. Our method demonstrates state-of-the-art accuracy in light estimation and shape recovery on real-world datasets.
  arXiv  Detail & Related papers  (2022-07-16T02:46:15Z)
• Transparent Shape from a Single View Polarization Image [6.18278691318801]
  This paper presents a learning-based method for transparent surface estimation from a single view polarization image. Existing shape from polarization(SfP) methods have the difficulty in estimating transparent shape since the inherent transmission interference heavily reduces the reliability of physics-based prior.
  arXiv  Detail & Related papers  (2022-04-13T12:24:32Z)
• PANDORA: Polarization-Aided Neural Decomposition Of Radiance [20.760987175553655]
  Inverse rendering is a fundamental problem in computer graphics and vision. Recent progress in representing scene properties as coordinate-based neural networks have facilitated neural inverse rendering. We propose PANDORA, a polarimetric inverse rendering approach based on implicit neural representations.
  arXiv  Detail & Related papers  (2022-03-25T05:41:52Z)
• Deep Polarization Imaging for 3D shape and SVBRDF Acquisition [7.86578678811226]
  We present a novel method for efficient acquisition of shape and spatially varying reflectance of 3D objects using polarization cues. Unlike previous works that have exploited polarization to estimate material or object appearance under certain constraints, we lift such restrictions by coupling polarization imaging with deep learning. We demonstrate our approach to achieve superior results compared to recent works employing deep learning in conjunction with flash illumination.
  arXiv  Detail & Related papers  (2021-05-06T17:58:43Z)
• Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
  Differentiable geometric computation has received increasing interest for image-based inverse problems. We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation. As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
  arXiv  Detail & Related papers  (2021-04-01T09:29:05Z)
• Polarimetric Monocular Dense Mapping Using Relative Deep Depth Prior [8.552832023331248]
  We propose an online reconstruction method that uses full polarimetric cues available from the polarization camera. Our method is able to significantly improve the accuracy of the depthmap as well as increase its density, specially in regions of poor texture.
  arXiv  Detail & Related papers  (2021-02-10T01:34:37Z)
• Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes [75.63464905190061]
  Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this problem. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images.
  arXiv  Detail & Related papers  (2020-04-22T23:51:30Z)
• Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
  We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
  arXiv  Detail & Related papers  (2020-03-28T13:29:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.