Dense Reconstruction of Transparent Objects by Altering Incident Light Paths Through Refraction

• URL: http://arxiv.org/abs/2105.09993v1
• Date: Thu, 20 May 2021 19:01:12 GMT
• Title: Dense Reconstruction of Transparent Objects by Altering Incident Light Paths Through Refraction
• Authors: Kai Han and Kwan-Yee K. Wong and Miaomiao Liu
• Abstract summary: We introduce a fixed viewpoint approach to dense surface reconstruction of transparent objects based on refraction of light. We present a setup that allows us to alter the incident light paths before light rays enter the object by immersing the object partially in a liquid.
• Score: 40.696591594772876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses the problem of reconstructing the surface shape of transparent objects. The difficulty of this problem originates from the viewpoint dependent appearance of a transparent object, which quickly makes reconstruction methods tailored for diffuse surfaces fail disgracefully. In this paper, we introduce a fixed viewpoint approach to dense surface reconstruction of transparent objects based on refraction of light. We present a simple setup that allows us to alter the incident light paths before light rays enter the object by immersing the object partially in a liquid, and develop a method for recovering the object surface through reconstructing and triangulating such incident light paths. Our proposed approach does not need to model the complex interactions of light as it travels through the object, neither does it assume any parametric form for the object shape nor the exact number of refractions and reflections taken place along the light paths. It can therefore handle transparent objects with a relatively complex shape and structure, with unknown and inhomogeneous refractive index. We also show that for thin transparent objects, our proposed acquisition setup can be further simplified by adopting a single refraction approximation. Experimental results on both synthetic and real data demonstrate the feasibility and accuracy of our proposed approach.

Related papers

• Neural Radiance Fields for Transparent Object Using Visual Hull [0.8158530638728501]
  Recently introduced Neural Radiance Fields (NeRF) is a view synthesis method. We propose a NeRF-based method consisting of the following three steps: First, we reconstruct a three-dimensional shape of a transparent object using visual hull. Second, we simulate the refraction of the rays inside of the transparent object according to Snell's law. Last, we sample points through refracted rays and put them into NeRF.
  arXiv  Detail & Related papers  (2023-12-13T13:15:19Z)
• Towards Monocular Shape from Refraction [23.60349429048409]
  We show that a simple energy function based on Snell's law enables the reconstruction of an arbitrary refractive surface geometry. We show that solving for an entire surface at once introduces implicit parameter-free spatial regularization.
  arXiv  Detail & Related papers  (2023-05-31T11:09:37Z)
• Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container [61.50401406132946]
  Transparent enclosures pose challenges of multiple light reflections and refractions at the interface between different propagation media. We use an existing neural reconstruction method (NeuS) that implicitly represents the geometry and appearance of the inner subspace. In order to account for complex light interactions, we develop a hybrid rendering strategy that combines volume rendering with ray tracing.
  arXiv  Detail & Related papers  (2023-03-24T04:58:27Z)
• NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects [28.62468618676557]
  We propose NEMTO, the first end-to-end neural rendering pipeline to model 3D transparent objects. With 2D images of the transparent object as input, our method is capable of high-quality novel view and relighting synthesis.
  arXiv  Detail & Related papers  (2023-03-21T15:50:08Z)
• NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing [44.22576861939435]
  We present a novel method, called NeTO, for capturing 3D geometry of solid transparent objects from 2D images via volume rendering. Our method achieves faithful reconstruction results and outperforms prior works by a large margin.
  arXiv  Detail & Related papers  (2023-03-20T15:50:00Z)
• Self-calibrating Photometric Stereo by Neural Inverse Rendering [88.67603644930466]
  This paper tackles the task of uncalibrated photometric stereo for 3D object reconstruction. We propose a new method that jointly optimize object shape, light directions, and light intensities. Our method demonstrates state-of-the-art accuracy in light estimation and shape recovery on real-world datasets.
  arXiv  Detail & Related papers  (2022-07-16T02:46:15Z)
• Neural Reflectance for Shape Recovery with Shadow Handling [88.67603644930466]
  This paper aims at recovering the shape of a scene with unknown, non-Lambertian, and possibly spatially-varying surface materials. We propose a coordinate-based deep reflectance (multilayer perceptron) to parameterize both the unknown 3D shape and the unknown at every surface point. This network is able to leverage the observed photometric variance and shadows on the surface, and recover both surface shape and general non-Lambertian reflectance.
  arXiv  Detail & Related papers  (2022-03-24T07:57:20Z)
• Multi-view 3D Reconstruction of a Texture-less Smooth Surface of Unknown Generic Reflectance [86.05191217004415]
  Multi-view reconstruction of texture-less objects with unknown surface reflectance is a challenging task. This paper proposes a simple and robust solution to this problem based on a co-light scanner.
  arXiv  Detail & Related papers  (2021-05-25T01:28:54Z)
• Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes [75.63464905190061]
  Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this problem. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images.
  arXiv  Detail & Related papers  (2020-04-22T23:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.