Polarimetric iToF: Measuring High-Fidelity Depth through Scattering Media

• URL: http://arxiv.org/abs/2306.17618v1
• Date: Fri, 30 Jun 2023 12:42:40 GMT
• Title: Polarimetric iToF: Measuring High-Fidelity Depth through Scattering Media
• Authors: Daniel S. Jeon, Andreas Meuleman, Seung-Hwan Baek, Min H. Kim
• Abstract summary: Indirect time-of-flight (iToF) imaging allows us to capture dense depth information at a low cost. iToF imaging often suffers from multipath interference (MPI) artifacts in the presence of scattering media. We propose a polarimetric iToF imaging method that can capture depth information robustly through scattering media.
• Score: 35.78671232637732
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Indirect time-of-flight (iToF) imaging allows us to capture dense depth information at a low cost. However, iToF imaging often suffers from multipath interference (MPI) artifacts in the presence of scattering media, resulting in severe depth-accuracy degradation. For instance, iToF cameras cannot measure depth accurately through fog because ToF active illumination scatters back to the sensor before reaching the farther target surface. In this work, we propose a polarimetric iToF imaging method that can capture depth information robustly through scattering media. Our observations on the principle of indirect ToF imaging and polarization of light allow us to formulate a novel computational model of scattering-aware polarimetric phase measurements that enables us to correct MPI errors. We first devise a scattering-aware polarimetric iToF model that can estimate the phase of unpolarized backscattered light. We then combine the optical filtering of polarization and our computational modeling of unpolarized backscattered light via scattering analysis of phase and amplitude. This allows us to tackle the MPI problem by estimating the scattering energy through the participating media. We validate our method on an experimental setup using a customized off-the-shelf iToF camera. Our method outperforms baseline methods by a significant margin by means of our scattering model and polarimetric phase measurements.

Related papers

• Physically-Based Photometric Bundle Adjustment in Non-Lambertian Environments [59.96101889715997]
  Photometric bundle adjustment (PBA) is widely used in estimating the camera pose and 3D geometry by assuming a Lambertian world. The assumption of photometric consistency is often violated since the non-diffuse reflection is common in real-world environments. We propose a novel physically-based PBA method to solve this problem.
  arXiv  Detail & Related papers  (2024-09-18T10:22:07Z)
• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation [8.78717459496649]
  We propose FlowDepth, where a Dynamic Motion Flow Module (DMFM) decouples the optical flow by a mechanism-based approach and warps the dynamic regions thus solving the mismatch problem. For the unfairness of photometric errors caused by high-freq and low-texture regions, we use Depth-Cue-Aware Blur (DCABlur) and Cost-Volume sparsity loss respectively at the input and the loss level to solve the problem.
  arXiv  Detail & Related papers  (2024-03-28T10:31:23Z)
• NeISF: Neural Incident Stokes Field for Geometry and Material Estimation [50.588983686271284]
  Multi-view inverse rendering is the problem of estimating the scene parameters such as shapes, materials, or illuminations from a sequence of images captured under different viewpoints. We propose Neural Incident Stokes Fields (NeISF), a multi-view inverse framework that reduces ambiguities using polarization cues.
  arXiv  Detail & Related papers  (2023-11-22T06:28:30Z)
• Microseismic source imaging using physics-informed neural networks with hard constraints [4.07926531936425]
  We propose a direct microseismic imaging framework based on physics-informed neural networks (PINNs) We use the PINNs to represent a multi-frequency wavefield and then apply inverse Fourier transform to extract the source image. We further apply our method to hydraulic fracturing monitoring field data, and demonstrate that our method can correctly image the source with fewer artifacts.
  arXiv  Detail & Related papers  (2023-04-09T21:10:39Z)
• All-photon Polarimetric Time-of-Flight Imaging [33.499684969102816]
  Time-of-flight (ToF) sensors provide an imaging modality fueling diverse applications, including LiDAR in autonomous driving. Conventional ToF imaging methods estimate the depth by sending pulses of light into a scene and measuring the ToF of the first-arriving photons. We propose an all-photon ToF imaging method by incorporating the temporal-polarimetric analysis of first- and late-arriving photons.
  arXiv  Detail & Related papers  (2021-12-17T01:51:47Z)
• Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments [56.306567220448684]
  We propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image. We show more than 40% RMSE improvement on the final depth map compared to the baseline approach.
  arXiv  Detail & Related papers  (2021-12-07T15:04:14Z)
• A learning-based view extrapolation method for axial super-resolution [52.748944517480155]
  Axial light field resolution refers to the ability to distinguish features at different depths by refocusing. We propose a learning-based method to extrapolate novel views from axial volumes of sheared epipolar plane images.
  arXiv  Detail & Related papers  (2021-03-11T07:22:13Z)
• Attaining quantum limited precision of localizing an object in passive imaging [1.8237412774861765]
  We investigate the ability to determine the mean position, or centroid, of a linear array of equally-bright incoherent point sources of light. We consider two receivers: an image-plane ideal direct-detection imager and a receiver that employs Hermite-Gaussian (HG) Spatial-mode Demultiplexing (SPADE) in the image plane.
  arXiv  Detail & Related papers  (2021-02-03T19:00:15Z)
• Monochrome and Color Polarization Demosaicking Using Edge-Aware Residual Interpolation [14.5106375775521]
  A microgrid image polarimeter enables us to acquire a set of polarization images in one shot. Since the polarimeter consists of an image sensor equipped with a monochrome or color polarization filter array, the demosaicking process to interpolate missing pixel values plays a crucial role in obtaining high-quality polarization images. We propose a novel MPFA demosaicking method based on edge-aware residual (EARI) and also extend it to CPFA demosaicking.
  arXiv  Detail & Related papers  (2020-07-28T15:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.