Deep Non-Line-of-Sight Reconstruction

• URL: http://arxiv.org/abs/2001.09067v2
• Date: Wed, 29 Jan 2020 12:42:53 GMT
• Title: Deep Non-Line-of-Sight Reconstruction
• Authors: Javier Grau Chopite, Matthias B. Hullin, Michael Wand and Julian Iseringhausen
• Abstract summary: In this paper, we employ convolutional feed-forward networks for solving the reconstruction problem efficiently. We devise a tailored autoencoder architecture, trained end-to-end reconstruction maps transient images directly to a depth map representation. We demonstrate that our feed-forward network, even though it is trained solely on synthetic data, generalizes to measured data from SPAD sensors and is able to obtain results that are competitive with model-based reconstruction methods.
• Score: 18.38481917675749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent years have seen a surge of interest in methods for imaging beyond the direct line of sight. The most prominent techniques rely on time-resolved optical impulse responses, obtained by illuminating a diffuse wall with an ultrashort light pulse and observing multi-bounce indirect reflections with an ultrafast time-resolved imager. Reconstruction of geometry from such data, however, is a complex non-linear inverse problem that comes with substantial computational demands. In this paper, we employ convolutional feed-forward networks for solving the reconstruction problem efficiently while maintaining good reconstruction quality. Specifically, we devise a tailored autoencoder architecture, trained end-to-end, that maps transient images directly to a depth map representation. Training is done using an efficient transient renderer for diffuse three-bounce indirect light transport that enables the quick generation of large amounts of training data for the network. We examine the performance of our method on a variety of synthetic and experimental datasets and its dependency on the choice of training data and augmentation strategies, as well as architectural features. We demonstrate that our feed-forward network, even though it is trained solely on synthetic data, generalizes to measured data from SPAD sensors and is able to obtain results that are competitive with model-based reconstruction methods.

Related papers

• Sparse Multi-baseline SAR Cross-modal 3D Reconstruction of Vehicle Targets [5.6680936716261705]
  We propose a Cross-Modal Reconstruction Network (CMR-Net), which integrates differentiable render and cross-modal supervision with optical images. CMR-Net, trained solely on simulated data, demonstrates high-resolution reconstruction capabilities on both publicly available simulation datasets and real measured datasets.
  arXiv  Detail & Related papers  (2024-06-06T15:18:59Z)
• Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations [37.42624848693373]
  We introduce a method that directly digests depth images for the task of high-fidelity 3D reconstruction. A simple sampling strategy is proposed to generate highly effective training data. Despite its simplicity, our method outperforms a range of both classical and learning-based baselines.
  arXiv  Detail & Related papers  (2023-06-03T12:23:17Z)
• A Deep Learning Approach for SAR Tomographic Imaging of Forested Areas [10.477070348391079]
  We show that light-weight neural networks can be trained to perform the tomographic inversion with a single feed-forward pass. We train our encoder-decoder network using simulated data and validate our technique on real L-band and P-band data.
  arXiv  Detail & Related papers  (2023-01-20T14:34:03Z)
• A Projection-Based K-space Transformer Network for Undersampled Radial MRI Reconstruction with Limited Training Subjects [1.5708535232255898]
  Non-Cartesian trajectories need to be transformed onto a Cartesian grid in each iteration of the network training. We propose novel data augmentation methods to generate a large amount of training data from a limited number of subjects. Experimental results show superior performance of the proposed framework compared to state-of-the-art deep neural networks.
  arXiv  Detail & Related papers  (2022-06-15T00:20:22Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging [13.271762773872476]
  We present a novel deep model that incorporates the complementary physics priors of wave propagation and volume rendering into a neural network for high-quality and robust NLOS reconstruction. Our method outperforms prior physics and learning based approaches on both synthetic and real measurements.
  arXiv  Detail & Related papers  (2022-05-03T02:47:02Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)
• Aerial Images Meet Crowdsourced Trajectories: A New Approach to Robust Road Extraction [110.61383502442598]
  We introduce a novel neural network framework termed Cross-Modal Message Propagation Network (CMMPNet) CMMPNet is composed of two deep Auto-Encoders for modality-specific representation learning and a tailor-designed Dual Enhancement Module for cross-modal representation refinement. Experiments on three real-world benchmarks demonstrate the effectiveness of our CMMPNet for robust road extraction.
  arXiv  Detail & Related papers  (2021-11-30T04:30:10Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation [47.984368369734995]
  We introduce a novel recurrent encoding-decoding neural network architecture for event-based optical flow estimation. The network is end-to-end trained with self-supervised learning on the Multi-Vehicle Stereo Event Camera dataset. We have shown that it outperforms all the existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-09-10T13:37:37Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.