Deep Non-Line-of-Sight Reconstruction

• URL: http://arxiv.org/abs/2001.09067v2
• Date: Wed, 29 Jan 2020 12:42:53 GMT
• Title: Deep Non-Line-of-Sight Reconstruction
• Authors: Javier Grau Chopite, Matthias B. Hullin, Michael Wand and Julian Iseringhausen
• Abstract summary: In this paper, we employ convolutional feed-forward networks for solving the reconstruction problem efficiently. We devise a tailored autoencoder architecture, trained end-to-end reconstruction maps transient images directly to a depth map representation. We demonstrate that our feed-forward network, even though it is trained solely on synthetic data, generalizes to measured data from SPAD sensors and is able to obtain results that are competitive with model-based reconstruction methods.
• Score: 18.38481917675749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent years have seen a surge of interest in methods for imaging beyond the direct line of sight. The most prominent techniques rely on time-resolved optical impulse responses, obtained by illuminating a diffuse wall with an ultrashort light pulse and observing multi-bounce indirect reflections with an ultrafast time-resolved imager. Reconstruction of geometry from such data, however, is a complex non-linear inverse problem that comes with substantial computational demands. In this paper, we employ convolutional feed-forward networks for solving the reconstruction problem efficiently while maintaining good reconstruction quality. Specifically, we devise a tailored autoencoder architecture, trained end-to-end, that maps transient images directly to a depth map representation. Training is done using an efficient transient renderer for diffuse three-bounce indirect light transport that enables the quick generation of large amounts of training data for the network. We examine the performance of our method on a variety of synthetic and experimental datasets and its dependency on the choice of training data and augmentation strategies, as well as architectural features. We demonstrate that our feed-forward network, even though it is trained solely on synthetic data, generalizes to measured data from SPAD sensors and is able to obtain results that are competitive with model-based reconstruction methods.

Related papers

• Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction [48.30341580103962]
  We propose a novel re-visible dual-domain self-supervised deep unfolding network to address these issues. We design a deep unfolding network based on Chambolle and Pock Proximal Point Algorithm (DUN-CP-PPA) to achieve end-to-end reconstruction. Experiments conducted on the fastMRI and IXI datasets demonstrate that our method significantly outperforms state-of-the-art approaches in terms of reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-07T12:29:32Z)
• Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations [37.42624848693373]
  We introduce a method that directly digests depth images for the task of high-fidelity 3D reconstruction. A simple sampling strategy is proposed to generate highly effective training data. Despite its simplicity, our method outperforms a range of both classical and learning-based baselines.
  arXiv  Detail & Related papers  (2023-06-03T12:23:17Z)
• A Deep Learning Approach for SAR Tomographic Imaging of Forested Areas [10.477070348391079]
  We show that light-weight neural networks can be trained to perform the tomographic inversion with a single feed-forward pass. We train our encoder-decoder network using simulated data and validate our technique on real L-band and P-band data.
  arXiv  Detail & Related papers  (2023-01-20T14:34:03Z)
• A Projection-Based K-space Transformer Network for Undersampled Radial MRI Reconstruction with Limited Training Subjects [1.5708535232255898]
  Non-Cartesian trajectories need to be transformed onto a Cartesian grid in each iteration of the network training. We propose novel data augmentation methods to generate a large amount of training data from a limited number of subjects. Experimental results show superior performance of the proposed framework compared to state-of-the-art deep neural networks.
  arXiv  Detail & Related papers  (2022-06-15T00:20:22Z)
• Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging [13.271762773872476]
  We present a novel deep model that incorporates the complementary physics priors of wave propagation and volume rendering into a neural network for high-quality and robust NLOS reconstruction. Our method outperforms prior physics and learning based approaches on both synthetic and real measurements.
  arXiv  Detail & Related papers  (2022-05-03T02:47:02Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)
• Aerial Images Meet Crowdsourced Trajectories: A New Approach to Robust Road Extraction [110.61383502442598]
  We introduce a novel neural network framework termed Cross-Modal Message Propagation Network (CMMPNet) CMMPNet is composed of two deep Auto-Encoders for modality-specific representation learning and a tailor-designed Dual Enhancement Module for cross-modal representation refinement. Experiments on three real-world benchmarks demonstrate the effectiveness of our CMMPNet for robust road extraction.
  arXiv  Detail & Related papers  (2021-11-30T04:30:10Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation [47.984368369734995]
  We introduce a novel recurrent encoding-decoding neural network architecture for event-based optical flow estimation. The network is end-to-end trained with self-supervised learning on the Multi-Vehicle Stereo Event Camera dataset. We have shown that it outperforms all the existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-09-10T13:37:37Z)
• ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation [3.964047152162558]
  This paper presents an innovative deep learning pipeline which estimates the relative pose of a spacecraft by incorporating the temporal information from a rendezvous sequence. It leverages the performance of long short-term memory (LSTM) units in modelling sequences of data for the processing of features extracted by a convolutional neural network (CNN) backbone. Three distinct training strategies, which follow a coarse-to-fine funnelled approach, are combined to facilitate feature learning and improve end-to-end pose estimation by regression.
  arXiv  Detail & Related papers  (2021-08-23T16:48:58Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.