How to Guide Adaptive Depth Sampling?

• URL: http://arxiv.org/abs/2205.10202v1
• Date: Fri, 20 May 2022 14:23:01 GMT
• Title: How to Guide Adaptive Depth Sampling?
• Authors: Ilya Tcenov, Guy Gilboa
• Abstract summary: We show that a neural network can learn to produce a highly faithful Importance Map, given an RGB image. We then suggest an algorithm to produce a sampling pattern for the scene, which is denser in regions that are harder to reconstruct.
• Score: 9.848252984349603
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in depth sensing technologies allow fast electronic maneuvering of the laser beam, as opposed to fixed mechanical rotations. This will enable future sensors, in principle, to vary in real-time the sampling pattern. We examine here the abstract problem of whether adapting the sampling pattern for a given frame can reduce the reconstruction error or allow a sparser pattern. We propose a constructive generic method to guide adaptive depth sampling algorithms. Given a sampling budget B, a depth predictor P and a desired quality measure M, we propose an Importance Map that highlights important sampling locations. This map is defined for a given frame as the per-pixel expected value of M produced by the predictor P, given a pattern of B random samples. This map can be well estimated in a training phase. We show that a neural network can learn to produce a highly faithful Importance Map, given an RGB image. We then suggest an algorithm to produce a sampling pattern for the scene, which is denser in regions that are harder to reconstruct. The sampling strategy of our modular framework can be adjusted according to hardware limitations, type of depth predictor, and any custom reconstruction error measure that should be minimized. We validate through simulations that our approach outperforms grid and random sampling patterns as well as recent state-of-the-art adaptive algorithms.

Related papers

• Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction [98.30014795224432]
  We introduce Fine Structured-Aware Sampling (FSS) to train pixel-aligned implicit models for single-view human reconstruction. FSS proactively adapts to the thickness and complexity of surfaces. It also proposes a mesh thickness loss signal for pixel-aligned implicit models.
  arXiv  Detail & Related papers  (2024-02-29T14:26:46Z)
• Deep Richardson-Lucy Deconvolution for Low-Light Image Deblurring [48.80983873199214]
  We develop a data-driven approach to model the saturated pixels by a learned latent map. Based on the new model, the non-blind deblurring task can be formulated into a maximum a posterior (MAP) problem. To estimate high-quality deblurred images without amplified artifacts, we develop a prior estimation network.
  arXiv  Detail & Related papers  (2023-08-10T12:53:30Z)
• Differentiable Rendering with Reparameterized Volume Sampling [2.717399369766309]
  In view synthesis, a neural radiance field approximates underlying density and radiance fields based on a sparse set of scene pictures. This rendering algorithm is fully differentiable and facilitates gradient-based optimization of the fields. We propose a simple end-to-end differentiable sampling algorithm based on inverse transform sampling.
  arXiv  Detail & Related papers  (2023-02-21T19:56:50Z)
• Super-resolution GANs of randomly-seeded fields [68.8204255655161]
  We propose a novel super-resolution generative adversarial network (GAN) framework to estimate field quantities from random sparse sensors. The algorithm exploits random sampling to provide incomplete views of the high-resolution underlying distributions. The proposed technique is tested on synthetic databases of fluid flow simulations, ocean surface temperature distributions measurements, and particle image velocimetry data.
  arXiv  Detail & Related papers  (2022-02-23T18:57:53Z)
• Parallelised Diffeomorphic Sampling-based Motion Planning [30.310891362316863]
  We propose Parallelised Diffeomorphic Sampling-based Motion Planning (PDMP) PDMP transforms sampling distributions of sampling-based motion planners, in a manner akin to normalising flows. PDMP is able to leverage gradient information of costs, to inject specifications, in a manner similar to optimisation-based motion planning methods.
  arXiv  Detail & Related papers  (2021-08-26T13:15:11Z)
• iNNformant: Boundary Samples as Telltale Watermarks [68.8204255655161]
  We show that it is possible to generate sets of boundary samples which can identify any of four tested microarchitectures. These sets can be built to not contain any sample with a worse peak signal-to-noise ratio than 70dB.
  arXiv  Detail & Related papers  (2021-06-14T11:18:32Z)
• Adaptive Illumination based Depth Sensing using Deep Learning [18.72398843488572]
  Various techniques have been proposed to estimate a dense depth map based on fusion of the sparse depth map measurement with the RGB image. Recent advances in hardware enable adaptive depth measurements resulting in further improvement of the dense depth map estimation. We show that such adaptive sampling masks can generalize well to many RGB and sparse depth fusion algorithms under a variety of sampling rates.
  arXiv  Detail & Related papers  (2021-03-23T04:21:07Z)
• Adaptive LiDAR Sampling and Depth Completion using Ensemble Variance [12.633386045916444]
  This work considers the problem of depth completion, with or without image data, where an algorithm may measure the depth of a prescribed limited number of pixels. The algorithmic challenge is to choose pixel positions strategically and dynamically to maximally reduce overall depth estimation error. This setting is realized in daytime or nighttime depth completion for autonomous vehicles with a programmable LiDAR.
  arXiv  Detail & Related papers  (2020-07-27T19:54:42Z)
• Non-Adaptive Adaptive Sampling on Turnstile Streams [57.619901304728366]
  We give the first relative-error algorithms for column subset selection, subspace approximation, projective clustering, and volume on turnstile streams that use space sublinear in $n$. Our adaptive sampling procedure has a number of applications to various data summarization problems that either improve state-of-the-art or have only been previously studied in the more relaxed row-arrival model.
  arXiv  Detail & Related papers  (2020-04-23T05:00:21Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)
• Learning to Importance Sample in Primary Sample Space [22.98252856114423]
  We propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. We show that our approach leads to effective variance reduction in several practical scenarios.
  arXiv  Detail & Related papers  (2018-08-23T16:55:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.