Learning based Deep Disentangling Light Field Reconstruction and Disparity Estimation Application

• URL: http://arxiv.org/abs/2311.08129v1
• Date: Tue, 14 Nov 2023 12:48:17 GMT
• Title: Learning based Deep Disentangling Light Field Reconstruction and Disparity Estimation Application
• Authors: Langqing Shi, Ping Zhou
• Abstract summary: We propose a Deep Disentangling Mechanism, which inherits the principle of the light field disentangling mechanism and adds advanced network structure. We design a light-field reconstruction network (i.e., DDASR) on the basis of the Deep Disentangling Mechanism, and achieve SOTA performance in the experiments.
• Score: 1.5603081929496316
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Light field cameras have a wide range of uses due to their ability to simultaneously record light intensity and direction. The angular resolution of light fields is important for downstream tasks such as depth estimation, yet is often difficult to improve due to hardware limitations. Conventional methods tend to perform poorly against the challenge of large disparity in sparse light fields, while general CNNs have difficulty extracting spatial and angular features coupled together in 4D light fields. The light field disentangling mechanism transforms the 4D light field into 2D image format, which is more favorable for CNN for feature extraction. In this paper, we propose a Deep Disentangling Mechanism, which inherits the principle of the light field disentangling mechanism and further develops the design of the feature extractor and adds advanced network structure. We design a light-field reconstruction network (i.e., DDASR) on the basis of the Deep Disentangling Mechanism, and achieve SOTA performance in the experiments. In addition, we design a Block Traversal Angular Super-Resolution Strategy for the practical application of depth estimation enhancement where the input views is often higher than 2x2 in the experiments resulting in a high memory usage, which can reduce the memory usage while having a better reconstruction performance.

Related papers

• Depth Monocular Estimation with Attention-based Encoder-Decoder Network from Single Image [7.753378095194288]
  Vision-based approaches have recently received much attention and can overcome these drawbacks. In this work, we explore an extreme scenario in vision-based settings: estimate a depth map from one monocular image severely plagued by grid artifacts and blurry edges. Our novel approach can find the focus of current image with minimal overhead and avoid losses of depth features.
  arXiv  Detail & Related papers  (2022-10-24T23:01:25Z)
• Disentangling Light Fields for Super-Resolution and Disparity Estimation [67.50796924758221]
  Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. It is challenging for convolutional neural networks (CNNs) to process LF images since the spatial and angular information are highly inter-twined with varying disparities. We propose a generic mechanism to disentangle these coupled information for LF image processing.
  arXiv  Detail & Related papers  (2022-02-22T01:04:41Z)
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications [78.63280020581662]
  A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We demonstrate the high performance and robustness of the proposed framework compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-03-24T08:16:32Z)
• A learning-based view extrapolation method for axial super-resolution [52.748944517480155]
  Axial light field resolution refers to the ability to distinguish features at different depths by refocusing. We propose a learning-based method to extrapolate novel views from axial volumes of sheared epipolar plane images.
  arXiv  Detail & Related papers  (2021-03-11T07:22:13Z)
• On Deep Learning Techniques to Boost Monocular Depth Estimation for Autonomous Navigation [1.9007546108571112]
  Inferring the depth of images is a fundamental inverse problem within the field of Computer Vision. We propose a new lightweight and fast supervised CNN architecture combined with novel feature extraction models. We also introduce an efficient surface normals module, jointly with a simple geometric 2.5D loss function, to solve SIDE problems.
  arXiv  Detail & Related papers  (2020-10-13T18:37:38Z)
• Deep Sparse Light Field Refocusing [35.796798137910066]
  Current methods require for this purpose a dense field of angle views. We present a novel implementation of digital refocusing based on sparse angular information using neural networks.
  arXiv  Detail & Related papers  (2020-09-05T18:34:55Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)
• Spatial-Angular Attention Network for Light Field Reconstruction [64.27343801968226]
  We propose a spatial-angular attention network to perceive correspondences in the light field non-locally. Motivated by the non-local attention mechanism, a spatial-angular attention module is introduced to compute the responses from all the positions in the epipolar plane for each pixel in the light field. We then propose a multi-scale reconstruction structure to efficiently implement the non-local attention in the low spatial scale.
  arXiv  Detail & Related papers  (2020-07-05T06:55:29Z)
• Learning Light Field Angular Super-Resolution via a Geometry-Aware Network [101.59693839475783]
  We propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$times$.
  arXiv  Detail & Related papers  (2020-02-26T02:36:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.