EPI-based Oriented Relation Networks for Light Field Depth Estimation

• URL: http://arxiv.org/abs/2007.04538v2
• Date: Wed, 26 Aug 2020 07:10:04 GMT
• Title: EPI-based Oriented Relation Networks for Light Field Depth Estimation
• Authors: Kunyuan Li, Jun Zhang, Rui Sun, Xudong Zhang, Jun Gao
• Abstract summary: We propose an end-to-end fully convolutional network (FCN) to estimate the depth value of the intersection point on the horizontal and vertical Epipolar Plane Image (EPI) We present a new feature-extraction module, called Oriented Relation Module (ORM), that constructs the relationship between the line orientations. To facilitate training, we also propose a refocusing-based data augmentation method to obtain different slopes from EPIs of the same scene point.
• Score: 13.120247042876175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Light field cameras record not only the spatial information of observed scenes but also the directions of all incoming light rays. The spatial and angular information implicitly contain geometrical characteristics such as multi-view or epipolar geometry, which can be exploited to improve the performance of depth estimation. An Epipolar Plane Image (EPI), the unique 2D spatial-angular slice of the light field, contains patterns of oriented lines. The slope of these lines is associated with the disparity. Benefiting from this property of EPIs, some representative methods estimate depth maps by analyzing the disparity of each line in EPIs. However, these methods often extract the optimal slope of the lines from EPIs while ignoring the relationship between neighboring pixels, which leads to inaccurate depth map predictions. Based on the observation that an oriented line and its neighboring pixels in an EPI share a similar linear structure, we propose an end-to-end fully convolutional network (FCN) to estimate the depth value of the intersection point on the horizontal and vertical EPIs. Specifically, we present a new feature-extraction module, called Oriented Relation Module (ORM), that constructs the relationship between the line orientations. To facilitate training, we also propose a refocusing-based data augmentation method to obtain different slopes from EPIs of the same scene point. Extensive experiments verify the efficacy of learning relations and show that our approach is competitive to other state-of-the-art methods. The code and the trained models are available at https://github.com/lkyahpu/EPI_ORM.git.

Related papers

• IDLS: Inverse Depth Line based Visual-Inertial SLAM [9.38589798999922]
  Inverse Depth Line SLAM (IDLS) is proposed to track the line features in SLAM in an accurate and efficient way. IDLS is extensively evaluated in multiple perceptually-challenging datasets.
  arXiv  Detail & Related papers  (2023-04-23T20:53:05Z)
• Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution [36.69391399634076]
  Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR) We propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line.
  arXiv  Detail & Related papers  (2023-02-16T03:40:40Z)
• GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs [49.55919802779889]
  We propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. In this work, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. Our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps.
  arXiv  Detail & Related papers  (2022-10-19T17:56:03Z)
• Geo-NI: Geometry-aware Neural Interpolation for Light Field Rendering [57.775678643512435]
  We present a Geometry-aware Neural Interpolation (Geo-NI) framework for light field rendering. By combining the superiorities of NI and DIBR, the proposed Geo-NI is able to render views with large disparity.
  arXiv  Detail & Related papers  (2022-06-20T12:25:34Z)
• Learning Implicit Feature Alignment Function for Semantic Segmentation [51.36809814890326]
  Implicit Feature Alignment function (IFA) is inspired by the rapidly expanding topic of implicit neural representations. We show that IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art accuracy trade-off on common benchmarks.
  arXiv  Detail & Related papers  (2022-06-17T09:40:14Z)
• Light Field Depth Estimation via Stitched Epipolar Plane Images [45.5616314866457]
  We propose the concept of stitched-EPI (SEPI) to enhance slope computation. SEPI achieves this by shifting and concatenating lines from different EPIs that correspond to the same 3D point. We also present a depth propagation strategy aimed at improving depth estimation in texture-less regions.
  arXiv  Detail & Related papers  (2022-03-29T02:43:40Z)
• Self-Supervised Light Field Depth Estimation Using Epipolar Plane Images [13.137957601685041]
  We propose a self-supervised learning framework for light field depth estimation. Compared with other state-of-the-art methods, the proposed method can also obtain higher quality results in real-world scenarios.
  arXiv  Detail & Related papers  (2022-03-29T01:18:59Z)
• Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications [78.63280020581662]
  A novel convolutional neural network (CNN)-based framework is developed for light field reconstruction from a sparse set of views. We demonstrate the high performance and robustness of the proposed framework compared with state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-03-24T08:16:32Z)
• Augmented Parallel-Pyramid Net for Attention Guided Pose-Estimation [90.28365183660438]
  This paper proposes an augmented parallel-pyramid net with attention partial module and differentiable auto-data augmentation. We define a new pose search space where the sequences of data augmentations are formulated as a trainable and operational CNN component. Notably, our method achieves the top-1 accuracy on the challenging COCO keypoint benchmark and the state-of-the-art results on the MPII datasets.
  arXiv  Detail & Related papers  (2020-03-17T03:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.