OccCasNet: Occlusion-aware Cascade Cost Volume for Light Field Depth Estimation

• URL: http://arxiv.org/abs/2305.17710v1
• Date: Sun, 28 May 2023 12:31:27 GMT
• Title: OccCasNet: Occlusion-aware Cascade Cost Volume for Light Field Depth Estimation
• Authors: Wentao Chao, Fuqing Duan, Xuechun Wang, Yingqian Wang, Guanghui Wang
• Abstract summary: We propose an occlusion-aware cascade cost volume for LF depth (disparity) estimation. Our strategy reduces the sampling number while keeping the sampling interval constant during the construction of a finer cost volume. Our method achieves a superior balance between accuracy and efficiency and ranks first in terms of MSE and Q25 metrics.
• Score: 26.572015989990845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Light field (LF) depth estimation is a crucial task with numerous practical applications. However, mainstream methods based on the multi-view stereo (MVS) are resource-intensive and time-consuming as they need to construct a finer cost volume. To address this issue and achieve a better trade-off between accuracy and efficiency, we propose an occlusion-aware cascade cost volume for LF depth (disparity) estimation. Our cascaded strategy reduces the sampling number while keeping the sampling interval constant during the construction of a finer cost volume. We also introduce occlusion maps to enhance accuracy in constructing the occlusion-aware cost volume. Specifically, we first obtain the coarse disparity map through the coarse disparity estimation network. Then, the sub-aperture images (SAIs) of side views are warped to the center view based on the initial disparity map. Next, we propose photo-consistency constraints between the warped SAIs and the center SAI to generate occlusion maps for each SAI. Finally, we introduce the coarse disparity map and occlusion maps to construct an occlusion-aware refined cost volume, enabling the refined disparity estimation network to yield a more precise disparity map. Extensive experiments demonstrate the effectiveness of our method. Compared with state-of-the-art methods, our method achieves a superior balance between accuracy and efficiency and ranks first in terms of MSE and Q25 metrics among published methods on the HCI 4D benchmark. The code and model of the proposed method are available at https://github.com/chaowentao/OccCasNet.

Related papers

• ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation [62.600382533322325]
  We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
  arXiv  Detail & Related papers  (2024-07-11T05:11:56Z)
• Unsupervised Light Field Depth Estimation via Multi-view Feature Matching with Occlusion Prediction [15.421219881815956]
  It is costly to obtain sufficient depth labels for supervised training. In this paper, we propose an unsupervised framework to estimate depth from LF images.
  arXiv  Detail & Related papers  (2023-01-20T06:11:17Z)
• Robust and accurate depth estimation by fusing LiDAR and Stereo [8.85338187686374]
  We propose a precision and robust method for fusing the LiDAR and stereo cameras. This method fully combines the advantages of the LiDAR and stereo camera. We evaluate the proposed pipeline on the KITTI benchmark.
  arXiv  Detail & Related papers  (2022-07-13T11:55:15Z)
• Non-parametric Depth Distribution Modelling based Depth Inference for Multi-view Stereo [43.415242967722804]
  Recent cost volume pyramid based deep neural networks have unlocked the potential of efficiently leveraging high-resolution images for depth inference from multi-view stereo. In general, those approaches assume that the depth of each pixel follows a unimodal distribution. We propose constructing the cost volume by non-parametric depth distribution modeling to handle pixels with unimodal and multi-modal distributions.
  arXiv  Detail & Related papers  (2022-05-08T05:13:04Z)
• Stereo Matching with Cost Volume based Sparse Disparity Propagation [27.74131924190943]
  We propose a simple yet novel scheme to improve general stereo matching based on matching cost volume and sparse matching feature points. Our scheme achieves promising performance comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-28T05:20:41Z)
• Cascaded Residual Density Network for Crowd Counting [63.714719914701014]
  We propose a novel Cascaded Residual Density Network (CRDNet) in a coarse-to-fine approach to generate the high-quality density map for crowd counting more accurately. A novel additional local count loss is presented to refine the accuracy of crowd counting.
  arXiv  Detail & Related papers  (2021-07-29T03:07:11Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• An Adaptive Framework for Learning Unsupervised Depth Completion [59.17364202590475]
  We present a method to infer a dense depth map from a color image and associated sparse depth measurements. We show that regularization and co-visibility are related via the fitness of the model to data and can be unified into a single framework.
  arXiv  Detail & Related papers  (2021-06-06T02:27:55Z)
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching [27.313740022587442]
  We propose CFNet, a Cascade and Fused cost volume based network to improve the robustness of the stereo matching network. We employ a variance-based uncertainty estimation to adaptively adjust the next stage disparity search space. Our proposed method achieves the state-of-the-art overall performance and obtains the 1st place on the stereo task of Robust Vision Challenge 2020.
  arXiv  Detail & Related papers  (2021-04-09T11:38:59Z)
• CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth [83.77839773394106]
  We present a lightweight, tightly-coupled deep depth network and visual-inertial odometry system. We provide the network with previously marginalized sparse features from VIO to increase the accuracy of initial depth prediction. We show that it can run in real-time with single-thread execution while utilizing GPU acceleration only for the network and code Jacobian.
  arXiv  Detail & Related papers  (2020-12-18T09:42:54Z)
• Direct Depth Learning Network for Stereo Matching [79.3665881702387]
  A novel Direct Depth Learning Network (DDL-Net) is designed for stereo matching. DDL-Net consists of two stages: the Coarse Depth Estimation stage and the Adaptive-Grained Depth Refinement stage. We show that DDL-Net achieves an average improvement of 25% on the SceneFlow dataset and $12%$ on the DrivingStereo dataset.
  arXiv  Detail & Related papers  (2020-12-10T10:33:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.