PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching

• URL: http://arxiv.org/abs/2103.07094v1
• Date: Fri, 12 Mar 2021 05:27:14 GMT
• Title: PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching
• Authors: Hengli Wang, Rui Fan, Peide Cai, Ming Liu
• Abstract summary: We propose a robust and effective self-supervised stereo matching approach, consisting of a pyramid voting module (PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically, our OptStereo first builds multi-scale cost volumes, and then adopts a recurrent unit to iteratively update disparity estimations at high resolution. We publish the HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under different illumination and weather conditions for research purposes.
• Score: 14.603116313499648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Supervised learning with deep convolutional neural networks (DCNNs) has seen huge adoption in stereo matching. However, the acquisition of large-scale datasets with well-labeled ground truth is cumbersome and labor-intensive, making supervised learning-based approaches often hard to implement in practice. To overcome this drawback, we propose a robust and effective self-supervised stereo matching approach, consisting of a pyramid voting module (PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically, our OptStereo first builds multi-scale cost volumes, and then adopts a recurrent unit to iteratively update disparity estimations at high resolution; while our PVM can generate reliable semi-dense disparity images, which can be employed to supervise OptStereo training. Furthermore, we publish the HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under different illumination and weather conditions for research purposes. Extensive experimental results demonstrate the effectiveness and efficiency of our self-supervised stereo matching approach on the KITTI Stereo benchmarks and our HKUST-Drive dataset. PVStereo, our best-performing implementation, greatly outperforms all other state-of-the-art self-supervised stereo matching approaches. Our project page is available at sites.google.com/view/pvstereo.

Related papers

• RomniStereo: Recurrent Omnidirectional Stereo Matching [6.153793254880079]
  We propose a recurrent omnidirectional stereo matching (RomniStereo) algorithm. Our best model improves the average MAE metric by 40.7% over the previous SOTA baseline. When visualizing the results, our models demonstrate clear advantages on both synthetic and realistic examples.
  arXiv  Detail & Related papers  (2024-01-09T04:06:01Z)
• OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline [25.4712469033627]
  We develop a flexible and efficient stereo matching, called OpenStereo. OpenStereo includes training and inference codes of more than 10 network models. We conduct an exhaustive analysis and deconstruction of recent developments in stereo matching through comprehensive ablative experiments. Our StereoBase ranks 1st on SceneFlow, KITTI 2015, 2012 (Reflective) among published methods and achieves the best performance across all metrics.
  arXiv  Detail & Related papers  (2023-12-01T04:35:47Z)
• Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching [77.133400999703]
  Correlation based stereo matching has achieved outstanding performance. Current methods with a fixed model do not work uniformly well across various datasets. This paper proposes a new perspective to dynamically calculate correlation for robust stereo matching.
  arXiv  Detail & Related papers  (2023-07-26T09:47:37Z)
• AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach [50.855679274530615]
  We present a novel domain-adaptive approach called AdaStereo to align multi-level representations for deep stereo matching networks. Our models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo. Our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.
  arXiv  Detail & Related papers  (2021-12-09T15:10:47Z)
• SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
  We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
  arXiv  Detail & Related papers  (2021-04-08T16:15:46Z)
• Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation [51.714092199995044]
  In many fields, self-supervised learning solutions are rapidly evolving and filling the gap with supervised approaches. We propose a novel self-supervised paradigm reversing the link between the two. In order to train deep stereo networks, we distill knowledge through a monocular completion network.
  arXiv  Detail & Related papers  (2020-08-17T07:40:22Z)
• Self-adapting confidence estimation for stereo [48.56220165347967]
  We propose a flexible and lightweight solution enabling self-adapting confidence estimation to the stereo algorithm or network. Our strategy allows us not only a seamless integration with any stereo system, but also, due to its self-adapting capability, for its out-of-the-box deployment in the field.
  arXiv  Detail & Related papers  (2020-08-14T16:17:28Z)
• Expanding Sparse Guidance for Stereo Matching [24.74333370941674]
  We propose a novel sparsity expansion technique to expand the sparse cues concerning RGB images for local feature enhancement. Our approach significantly boosts the existing state-of-the-art stereo algorithms with extremely sparse cues.
  arXiv  Detail & Related papers  (2020-04-24T06:41:11Z)
• AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching [50.06646151004375]
  A novel domain-adaptive pipeline called AdaStereo aims to align multi-level representations for deep stereo matching networks. Our AdaStereo models achieve state-of-the-art cross-domain performance on multiple stereo benchmarks, including KITTI, Middlebury, ETH3D, and DrivingStereo.
  arXiv  Detail & Related papers  (2020-04-09T16:15:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.