Learning Stereo Matchability in Disparity Regression Networks
- URL: http://arxiv.org/abs/2008.04800v1
- Date: Tue, 11 Aug 2020 15:55:49 GMT
- Title: Learning Stereo Matchability in Disparity Regression Networks
- Authors: Jingyang Zhang, Yao Yao, Zixin Luo, Shiwei Li, Tianwei Shen, Tian
Fang, Long Quan
- Abstract summary: This paper proposes a stereo matching network that considers pixel-wise matchability.
The proposed deep stereo matchability (DSM) framework can improve the matching result or accelerate the computation while still guaranteeing the quality.
- Score: 40.08209864470944
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning-based stereo matching has recently achieved promising results, yet
still suffers difficulties in establishing reliable matches in weakly matchable
regions that are textureless, non-Lambertian, or occluded. In this paper, we
address this challenge by proposing a stereo matching network that considers
pixel-wise matchability. Specifically, the network jointly regresses disparity
and matchability maps from 3D probability volume through expectation and
entropy operations. Next, a learned attenuation is applied as the robust loss
function to alleviate the influence of weakly matchable pixels in the training.
Finally, a matchability-aware disparity refinement is introduced to improve the
depth inference in weakly matchable regions. The proposed deep stereo
matchability (DSM) framework can improve the matching result or accelerate the
computation while still guaranteeing the quality. Moreover, the DSM framework
is portable to many recent stereo networks. Extensive experiments are conducted
on Scene Flow and KITTI stereo datasets to demonstrate the effectiveness of the
proposed framework over the state-of-the-art learning-based stereo methods.
Related papers
- Left-right Discrepancy for Adversarial Attack on Stereo Networks [8.420135490466851]
We introduce a novel adversarial attack approach that generates perturbation noise specifically designed to maximize the discrepancy between left and right image features.
Experiments demonstrate the superior capability of our method to induce larger prediction errors in stereo neural networks.
arXiv Detail & Related papers (2024-01-14T02:30:38Z) - PointFix: Learning to Fix Domain Bias for Robust Online Stereo
Adaptation [67.41325356479229]
We propose to incorporate an auxiliary point-selective network into a meta-learning framework, called PointFix.
In a nutshell, our auxiliary network learns to fix local variants intensively by effectively back-propagating local information through the meta-gradient.
This network is model-agnostic, so can be used in any kind of architectures in a plug-and-play manner.
arXiv Detail & Related papers (2022-07-27T07:48:29Z) - Enhancing Multi-view Stereo with Contrastive Matching and Weighted Focal
Loss [10.847120224170698]
We propose a new method to enhance the performance of existing networks inspired by contrastive learning and feature matching.
Our method achieves state-of-the-art performance and significant improvement over baseline network.
arXiv Detail & Related papers (2022-06-21T13:10:14Z) - Revisiting Domain Generalized Stereo Matching Networks from a Feature
Consistency Perspective [65.37571681370096]
We propose a simple pixel-wise contrastive learning across the viewpoints.
A stereo selective whitening loss is introduced to better preserve the stereo feature consistency across domains.
Our method achieves superior performance over several state-of-the-art networks.
arXiv Detail & Related papers (2022-03-21T11:21:41Z) - AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach [50.855679274530615]
We present a novel domain-adaptive approach called AdaStereo to align multi-level representations for deep stereo matching networks.
Our models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo.
Our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.
arXiv Detail & Related papers (2021-12-09T15:10:47Z) - Towards Adversarially Robust and Domain Generalizable Stereo Matching by
Rethinking DNN Feature Backbones [14.569829985753346]
This paper shows that a type of weak white-box attacks can fail state-of-the-art methods.
The proposed method is tested in the SceneFlow dataset and the KITTI2015 benchmark.
It significantly improves the adversarial robustness, while retaining accuracy performance comparable to state-of-the-art methods.
arXiv Detail & Related papers (2021-07-31T22:44:18Z) - SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures.
Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities.
We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
arXiv Detail & Related papers (2021-04-08T16:15:46Z) - Adaptive Deconvolution-based stereo matching Net for Local Stereo
Matching [11.214543038438055]
In deep learning-based local stereo matching methods, larger image patches usually bring better stereo matching accuracy.
We propose an efficient CNN based structure, namely Adaptive Deconvolution-based disparity matching Net (ADSM net)
Experimental results on the KITTI 2012 and 2015 datasets demonstrate that the proposed method can achieve a good trade-off between accuracy and complexity.
arXiv Detail & Related papers (2021-01-01T12:18:53Z) - AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching [50.06646151004375]
A novel domain-adaptive pipeline called AdaStereo aims to align multi-level representations for deep stereo matching networks.
Our AdaStereo models achieve state-of-the-art cross-domain performance on multiple stereo benchmarks, including KITTI, Middlebury, ETH3D, and DrivingStereo.
arXiv Detail & Related papers (2020-04-09T16:15:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.