Revisiting Domain Generalized Stereo Matching Networks from a Feature
Consistency Perspective
- URL: http://arxiv.org/abs/2203.10887v1
- Date: Mon, 21 Mar 2022 11:21:41 GMT
- Title: Revisiting Domain Generalized Stereo Matching Networks from a Feature
Consistency Perspective
- Authors: Jiawei Zhang, Xiang Wang, Xiao Bai, Chen Wang, Lei Huang, Yimin Chen,
Lin Gu, Jun Zhou, Tatsuya Harada, Edwin R. Hancock
- Abstract summary: We propose a simple pixel-wise contrastive learning across the viewpoints.
A stereo selective whitening loss is introduced to better preserve the stereo feature consistency across domains.
Our method achieves superior performance over several state-of-the-art networks.
- Score: 65.37571681370096
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Despite recent stereo matching networks achieving impressive performance
given sufficient training data, they suffer from domain shifts and generalize
poorly to unseen domains. We argue that maintaining feature consistency between
matching pixels is a vital factor for promoting the generalization capability
of stereo matching networks, which has not been adequately considered. Here we
address this issue by proposing a simple pixel-wise contrastive learning across
the viewpoints. The stereo contrastive feature loss function explicitly
constrains the consistency between learned features of matching pixel pairs
which are observations of the same 3D points. A stereo selective whitening loss
is further introduced to better preserve the stereo feature consistency across
domains, which decorrelates stereo features from stereo viewpoint-specific
style information. Counter-intuitively, the generalization of feature
consistency between two viewpoints in the same scene translates to the
generalization of stereo matching performance to unseen domains. Our method is
generic in nature as it can be easily embedded into existing stereo networks
and does not require access to the samples in the target domain. When trained
on synthetic data and generalized to four real-world testing sets, our method
achieves superior performance over several state-of-the-art networks.
Related papers
- Playing to Vision Foundation Model's Strengths in Stereo Matching [13.887661472501618]
This study serves as the first exploration of a viable approach for adapting vision foundation models (VFMs) to stereo matching.
Our ViT adapter, referred to as ViTAS, is constructed upon three types of modules: spatial differentiation, patch attention fusion, and cross-attention.
ViTAStereo outperforms the second-best network StereoBase by approximately 7.9% in terms of the percentage of error pixels, with a tolerance of 3 pixels.
arXiv Detail & Related papers (2024-04-09T12:34:28Z) - Stereoscopic Universal Perturbations across Different Architectures and
Datasets [60.021985610201156]
We study the effect of adversarial perturbations of images on deep stereo matching networks for the disparity estimation task.
We present a method to craft a single set of perturbations that, when added to any stereo image pair in a dataset, can fool a stereo network.
Our perturbations can increase D1-error (akin to fooling rate) of state-of-the-art stereo networks from 1% to as much as 87%.
arXiv Detail & Related papers (2021-12-12T02:11:31Z) - AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach [50.855679274530615]
We present a novel domain-adaptive approach called AdaStereo to align multi-level representations for deep stereo matching networks.
Our models achieve state-of-the-art cross-domain performance on multiple benchmarks, including KITTI, Middlebury, ETH3D and DrivingStereo.
Our method is robust to various domain adaptation settings, and can be easily integrated into quick adaptation application scenarios and real-world deployments.
arXiv Detail & Related papers (2021-12-09T15:10:47Z) - Survey on Semantic Stereo Matching / Semantic Depth Estimation [0.0]
Finding pixel correspondences in non-textured, occluded and reflective areas is the major challenge in stereo matching.
Deep neural network architectures have been proposed to leverage the advantages of semantic segmentation in stereo matching.
arXiv Detail & Related papers (2021-09-21T12:11:56Z) - SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures.
Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities.
We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
arXiv Detail & Related papers (2021-04-08T16:15:46Z) - Parallax Attention for Unsupervised Stereo Correspondence Learning [46.035892564279564]
Stereo image pairs encode 3D scene cues into stereo correspondences between the left and right images.
Recent CNN based methods commonly use cost volume techniques to capture stereo correspondence over large disparities.
We propose a generic parallax-attention mechanism (PAM) to capture stereo correspondence regardless of disparity variations.
arXiv Detail & Related papers (2020-09-16T01:30:13Z) - StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization
of Domain Translation and Stereo Matching [56.95846963856928]
Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias.
We propose an end-to-end training framework with domain translation and stereo matching networks to tackle this challenge.
arXiv Detail & Related papers (2020-05-05T03:11:38Z) - AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching [50.06646151004375]
A novel domain-adaptive pipeline called AdaStereo aims to align multi-level representations for deep stereo matching networks.
Our AdaStereo models achieve state-of-the-art cross-domain performance on multiple stereo benchmarks, including KITTI, Middlebury, ETH3D, and DrivingStereo.
arXiv Detail & Related papers (2020-04-09T16:15:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.