Multi-scale Matching Networks for Semantic Correspondence
- URL: http://arxiv.org/abs/2108.00211v1
- Date: Sat, 31 Jul 2021 10:57:24 GMT
- Title: Multi-scale Matching Networks for Semantic Correspondence
- Authors: Dongyang Zhao, Ziyang Song, Zhenghao Ji, Gangming Zhao, Weifeng Ge and
Yizhou Yu
- Abstract summary: The proposed method achieves state-of-the-art performance on three popular benchmarks with high computational efficiency.
Our multi-scale matching network can be trained end-to-end easily with few additional learnable parameters.
- Score: 38.904735120815346
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Deep features have been proven powerful in building accurate dense semantic
correspondences in various previous works. However, the multi-scale and
pyramidal hierarchy of convolutional neural networks has not been well studied
to learn discriminative pixel-level features for semantic correspondence. In
this paper, we propose a multi-scale matching network that is sensitive to tiny
semantic differences between neighboring pixels. We follow the coarse-to-fine
matching strategy and build a top-down feature and matching enhancement scheme
that is coupled with the multi-scale hierarchy of deep convolutional neural
networks. During feature enhancement, intra-scale enhancement fuses
same-resolution feature maps from multiple layers together via local
self-attention and cross-scale enhancement hallucinates higher-resolution
feature maps along the top-down hierarchy. Besides, we learn complementary
matching details at different scales thus the overall matching score is refined
by features of different semantic levels gradually. Our multi-scale matching
network can be trained end-to-end easily with few additional learnable
parameters. Experimental results demonstrate that the proposed method achieves
state-of-the-art performance on three popular benchmarks with high
computational efficiency.
Related papers
- Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural
Networks [49.808194368781095]
We show that three-layer neural networks have provably richer feature learning capabilities than two-layer networks.
This work makes progress towards understanding the provable benefit of three-layer neural networks over two-layer networks in the feature learning regime.
arXiv Detail & Related papers (2023-05-11T17:19:30Z) - DeepSim-Nets: Deep Similarity Networks for Stereo Image Matching [0.0]
We present three multi-scale similarity learning architectures, or DeepSim networks.
These models learn pixel-level matching with a contrastive loss and are agnostic to the geometry of the considered scene.
We establish a middle ground between hybrid and end-to-end approaches by learning to densely allocate all corresponding pixels of an epipolar pair at once.
arXiv Detail & Related papers (2023-04-17T08:15:47Z) - Multi-scale and Cross-scale Contrastive Learning for Semantic
Segmentation [5.281694565226513]
We apply contrastive learning to enhance the discriminative power of the multi-scale features extracted by semantic segmentation networks.
By first mapping the encoder's multi-scale representations to a common feature space, we instantiate a novel form of supervised local-global constraint.
arXiv Detail & Related papers (2022-03-25T01:24:24Z) - Learning distinct features helps, provably [98.78384185493624]
We study the diversity of the features learned by a two-layer neural network trained with the least squares loss.
We measure the diversity by the average $L$-distance between the hidden-layer features.
arXiv Detail & Related papers (2021-06-10T19:14:45Z) - A Holistically-Guided Decoder for Deep Representation Learning with
Applications to Semantic Segmentation and Object Detection [74.88284082187462]
One common strategy is to adopt dilated convolutions in the backbone networks to extract high-resolution feature maps.
We propose one novel holistically-guided decoder which is introduced to obtain the high-resolution semantic-rich feature maps.
arXiv Detail & Related papers (2020-12-18T10:51:49Z) - GOCor: Bringing Globally Optimized Correspondence Volumes into Your
Neural Network [176.3781969089004]
Feature correlation layer serves as a key neural network module in computer vision problems that involve dense correspondences between image pairs.
We propose GOCor, a fully differentiable dense matching module, acting as a direct replacement to the feature correlation layer.
Our approach significantly outperforms the feature correlation layer for the tasks of geometric matching, optical flow, and dense semantic matching.
arXiv Detail & Related papers (2020-09-16T17:33:01Z) - Sparse Coding Driven Deep Decision Tree Ensembles for Nuclear
Segmentation in Digital Pathology Images [15.236873250912062]
We propose an easily trained yet powerful representation learning approach with performance highly competitive to deep neural networks in a digital pathology image segmentation task.
The method, called sparse coding driven deep decision tree ensembles that we abbreviate as ScD2TE, provides a new perspective on representation learning.
arXiv Detail & Related papers (2020-08-13T02:59:31Z) - Recursive Multi-model Complementary Deep Fusion forRobust Salient Object
Detection via Parallel Sub Networks [62.26677215668959]
Fully convolutional networks have shown outstanding performance in the salient object detection (SOD) field.
This paper proposes a wider'' network architecture which consists of parallel sub networks with totally different network architectures.
Experiments on several famous benchmarks clearly demonstrate the superior performance, good generalization, and powerful learning ability of the proposed wider framework.
arXiv Detail & Related papers (2020-08-07T10:39:11Z) - Boosting Connectivity in Retinal Vessel Segmentation via a Recursive
Semantics-Guided Network [23.936946593048987]
A U-shape network is enhanced by introducing a semantics-guided module, which integrates the enriched semantics information to shallow layers for guiding the network to explore more powerful features.
The carefully designed semantics-guided network has been extensively evaluated on several public datasets.
arXiv Detail & Related papers (2020-04-24T09:18:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.