Transformer-Based Deep Image Matching for Generalizable Person Re-identification

• URL: http://arxiv.org/abs/2105.14432v1
• Date: Sun, 30 May 2021 05:38:33 GMT
• Title: Transformer-Based Deep Image Matching for Generalizable Person Re-identification
• Authors: Shengcai Liao and Ling Shao
• Abstract summary: We investigate the possibility of applying Transformers for image matching and metric learning given pairs of images. We find that the Vision Transformer (ViT) and the vanilla Transformer with decoders are not adequate for image matching due to their lack of image-to-image attention. We propose a new simplified decoder, which drops the full attention implementation with the softmax weighting, keeping only the query-key similarity.
• Score: 114.56752624945142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformers have recently gained increasing attention in computer vision. However, existing studies mostly use Transformers for feature representation learning, e.g. for image classification and dense predictions. In this work, we further investigate the possibility of applying Transformers for image matching and metric learning given pairs of images. We find that the Vision Transformer (ViT) and the vanilla Transformer with decoders are not adequate for image matching due to their lack of image-to-image attention. Thus, we further design two naive solutions, i.e. query-gallery concatenation in ViT, and query-gallery cross-attention in the vanilla Transformer. The latter improves the performance, but it is still limited. This implies that the attention mechanism in Transformers is primarily designed for global feature aggregation, which is not naturally suitable for image matching. Accordingly, we propose a new simplified decoder, which drops the full attention implementation with the softmax weighting, keeping only the query-key similarity computation. Additionally, global max pooling and a multilayer perceptron (MLP) head are applied to decode the matching result. This way, the simplified decoder is computationally more efficient, while at the same time more effective for image matching. The proposed method, called TransMatcher, achieves state-of-the-art performance in generalizable person re-identification, with up to 6.1% and 5.7% performance gains in Rank-1 and mAP, respectively, on several popular datasets. The source code of this study will be made publicly available.

Related papers

• Boosting vision transformers for image retrieval [11.441395750267052]
  Vision transformers have achieved remarkable progress in vision tasks such as image classification and detection. However, in instance-level image retrieval, transformers have not yet shown good performance compared to convolutional networks. We propose a number of improvements that make transformers outperform the state of the art for the first time.
  arXiv  Detail & Related papers  (2022-10-21T12:17:12Z)
• Efficient Attention-free Video Shift Transformers [56.87581500474093]
  This paper tackles the problem of efficient video recognition. Video transformers have recently dominated the efficiency (top-1 accuracy vs FLOPs) spectrum. We extend our formulation in the video domain to construct Video Affine-Shift Transformer.
  arXiv  Detail & Related papers  (2022-08-23T17:48:29Z)
• Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning [50.95116994162883]
  Vision transformers have been thought of as a promising alternative to convolutional neural networks for visual recognition. This paper presents hierarchically cascaded transformers that exploit intrinsic image structures through spectral tokens pooling. HCTransformers surpass the DINO baseline by a large margin of 9.7% 5-way 1-shot accuracy and 9.17% 5-way 5-shot accuracy on miniImageNet.
  arXiv  Detail & Related papers  (2022-03-17T03:49:58Z)
• Multi-Tailed Vision Transformer for Efficient Inference [44.43126137573205]
  Vision Transformer (ViT) has achieved promising performance in image recognition. We propose a Multi-Tailed Vision Transformer (MT-ViT) in the paper. MT-ViT adopts multiple tails to produce visual sequences of different lengths for the following Transformer encoder.
  arXiv  Detail & Related papers  (2022-03-03T09:30:55Z)
• Towards End-to-End Image Compression and Analysis with Transformers [99.50111380056043]
  We propose an end-to-end image compression and analysis model with Transformers, targeting to the cloud-based image classification application. We aim to redesign the Vision Transformer (ViT) model to perform image classification from the compressed features and facilitate image compression with the long-term information from the Transformer. Experimental results demonstrate the effectiveness of the proposed model in both the image compression and the classification tasks.
  arXiv  Detail & Related papers  (2021-12-17T03:28:14Z)
• Vision Transformer with Progressive Sampling [73.60630716500154]
  We propose an iterative and progressive sampling strategy to locate discriminative regions. When trained from scratch on ImageNet, PS-ViT performs 3.8% higher than the vanilla ViT in terms of top-1 accuracy.
  arXiv  Detail & Related papers  (2021-08-03T18:04:31Z)
• XCiT: Cross-Covariance Image Transformers [73.33400159139708]
  We propose a "transposed" version of self-attention that operates across feature channels rather than tokens. The resulting cross-covariance attention (XCA) has linear complexity in the number of tokens, and allows efficient processing of high-resolution images.
  arXiv  Detail & Related papers  (2021-06-17T17:33:35Z)
• Fully Transformer Networks for Semantic ImageSegmentation [26.037770622551882]
  We explore a novel framework for semantic image segmentation, which is encoder-decoder based Fully Transformer Networks (FTN) We propose a Pyramid Group Transformer (PGT) as the encoder for progressively learning hierarchical features, while reducing the computation complexity of the standard visual transformer(ViT) Then, we propose a Feature Pyramid Transformer (FPT) to fuse semantic-level and spatial-level information from multiple levels of the PGT encoder for semantic image segmentation.
  arXiv  Detail & Related papers  (2021-06-08T05:15:28Z)
• CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification [17.709880544501758]
  We propose a dual-branch transformer to combine image patches of different sizes to produce stronger image features. Our approach processes small-patch and large-patch tokens with two separate branches of different computational complexity. Our proposed cross-attention only requires linear time for both computational and memory complexity instead of quadratic time otherwise.
  arXiv  Detail & Related papers  (2021-03-27T13:03:17Z)
• Training Vision Transformers for Image Retrieval [32.09708181236154]
  We adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective. Our results show consistent and significant improvements of transformers over convolution-based approaches.
  arXiv  Detail & Related papers  (2021-02-10T18:56:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.