VcT: Visual change Transformer for Remote Sensing Image Change Detection

• URL: http://arxiv.org/abs/2310.11417v1
• Date: Tue, 17 Oct 2023 17:25:31 GMT
• Title: VcT: Visual change Transformer for Remote Sensing Image Change Detection
• Authors: Bo Jiang, Zitian Wang, Xixi Wang, Ziyan Zhang, Lan Chen, Xiao Wang, Bin Luo
• Abstract summary: We propose a novel Visual change Transformer (VcT) model for visual change detection problem. Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm. Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.
• Score: 16.778418602705287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing visual change detectors usually adopt CNNs or Transformers for feature representation learning and focus on learning effective representation for the changed regions between images. Although good performance can be obtained by enhancing the features of the change regions, however, these works are still limited mainly due to the ignorance of mining the unchanged background context information. It is known that one main challenge for change detection is how to obtain the consistent representations for two images involving different variations, such as spatial variation, sunlight intensity, etc. In this work, we demonstrate that carefully mining the common background information provides an important cue to learn the consistent representations for the two images which thus obviously facilitates the visual change detection problem. Based on this observation, we propose a novel Visual change Transformer (VcT) model for visual change detection problem. To be specific, a shared backbone network is first used to extract the feature maps for the given image pair. Then, each pixel of feature map is regarded as a graph node and the graph neural network is proposed to model the structured information for coarse change map prediction. Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm. Then, these reliable tokens are enhanced by first utilizing self/cross-attention schemes and then interacting with original features via an anchor-primary attention learning module. Finally, the prediction head is proposed to get a more accurate change map. Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.

Related papers

• Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning [49.24306593078429]
  We propose a novel framework for remote sensing image change captioning, guided by Key Change Features and Instruction-tuned (KCFI) KCFI includes a ViTs encoder for extracting bi-temporal remote sensing image features, a key feature perceiver for identifying critical change areas, and a pixel-level change detection decoder. To validate the effectiveness of our approach, we compare it against several state-of-the-art change captioning methods on the LEVIR-CC dataset.
  arXiv  Detail & Related papers  (2024-09-19T09:33:33Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Gramformer: Learning Crowd Counting via Graph-Modulated Transformer [68.26599222077466]
  Gramformer is a graph-modulated transformer to enhance the network by adjusting the attention and input node features respectively. A feature-based encoding is proposed to discover the centrality positions or importance of nodes. Experiments on four challenging crowd counting datasets have validated the competitiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-01-08T13:01:54Z)
• Explicit Change Relation Learning for Change Detection in VHR Remote Sensing Images [12.228675703851733]
  We propose a network architecture NAME for the explicit mining of change relation features. The change features of change detection should be divided into pre-changed image features, post-changed image features and change relation features. Our network performs better, in terms of F1, IoU, and OA, than those of the existing advanced networks for change detection.
  arXiv  Detail & Related papers  (2023-11-14T08:47:38Z)
• Self-supervised Cross-view Representation Reconstruction for Change Captioning [113.08380679787247]
  Change captioning aims to describe the difference between a pair of similar images. Its key challenge is how to learn a stable difference representation under pseudo changes caused by viewpoint change. We propose a self-supervised cross-view representation reconstruction network.
  arXiv  Detail & Related papers  (2023-09-28T09:28:50Z)
• MapFormer: Boosting Change Detection by Using Pre-change Information [2.436285270638041]
  We leverage existing maps describing features of the earth's surface for change detection in bi-temporal images. We show that the simple integration of the additional information via concatenation of latent representations suffices to significantly outperform state-of-the-art change detection methods. Our approach outperforms existing change detection methods by an absolute 11.7% and 18.4% in terms of binary change IoU on DynamicEarthNet and HRSCD, respectively.
  arXiv  Detail & Related papers  (2023-03-31T07:39:12Z)
• Precise Facial Landmark Detection by Reference Heatmap Transformer [52.417964103227696]
  We propose a novel Reference Heatmap Transformer (RHT) for more precise facial landmark detection. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.
  arXiv  Detail & Related papers  (2023-03-14T12:26:48Z)
• IDAN: Image Difference Attention Network for Change Detection [3.5366052026723547]
  We propose a novel image difference attention network (IDAN) for remote sensing image change detection. IDAN considers the differences in regional and edge features of images and thus optimize the extracted image features. The experimental results demonstrate that the F1-score of IDAN improves 1.62% and 1.98% compared to the baseline model on WHU dataset and LEVIR-CD dataset.
  arXiv  Detail & Related papers  (2022-08-17T13:46:13Z)
• dual unet:a novel siamese network for change detection with cascade differential fusion [4.651756476458979]
  We propose a novel Siamese neural network for change detection task, namely Dual-UNet. In contrast to previous individually encoded the bitemporal images, we design an encoder differential-attention module to focus on the spatial difference relationships of pixels. Experiments demonstrate that the proposed approach consistently outperforms the most advanced methods on popular seasonal change detection datasets.
  arXiv  Detail & Related papers  (2022-08-12T14:24:09Z)
• Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
  Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries. This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management. We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
  arXiv  Detail & Related papers  (2020-10-12T13:26:30Z)
• From W-Net to CDGAN: Bi-temporal Change Detection via Deep Learning Techniques [43.58400031452662]
  We propose an end-to-end dual-branch architecture termed as the W-Net, with each branch taking as input one of the two bi-temporal images. We also apply the recently popular Generative Adversarial Network (GAN) in which our W-Net serves as the Generator. To train our networks and also facilitate future research, we construct a large scale dataset by collecting images from Google Earth.
  arXiv  Detail & Related papers  (2020-03-14T09:24:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.