MapFormer: Boosting Change Detection by Using Pre-change Information

• URL: http://arxiv.org/abs/2303.17859v4
• Date: Thu, 7 Dec 2023 16:44:44 GMT
• Title: MapFormer: Boosting Change Detection by Using Pre-change Information
• Authors: Maximilian Bernhard, Niklas Strau{\ss}, Matthias Schubert
• Abstract summary: We leverage existing maps describing features of the earth's surface for change detection in bi-temporal images. We show that the simple integration of the additional information via concatenation of latent representations suffices to significantly outperform state-of-the-art change detection methods. Our approach outperforms existing change detection methods by an absolute 11.7% and 18.4% in terms of binary change IoU on DynamicEarthNet and HRSCD, respectively.
• Score: 2.436285270638041
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Change detection in remote sensing imagery is essential for a variety of applications such as urban planning, disaster management, and climate research. However, existing methods for identifying semantically changed areas overlook the availability of semantic information in the form of existing maps describing features of the earth's surface. In this paper, we leverage this information for change detection in bi-temporal images. We show that the simple integration of the additional information via concatenation of latent representations suffices to significantly outperform state-of-the-art change detection methods. Motivated by this observation, we propose the new task of *Conditional Change Detection*, where pre-change semantic information is used as input next to bi-temporal images. To fully exploit the extra information, we propose *MapFormer*, a novel architecture based on a multi-modal feature fusion module that allows for feature processing conditioned on the available semantic information. We further employ a supervised, cross-modal contrastive loss to guide the learning of visual representations. Our approach outperforms existing change detection methods by an absolute 11.7\% and 18.4\% in terms of binary change IoU on DynamicEarthNet and HRSCD, respectively. Furthermore, we demonstrate the robustness of our approach to the quality of the pre-change semantic information and the absence pre-change imagery. The code is available at https://github.com/mxbh/mapformer.

Related papers

• Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detection [82.65760006883248]
  We introduce a new task named Change Detection Question Answering and Grounding (CDQAG) CDQAG extends the traditional change detection task by providing interpretable textual answers and intuitive visual evidence. We construct the first CDQAG benchmark dataset, termed QAG-360K, comprising over 360K triplets of questions, textual answers, and corresponding high-quality visual masks.
  arXiv  Detail & Related papers  (2024-10-31T11:20:13Z)
• A Late-Stage Bitemporal Feature Fusion Network for Semantic Change Detection [32.112311027857636]
  We propose a novel late-stage bitemporal feature fusion network to address the issue of semantic change detection. Specifically, we propose local global attentional aggregation module to strengthen feature fusion, and propose local global context enhancement module to highlight pivotal semantics. Our proposed model achieves new state-of-the-art performance on both datasets.
  arXiv  Detail & Related papers  (2024-06-15T16:02:10Z)
• DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusion [82.2425759608975]
  Infrared-visible object detection aims to achieve robust even full-day object detection by fusing the complementary information of infrared and visible images. We propose a Dynamic Adaptive Multispectral Detection Transformer (DAMSDet) to address these two challenges. Experiments on four public datasets demonstrate significant improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-01T07:03:27Z)
• MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations [50.79913333804232]
  We propose a memory-supported transformer (MS-Former) for weakly supervised change detection. MS-Former consists of a bi-directional attention block (BAB) and a patch-level supervision scheme (PSS) Experimental results on three benchmark datasets demonstrate the effectiveness of our proposed method in the change detection task.
  arXiv  Detail & Related papers  (2023-11-16T09:57:29Z)
• VcT: Visual change Transformer for Remote Sensing Image Change Detection [16.778418602705287]
  We propose a novel Visual change Transformer (VcT) model for visual change detection problem. Top-K reliable tokens can be mined from the map and refined by using the clustering algorithm. Extensive experiments on multiple benchmark datasets validated the effectiveness of our proposed VcT model.
  arXiv  Detail & Related papers  (2023-10-17T17:25:31Z)
• SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation [68.42476385214785]
  We propose a novel Spatial-Semantic Map Guided (SSMG) diffusion model that adopts the feature map, derived from the layout, as guidance. SSMG achieves superior generation quality with sufficient spatial and semantic controllability compared to previous works. We also propose the Relation-Sensitive Attention (RSA) and Location-Sensitive Attention (LSA) mechanisms.
  arXiv  Detail & Related papers  (2023-08-20T04:09:12Z)
• Precise Facial Landmark Detection by Reference Heatmap Transformer [52.417964103227696]
  We propose a novel Reference Heatmap Transformer (RHT) for more precise facial landmark detection. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.
  arXiv  Detail & Related papers  (2023-03-14T12:26:48Z)
• Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery [6.586756080460231]
  We train a change detector using two spatially unrelated images with corresponding semantic labels such as building. We show that manipulating the source image as an after-image is crucial to the performance of change detection. Our method outperforms existing methods based on single-temporal supervision.
  arXiv  Detail & Related papers  (2022-12-20T13:26:42Z)
• Situational Perception Guided Image Matting [16.1897179939677]
  We propose a Situational Perception Guided Image Matting (SPG-IM) method that mitigates subjective bias of matting annotations. SPG-IM can better associate inter-objects and object-to-environment saliency, and compensate the subjective nature of image matting.
  arXiv  Detail & Related papers  (2022-04-20T07:35:51Z)
• Ventral-Dorsal Neural Networks: Object Detection via Selective Attention [51.79577908317031]
  We propose a new framework called Ventral-Dorsal Networks (VDNets) Inspired by the structure of the human visual system, we propose the integration of a "Ventral Network" and a "Dorsal Network" Our experimental results reveal that the proposed method outperforms state-of-the-art object detection approaches.
  arXiv  Detail & Related papers  (2020-05-15T23:57:36Z)
• Semantic Change Pattern Analysis [31.829389559219276]
  We propose a new task called semantic change pattern analysis for aerial images. Given a pair of co-registered aerial images, the task requires a result including both where and what changes happen. We provide the first well-annotated aerial image dataset for this task.
  arXiv  Detail & Related papers  (2020-03-07T02:22:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.