Cross-Domain Separable Translation Network for Multimodal Image Change Detection
- URL: http://arxiv.org/abs/2407.16158v1
- Date: Tue, 23 Jul 2024 03:56:02 GMT
- Title: Cross-Domain Separable Translation Network for Multimodal Image Change Detection
- Authors: Tao Zhan, Yuanyuan Zhu, Jie Lan, Qianlong Dang,
- Abstract summary: multimodal change detection (MCD) is particularly critical in the remote sensing community.
This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors.
A novel unsupervised cross-domain separable translation network (CSTN) is proposed to overcome these limitations.
- Score: 11.25422609271201
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the remote sensing community, multimodal change detection (MCD) is particularly critical due to its ability to track changes across different imaging conditions and sensor types, making it highly applicable to a wide range of real-world scenarios. This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors with varying styles and statistical characteristics of geospatial objects. Traditional MCD methods often struggle with these variations, leading to inaccurate and unreliable results. To overcome these limitations, a novel unsupervised cross-domain separable translation network (CSTN) is proposed, which uniquely integrates a within-domain self-reconstruction and a cross-domain image translation and cycle-reconstruction workflow with change detection constraints. The model is optimized by implementing both the tasks of image translation and MCD simultaneously, thereby guaranteeing the comparability of learned features from multimodal images. Specifically, a simple yet efficient dual-branch convolutional architecture is employed to separate the content and style information of multimodal images. This process generates a style-independent content-comparable feature space, which is crucial for achieving accurate change detection even in the presence of significant sensor variations. Extensive experimental results demonstrate the effectiveness of the proposed method, showing remarkable improvements over state-of-the-art approaches in terms of accuracy and efficacy for MCD. The implementation of our method will be publicly available at \url{https://github.com/OMEGA-RS/CSTN}
Related papers
- Patch-Consistent Optical Translation Across Sensors: Large-Scale Denoising Diffusion with Heterogeneous Change Detection as a Use Case [3.4085512042262374]
In remote sensing, the challenge of comparing images captured by disparate sensors is a common obstacle.
We propose a novel method that leverages DDIM for effective optical image translation over large areas.
Our approach is tailored to super-resolve large-scale low spatial resolution images into high-resolution equivalents from disparate optical sensors.
arXiv Detail & Related papers (2024-04-17T10:49:00Z) - BD-MSA: Body decouple VHR Remote Sensing Image Change Detection method
guided by multi-scale feature information aggregation [4.659935767219465]
The purpose of remote sensing image change detection (RSCD) is to detect differences between bi-temporal images taken at the same place.
Deep learning has been extensively used to RSCD tasks, yielding significant results in terms of result recognition.
arXiv Detail & Related papers (2024-01-09T02:53:06Z) - Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing.
Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery.
We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
arXiv Detail & Related papers (2023-12-19T08:14:14Z) - dual unet:a novel siamese network for change detection with cascade
differential fusion [4.651756476458979]
We propose a novel Siamese neural network for change detection task, namely Dual-UNet.
In contrast to previous individually encoded the bitemporal images, we design an encoder differential-attention module to focus on the spatial difference relationships of pixels.
Experiments demonstrate that the proposed approach consistently outperforms the most advanced methods on popular seasonal change detection datasets.
arXiv Detail & Related papers (2022-08-12T14:24:09Z) - Multi-domain Unsupervised Image-to-Image Translation with Appearance
Adaptive Convolution [62.4972011636884]
We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework.
We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance.
We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
arXiv Detail & Related papers (2022-02-06T14:12:34Z) - LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution
Homography Estimation [52.63874513999119]
Cross-resolution image alignment is a key problem in multiscale giga photography.
Existing deep homography methods neglecting the explicit formulation of correspondences between them, which leads to degraded accuracy in cross-resolution challenges.
We propose a local transformer network embedded within a multiscale structure to explicitly learn correspondences between the multimodal inputs.
arXiv Detail & Related papers (2021-06-08T02:51:45Z) - A Multiscale Graph Convolutional Network for Change Detection in
Homogeneous and Heterogeneous Remote Sensing Images [12.823633963080281]
Change detection (CD) in remote sensing images has been an ever-expanding area of research.
In this paper, a novel CD method based on the graph convolutional network (GCN) and multiscale object-based technique is proposed for both homogeneous and heterogeneous images.
arXiv Detail & Related papers (2021-02-16T09:26:31Z) - Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries.
This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management.
We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
arXiv Detail & Related papers (2020-10-12T13:26:30Z) - MuCAN: Multi-Correspondence Aggregation Network for Video
Super-Resolution [63.02785017714131]
Video super-resolution (VSR) aims to utilize multiple low-resolution frames to generate a high-resolution prediction for each frame.
Inter- and intra-frames are the key sources for exploiting temporal and spatial information.
We build an effective multi-correspondence aggregation network (MuCAN) for VSR.
arXiv Detail & Related papers (2020-07-23T05:41:27Z) - Unsupervised Bidirectional Cross-Modality Adaptation via Deeply
Synergistic Image and Feature Alignment for Medical Image Segmentation [73.84166499988443]
We present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA)
Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives.
Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images.
arXiv Detail & Related papers (2020-02-06T13:49:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.