Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images

• URL: http://arxiv.org/abs/2604.05527v1
• Date: Tue, 07 Apr 2026 07:28:17 GMT
• Title: Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images
• Abstract summary: Multimodal change detection (MMCD) identifies changed areas in multimodal remote sensing (RS) data.<n>MMCD approaches exhibit limitations in cross-modal interaction and exploiting modality-specific characteristics.<n>We propose STSF-Net, a framework for MMCD between optical semantic and SAR images.
• Score: 25.450869122142787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal change detection (MMCD) identifies changed areas in multimodal remote sensing (RS) data, demonstrating significant application value in land use monitoring, disaster assessment, and urban sustainable development. However, literature MMCD approaches exhibit limitations in cross-modal interaction and exploiting modality-specific characteristics. This leads to insufficient modeling of fine-grained change information, thus hindering the precise detection of semantic changes in multimodal data. To address the above problems, we propose STSF-Net, a framework designed for MMCD between optical and SAR images. STSF-Net jointly models modality-specific and spatio-temporal common features to enhance change representations. Specifically, modality-specific features are exploited to capture genuine semantic change signals, while spatio-temporal common features are embedded to suppress pseudo-changes caused by differences in imaging mechanisms. Furthermore, we introduce an optical and SAR feature fusion strategy that adaptively adjusts feature importance based on semantic priors obtained from pre-trained foundational models, enabling semantic-guided adaptive fusion of multi-modal information. In addition, we introduce the Delta-SN6 dataset, the first openly-accessible multiclass MMCD benchmark consisting of very-high-resolution (VHR) fully polarimetric SAR and optical images. Experimental results on Delta-SN6, BRIGHT, and Wuhan-Het datasets demonstrate that our method outperforms the state-of-the-art (SOTA) by 3.21%, 1.08%, and 1.32% in mIoU, respectively. The associated code and Delta-SN6 dataset will be released at: https://github.com/liuxuanguang/STSF-Net.

Related papers

• Synthetic Aperture Radar Image Change Detection Based on Global Dynamic Context-Aware Network [41.25478399274141]
  Convolutional neural networks (CNNs) have been extensively and successfully applied to the task of synthetic aperture radar (SAR) image change detection.<n>We propose a novel Global Dynamic Context-Aware Network (GDNet) specifically tailored for SAR image change detection.<n>At the core of our approach lies a novel global dynamic convolution module, which adaptively modulates convolution kernel weights according to the global semantic information extracted from the input features.
  arXiv  Detail & Related papers  (2026-05-16T02:33:10Z)
• Multi-Modal Building Change Detection for Large-Scale Small Changes: Benchmark and Baseline [16.08630411584202]
  Change detection in optical remote sensing imagery is susceptible to illumination fluctuations, seasonal changes, and variations in surface land-cover materials.<n>We introduce the Large-scale Small-change Multi-modal dataset (LSMD), a bi-temporal RGB-NIR building change detection benchmark dataset.<n>We further propose the Multi-modal Spectral Complementarity Network (MSCNet) to achieve effective cross-modal feature fusion.
  arXiv  Detail & Related papers  (2026-03-19T16:05:05Z)
• LDGNet: A Lightweight Difference Guiding Network for Remote Sensing Change Detection [6.554696547472252]
  We propose a Lightweight Difference Guiding Network (LDGNet) to guide optical remote sensing change detection.<n>First, to enhance the feature representation capability of the lightweight backbone network, we propose the Difference Guiding Module (DGM)<n>Second, we propose the Difference-Aware Dynamic Fusion (DADF) module with Visual State Space Model (VSSM) for lightweight long-range dependency modeling.
  arXiv  Detail & Related papers  (2025-04-07T13:33:54Z)
• Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation [54.96563068182733]
  We propose Modality Adaptation with text-to-image Diffusion Models (MADM) for semantic segmentation task. MADM utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model's cross-modality capabilities. We show that MADM achieves state-of-the-art adaptation performance across various modality tasks, including images to depth, infrared, and event modalities.
  arXiv  Detail & Related papers  (2024-10-29T03:49:40Z)
• Cross-Domain Separable Translation Network for Multimodal Image Change Detection [11.25422609271201]
  multimodal change detection (MCD) is particularly critical in the remote sensing community. This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors. A novel unsupervised cross-domain separable translation network (CSTN) is proposed to overcome these limitations.
  arXiv  Detail & Related papers  (2024-07-23T03:56:02Z)
• Modality Prompts for Arbitrary Modality Salient Object Detection [57.610000247519196]
  This paper delves into the task of arbitrary modality salient object detection (AM SOD) It aims to detect salient objects from arbitrary modalities, eg RGB images, RGB-D images, and RGB-D-T images. A novel modality-adaptive Transformer (MAT) will be proposed to investigate two fundamental challenges of AM SOD.
  arXiv  Detail & Related papers  (2024-05-06T11:02:02Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Siamese Meets Diffusion Network: SMDNet for Enhanced Change Detection in High-Resolution RS Imagery [7.767708235606408]
  We propose a new network, Siamese-U2Net Feature Differential Meets Network (SMDNet) This network combines the Siam-U2Net Feature Differential (SU-FDE) and the denoising diffusion implicit model to improve the accuracy of image edge change detection. Our method's combination of feature extraction and diffusion models demonstrates effectiveness in change detection in remote sensing images.
  arXiv  Detail & Related papers  (2024-01-17T16:48:55Z)
• Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities [76.08541852988536]
  We propose to use invariant features for a missing modality imagination network (IF-MMIN) We show that the proposed model outperforms all baselines and invariantly improves the overall emotion recognition performance under uncertain missing-modality conditions.
  arXiv  Detail & Related papers  (2022-10-27T12:16:25Z)
• Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
  Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries. This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management. We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
  arXiv  Detail & Related papers  (2020-10-12T13:26:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.