Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework

• URL: http://arxiv.org/abs/2501.10075v1
• Date: Fri, 17 Jan 2025 09:47:27 GMT
• Title: Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework
• Abstract summary: Remote sensing change captioning (RSICC) aims to describe changes between bitemporal images in natural language.<n> SECOND-CC is a novel RSICC dataset featuring high-resolution RGB image pairs, semantic segmentation maps, and diverse real-world scenarios.<n>MModalCC is a multimodal framework that integrates semantic and visual data using advanced attention mechanisms.
• Score: 1.5943223374606597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remote sensing change captioning (RSICC) aims to describe changes between bitemporal images in natural language. Existing methods often fail under challenges like illumination differences, viewpoint changes, blur effects, leading to inaccuracies, especially in no-change regions. Moreover, the images acquired at different spatial resolutions and have registration errors tend to affect the captions. To address these issues, we introduce SECOND-CC, a novel RSICC dataset featuring high-resolution RGB image pairs, semantic segmentation maps, and diverse real-world scenarios. SECOND-CC which contains 6,041 pairs of bitemporal RS images and 30,205 sentences describing the differences between images. Additionally, we propose MModalCC, a multimodal framework that integrates semantic and visual data using advanced attention mechanisms, including Cross-Modal Cross Attention (CMCA) and Multimodal Gated Cross Attention (MGCA). Detailed ablation studies and attention visualizations further demonstrate its effectiveness and ability to address RSICC challenges. Comprehensive experiments show that MModalCC outperforms state-of-the-art RSICC methods, including RSICCformer, Chg2Cap, and PSNet with +4.6% improvement on BLEU4 score and +9.6% improvement on CIDEr score. We will make our dataset and codebase publicly available to facilitate future research at https://github.com/ChangeCapsInRS/SecondCC

Related papers

• CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation [22.23249458077542]
  Change-aware Captioning and Reasoning Chain (CCRC) is a dual-chain framework that decouples semantic reasoning from spatial segmentation.<n>CCRC achieves state-of-the-art performance on both synthetic and real-world change detection benchmarks.
  arXiv  Detail & Related papers  (2026-06-27T04:07:31Z)
• RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning [49.24254347822435]
  Remote Sensing Image Change Captioning (RSICC) aims to describe changes between bi-temporal remote sensing images.<n>We propose RSICCLLM, the first post-training framework for large vision-language models in RSICC.
  arXiv  Detail & Related papers  (2026-06-26T16:57:40Z)
• ViLaCD-R1: A Vision-Language Framework for Semantic Change Detection in Remote Sensing [5.966253859501895]
  ViLaCD-R1 is a two-stage framework comprising a Multi-Image Reasoner (MIR) and a Mask-Guided Decoder (MGD)<n>We show that ViLaCD-R1 substantially improves true semantic change recognition and localization, robustly suppresses non-semantic variations, and achieves imprecise state-of-the-art accuracy in complex real-world scenarios.
  arXiv  Detail & Related papers  (2025-12-29T06:58:46Z)
• Referring Change Detection in Remote Sensing Imagery [49.841833753558575]
  We introduce Referring Change Detection (RCD), which leverages natural language prompts to detect specific classes of changes in remote sensing images.<n>We propose a two-stage framework consisting of (I) textbfRCDNet, a cross-modal fusion network designed for referring change detection, and (II) textbfRCDGen, a diffusion-based synthetic data generation pipeline.
  arXiv  Detail & Related papers  (2025-12-12T16:57:12Z)
• CSD: Change Semantic Detection with only Semantic Change Masks for Damage Assessment in Conflict Zones [3.640978477877182]
  We introduce a pre-trained DINOv3 model and propose a multi-scale cross-attention difference siamese network (MC-DiSNet)<n>The powerful visual representation capability of the DINOv3 backbone enables robust and rich feature extraction from bi-temporal remote sensing images.<n>Unlike conventional semantic change detection (SCD), our approach eliminates the need for large-scale semantic annotations of bi-temporal images.
  arXiv  Detail & Related papers  (2025-11-24T12:16:21Z)
• OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval [59.377821673653436]
  Composed Image Retrieval (CIR) is capable of expressing users' intricate retrieval requirements flexibly.<n>CIR remains in its nascent stages due to two limitations: 1) inhomogeneity between dominant and noisy portions in visual data is ignored, leading to query feature degradation.<n>This work presents a focus mapping-based feature extractor, which consists of two modules: dominant portion segmentation and dual focus mapping.
  arXiv  Detail & Related papers  (2025-07-08T03:27:46Z)
• S2C: Learning Noise-Resistant Differences for Unsupervised Change Detection in Multimodal Remote Sensing Images [24.75086641416994]
  Untemporal Change Detection (UCD) in multimodal Remote Sensing (RS) images remains a difficult challenge. Inspired by recent advancements in Visual Foundation Models (VFMs) and Contrastive Learning (CL) methodologies, this research aims to develop CL methodologies to translate implicit knowledge in representations into change.
  arXiv  Detail & Related papers  (2025-02-18T07:34:54Z)
• Semi-supervised Semantic Segmentation for Remote Sensing Images via Multi-scale Uncertainty Consistency and Cross-Teacher-Student Attention [59.19580789952102]
  This paper proposes a novel semi-supervised Multi-Scale Uncertainty and Cross-Teacher-Student Attention (MUCA) model for RS image semantic segmentation tasks. MUCA constrains the consistency among feature maps at different layers of the network by introducing a multi-scale uncertainty consistency regularization. MUCA utilizes a Cross-Teacher-Student attention mechanism to guide the student network, guiding the student network to construct more discriminative feature representations.
  arXiv  Detail & Related papers  (2025-01-18T11:57:20Z)
• Cross-Domain Separable Translation Network for Multimodal Image Change Detection [11.25422609271201]
  multimodal change detection (MCD) is particularly critical in the remote sensing community. This paper focuses on addressing the challenges of MCD, especially the difficulty in comparing images from different sensors. A novel unsupervised cross-domain separable translation network (CSTN) is proposed to overcome these limitations.
  arXiv  Detail & Related papers  (2024-07-23T03:56:02Z)
• Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance [19.663899648983417]
  We introduce a novel change captioning (CC) method based on the foundational knowledge and semantic guidance. We validate the proposed method on the LEVIR-CC and LEVIR-CD datasets.
  arXiv  Detail & Related papers  (2024-07-19T05:07:41Z)
• Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
  Change captioning aims to succinctly describe the semantic change between a pair of similar images. Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
  arXiv  Detail & Related papers  (2024-07-16T13:00:33Z)
• CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation [9.493755431645313]
  We propose a novel, fully automatic method to sample additional focused and visually grounded captions. We leverage Abstract Meaning Representation (AMR) to encode all possible semantic-semantic relations between entities. We then develop a new model, CIC-BART-SSA, that sources its control signals from SSA-diversified datasets.
  arXiv  Detail & Related papers  (2024-07-16T05:26:12Z)
• AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization [57.34659640776723]
  We propose an end-to-end framework named AddressCLIP to solve the problem with more semantics. We have built three datasets from Pittsburgh and San Francisco on different scales specifically for the IAL problem.
  arXiv  Detail & Related papers  (2024-07-11T03:18:53Z)
• RSCaMa: Remote Sensing Image Change Captioning with State Space Model [29.945966783242337]
  Remote Sensing Image Change Captioning (RSICC) aims to describe surface changes between multi-temporal remote sensing images in language. This poses challenges to spatial and temporal modeling of bi-temporal features. We propose a novel RSCaMa model, which achieves efficient joint spatial-temporal modeling through multiple CaMa layers.
  arXiv  Detail & Related papers  (2024-04-29T17:31:00Z)
• BD-MSA: Body decouple VHR Remote Sensing Image Change Detection method guided by multi-scale feature information aggregation [4.659935767219465]
  The purpose of remote sensing image change detection (RSCD) is to detect differences between bi-temporal images taken at the same place. Deep learning has been extensively used to RSCD tasks, yielding significant results in terms of result recognition.
  arXiv  Detail & Related papers  (2024-01-09T02:53:06Z)
• ClusterFormer: Clustering As A Universal Visual Learner [80.79669078819562]
  CLUSTERFORMER is a universal vision model based on the CLUSTERing paradigm with TransFORMER. It is capable of tackling heterogeneous vision tasks with varying levels of clustering granularity. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.
  arXiv  Detail & Related papers  (2023-09-22T22:12:30Z)
• Multi-Content Complementation Network for Salient Object Detection in Optical Remote Sensing Images [108.79667788962425]
  salient object detection in optical remote sensing images (RSI-SOD) remains to be a challenging emerging topic. We propose a novel Multi-Content Complementation Network (MCCNet) to explore the complementarity of multiple content for RSI-SOD. In MCCM, we consider multiple types of features that are critical to RSI-SOD, including foreground features, edge features, background features, and global image-level features.
  arXiv  Detail & Related papers  (2021-12-02T04:46:40Z)
• Dual-Level Collaborative Transformer for Image Captioning [126.59298716978577]
  We introduce a novel Dual-Level Collaborative Transformer (DLCT) network to realize the complementary advantages of the two features. In addition, we propose a Locality-Constrained Cross Attention module to address the semantic noises caused by the direct fusion of these two features.
  arXiv  Detail & Related papers  (2021-01-16T15:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.