HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning

• URL: http://arxiv.org/abs/2605.15024v1
• Date: Thu, 14 May 2026 16:23:16 GMT
• Title: HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning
• Abstract summary: Remote sensing image change captioning (RSICC) aims to achieve high-level semantic understanding of genuine changes occurring between bi-temporal images.<n>Existing methods are fundamentally limited by a shared modeling assumption.<n>We propose a novel hierarchical semantic disentangling network (HiSem) that explicitly disentangles semantic representations of different granularities.
• Score: 30.51174005122181
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Remote sensing image change captioning (RSICC) aims to achieve high-level semantic understanding of genuine changes occurring between bi-temporal images. Despite notable progress, existing methods are fundamentally limited by a shared modeling assumption: changed and unchanged image pairs, which have intrinsically different semantic granularities, are processed under a unified modeling strategy. This modeling inconsistency leads to semantic entanglement between coarse-grained change existence judgment and fine-grained semantic understanding.To address the above limitation, we propose a novel hierarchical semantic disentangling network (HiSem) that explicitly disentangles semantic representations of different granularities. Specifically, we first introduce the Bidirectional Differential Attention Modulation (BDAM) module that leverages discrepancy-aware attention to enhance cross-temporal interactions, thereby amplifying true change signals while suppressing irrelevant variations. Building upon this, we design a Hierarchical Adaptive Semantic Disentanglement (HASD) module that performs adaptive routing at two hierarchical levels: a coarse-grained image-level routing mechanism distinguishes changed and unchanged image pairs, while a fine-grained token-level Mixture-of-Experts (MoE) block models diverse and heterogeneous change semantics for changed samples. Extensive experiments on two benchmark datasets demonstrate that HiSem outperfoms previous methods, achieving a significant improvement of +7.52\% BLEU-4 on the WHU-CDC dataset. More importantly, our approach provides a structured perspective for RSICC by explicitly aligning model design with the intrinsic semantic heterogeneity of bi-temporal scenes. The code will be available at https://github.com/Man-Wang-star/HiSem

Related papers

• Learning Semantic-Robust Change Detection via Semantic-Invariant Self-Distillation [16.080049512272026]
  SCDistill is a framework for learning semantic-robust change detection via semantic-invariant self-distillation.<n>It achieves state-of-the-art performance on multiple semantic change detection benchmarks and exhibits strong generalization to binary change detection and change captioning tasks.
  arXiv  Detail & Related papers  (2026-07-21T11:33:11Z)
• SemDINO: A DINOv3-Driven Network for Cross-Temporal Semantic Alignment in Change Detection [5.181191495929067]
  SemDINO is a novel end-to-end semantic change detection network.<n>It integrates a dual-branch encoder, multi-scale temporal interaction, semantic purification, change enhancement, and decoupled multi-task prediction.<n>Experiments on public remote sensing CD datasets demonstrate that SemDINO achieves superior performance and generalization ability against state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-08T17:32:31Z)
• Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generation [11.600113323546404]
  We propose a representation-driven framework that disentangles semantics from primitives for robust few-shot adaptation.<n>Semantic Anchoring aggregates categorical semantics into anchors for stable identity, while Primitive Imbuing models recomposable primitives for robust local detail modeling.<n>Experiments demonstrate consistent improvements in the 5-shot regime over state-of-the-art L2I methods in both visual fidelity and alignment across diverse domains.
  arXiv  Detail & Related papers  (2026-05-29T13:00:00Z)
• Detect Changes like Humans: Incorporating Semantic Priors for Improved Change Detection [52.62459671461816]
  This paper explores incorporating semantic priors from visual foundation models to improve the ability to detect changes.<n>Inspired by the human visual paradigm, a novel dual-stream feature decoder is derived to distinguish changes by combining semantic-aware features and difference-aware features.
  arXiv  Detail & Related papers  (2024-12-22T08:27:15Z)
• Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
  Change captioning aims to succinctly describe the semantic change between a pair of similar images. Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
  arXiv  Detail & Related papers  (2024-07-16T13:00:33Z)
• Align, Perturb and Decouple: Toward Better Leverage of Difference Information for RSI Change Detection [24.249552791014644]
  Change detection is a widely adopted technique in remote sense imagery (RSI) analysis. We propose a series of operations to fully exploit the difference information: Alignment, Perturbation and Decoupling.
  arXiv  Detail & Related papers  (2023-05-30T03:39:53Z)
• Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learning [74.48337375174297]
  Generalized Zero-Shot Learning (GZSL) identifies unseen categories by knowledge transferred from the seen domain. We deploy the dual semantic-visual transformer module (DSVTM) to progressively model the correspondences between prototypes and visual features. DSVTM devises an instance-motivated semantic encoder that learns instance-centric prototypes to adapt to different images, enabling the recast of the unmatched semantic-visual pair into the matched one.
  arXiv  Detail & Related papers  (2023-03-27T15:21:43Z)
• Joint Spatio-Temporal Modeling for the Semantic Change Detection in Remote Sensing Images [22.72105435238235]
  We propose a Semantic Change (SCanFormer) to explicitly model the 'from-to' semantic transitions between the bi-temporal RSIss. Then, we introduce a semantic learning scheme to leverage the Transformer-temporal constraints, which are coherent to the SCD task, to guide the learning of semantic changes. The resulting network (SCanNet) outperforms the baseline method in terms of both detection of critical semantic changes and semantic consistency in the obtained bi-temporal results.
  arXiv  Detail & Related papers  (2022-12-10T08:49:19Z)
• Diffusion Visual Counterfactual Explanations [51.077318228247925]
  Visual Counterfactual Explanations (VCEs) are an important tool to understand the decisions of an image. Current approaches for the generation of VCEs are restricted to adversarially robust models and often contain non-realistic artefacts. In this paper, we overcome this by generating Visual Diffusion Counterfactual Explanations (DVCEs) for arbitrary ImageNet classifiers.
  arXiv  Detail & Related papers  (2022-10-21T09:35:47Z)
• Semantic Image Synthesis via Diffusion Models [174.24523061460704]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.<n>Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches.<n>We propose a novel framework based on DDPM for semantic image synthesis.
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning [74.76431541169342]
  Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. We propose a novel hierarchical semantic-visual adaptation (HSVA) framework to align semantic and visual domains. Experiments on four benchmark datasets demonstrate HSVA achieves superior performance on both conventional and generalized ZSL.
  arXiv  Detail & Related papers  (2021-09-30T14:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.