Scene Change Detection with Vision-Language Representation Learning

• URL: http://arxiv.org/abs/2604.11402v1
• Date: Mon, 13 Apr 2026 12:43:35 GMT
• Title: Scene Change Detection with Vision-Language Representation Learning
• Abstract summary: LangSCD is a vision-language framework for scene change detection that incorporates semantic reasoning through language.<n>We introduce NYC-CD, a large-scale dataset of 8,122 real-world image pairs collected in New York City with multiclass change annotations generated through a semi-automatic pipeline.
• Score: 11.186360973626114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scene change detection (SCD) is crucial for urban monitoring and navigation but remains challenging in real-world environments due to lighting variations, seasonal shifts, viewpoint differences, and complex urban layouts. Existing methods rely primarily on low-level visual features, limiting their ability to accurately identify changed objects amid the visual complexity of urban scenes. In this paper, we propose LangSCD, a vision-language framework for scene change detection that overcomes this single-modal limitation by incorporating semantic reasoning through language. Our approach introduces a modular language component that leverages vision-language models (VLMs) to generate textual descriptions of scene changes, which are fused with visual features through a cross-modal feature enhancer. We further introduce a geometric-semantic matching module that refines the predicted masks by enforcing semantic consistency and spatial completeness. Existing real-world scene change detection benchmarks provide only binary change annotations, which are insufficient for downstream applications requiring fine-grained understanding of scene dynamics. To address this limitation, we introduce NYC-CD, a large-scale dataset of 8,122 real-world image pairs collected in New York City with multiclass change annotations generated through a semi-automatic pipeline. Extensive experiments across multiple street-view benchmarks demonstrate that our language and matching modules consistently improve existing change-detection architectures, achieving state-of-the-art performance and highlighting the value of integrating linguistic reasoning with visual representations for robust scene change detection.

Related papers

• Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes [60.0903155031313]
  Long-term change understanding from images of the same place revisited over time is a challenging task with applications in map maintenance and urban infrastructure monitoring.<n>We introduce SPOT-THE-SHIFT, a human-verified benchmark for grounded image difference captioning of long-term changes in real-world driving scenes.<n>We propose an evaluation protocol that reliably assesses models' captioning ability, validated through human studies.
  arXiv  Detail & Related papers  (2026-09-09T15:52:04Z)
• UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model [17.995187387417563]
  Remote Sensing Image Change Captioning (RSICC) aims to generate spatially grounded natural language descriptions of scene evolution from bi-temporal imagery.<n>Current methods rely on implicit feature differencing without explicitly modeling structured change semantics.<n>We propose PTNet, a prototype-guided framework for joint change captioning and detection.
  arXiv  Detail & Related papers  (2026-05-06T02:04:18Z)
• Chatting with Images for Introspective Visual Thinking [50.7747647794877]
  ''Chatting with images'' is a new framework that reframes visual manipulation as language-guided feature modulation.<n>Under the guidance of expressive language prompts, the model dynamically performs joint re-encoding over multiple image regions.<n>ViLaVT achieves strong and consistent improvements on complex multi-image and video-based spatial reasoning tasks.
  arXiv  Detail & Related papers  (2026-02-11T17:42:37Z)
• Dynamic Context-Aware Scene Reasoning Using Vision-Language Alignment in Zero-Shot Real-World Scenarios [0.0]
  This work introduces a Dynamic Context-Aware Scene Reasoning framework to address zero-shot real-world scenarios.<n>The proposed approach integrates pre-trained vision transformers and large language models to align visual semantics with natural language descriptions.<n>Experiments demonstrate up to 18% improvement in scene understanding accuracy over baseline models in complex and unseen environments.
  arXiv  Detail & Related papers  (2025-10-30T15:07:55Z)
• VLHSA: Vision-Language Hierarchical Semantic Alignment for Jigsaw Puzzle Solving with Eroded Gaps [3.6380495892295173]
  We propose a vision-language framework that leverages textual context to enhance puzzle assembly performance.<n>Our approach centers on the Vision-Language Hierarchical Semantic Alignment (VLHSA) module.<n>Our work establishes a new paradigm for jigsaw puzzle solving by incorporating multimodal semantic insights.
  arXiv  Detail & Related papers  (2025-09-17T20:40:52Z)
• Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models [57.357091028792325]
  Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment.<n>We propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment.<n>Our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS)
  arXiv  Detail & Related papers  (2025-08-24T15:45:22Z)
• Linguistics-aware Masked Image Modeling for Self-supervised Scene Text Recognition [50.86415025650168]
  Masked image modeling (MIM) tends to exploit local structures to reconstruct visual patterns, resulting in limited linguistic knowledge.<n>We propose a Linguistics-aware Masked Image Modeling (LMIM) approach, which channels the linguistic information into the decoding process of MIM through a separate branch.
  arXiv  Detail & Related papers  (2025-03-24T14:53:35Z)
• OSCaR: Object State Captioning and State Change Representation [52.13461424520107]
  This paper introduces the Object State Captioning and State Change Representation (OSCaR) dataset and benchmark. OSCaR consists of 14,084 annotated video segments with nearly 1,000 unique objects from various egocentric video collections. It sets a new testbed for evaluating multimodal large language models (MLLMs)
  arXiv  Detail & Related papers  (2024-02-27T01:48:19Z)
• Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects [11.117055725415446]
  Large Vision Language Models (LVLMs) have demonstrated impressive zero-shot capabilities in various vision-language dialogue scenarios. The absence of fine-grained visual object detection hinders the model from understanding the details of images, leading to irreparable visual hallucinations and factual errors. We propose Lyrics, a novel multi-modal pre-training and instruction fine-tuning paradigm that bootstraps vision-language alignment from fine-grained cross-modal collaboration.
  arXiv  Detail & Related papers  (2023-12-08T09:02:45Z)
• CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representations [98.30038910061894]
  Vision-and-Language Navigation (VLN) tasks require an agent to navigate through the environment based on language instructions. We propose CLEAR: Cross-Lingual and Environment-Agnostic Representations. Our language and visual representations can be successfully transferred to the Room-to-Room and Cooperative Vision-and-Dialogue Navigation task.
  arXiv  Detail & Related papers  (2022-07-05T17:38:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.