LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning

• URL: http://arxiv.org/abs/2607.03320v1
• Date: Fri, 03 Jul 2026 13:38:38 GMT
• Title: LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning
• Abstract summary: Remote sensing image change captioning (RSICC) generates descriptions of semantic changes between paired remote sensing images (RSIs)<n>LBTCap is a lightweight RSICC framework built on a bilateral Transformer that jointly models pre- and post-change features for efficient processing of paired RSIs.<n> experiments show that LBTCap matches or closely approaches the accuracy of state-of-the-art methods while using far fewer parameters and running at markedly higher inference speed.
• Score: 44.00535468968033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remote sensing image change captioning (RSICC) generates natural-language descriptions of semantic changes between paired remote sensing images (RSIs), supporting applications such as urban planning, disaster response, and environmental monitoring. Although recent methods achieve strong captioning accuracy, most overlook computational efficiency and inference speed, which are essential for real-time deployment in practice. To this end, we propose LBTCap, a lightweight RSICC framework built on a bilateral Transformer that jointly models pre- and post-change features for efficient processing of paired RSIs. Specifically, we introduce a bilateral attention mechanism for paired inputs: the two temporal images are projected into separate queries and keys by the same query and key matrices shared across the two images, the value is formed from their concatenation, and the two resulting attention maps are combined by a learnable, structurally bilateral weighting instead of a fixed subtraction. This design keeps both temporal branches explicit while remaining compact, and, together with a truncated backbone and grouped-query attention, LBTCap uses only 39.99M parameters, of which the change-aware encoder accounts for just 2.78M. Extensive experiments on two public RSICC datasets show that LBTCap matches or closely approaches the accuracy of state-of-the-art methods while using far fewer parameters and running at markedly higher inference speed, with the benefit of the bilateral formulation most pronounced in the low-resource setting, demonstrating a favorable accuracy-efficiency trade-off for practical RSICC.

Related papers

• Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation [6.274164868115626]
  Synthetic Aperture Radar (SAR) images have all-weather, day-and-night observation capabilities.<n>Generative models for SAR-to-optical (S2O) conversion can improve visual interpretability, but existing methods often ignore the constraints on semantic structure.<n>We propose a unified collaborative dual-task learning framework, termed BMT (Bridging Modalities and Tasks), that jointly optimize S2O image translation and semantic segmentation.
  arXiv  Detail & Related papers  (2026-09-04T04:50:19Z)
• Coupled Continuous-Discrete Generation for Scene Text Image Super-Resolution [51.31474187379532]
  Scene text image super-resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs.<n>We present DualTSR, a unified framework that formulates STISR as coupled continuous-discrete generation.<n> Conditional flow matching restores continuous image latents, while absorbing-state discrete diffusion reconstructs text tokens.
  arXiv  Detail & Related papers  (2026-08-05T06:57:30Z)
• DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction [8.972672149050627]
  Offsite conversion rate (OCVR) prediction is an important ranking problem in computational recommendation systems.<n>This task presents a modeling challenge: click signals are abundant and exhibit short temporal horizons, whereas conversion signals are inherently sparse, long-delayed, and frequently unattributed.<n>We propose DUET, a framework that explicitly partitions user behavioral data into two domain-coherent streams -- clicks and conversions.
  arXiv  Detail & Related papers  (2026-06-08T23:13:58Z)
• How Modality Shapes Perception and Reasoning: A Study of Error Propagation in ARC-AGI [7.226300346775942]
  ARC-AGI and ARC-AGI-2 measure generalization-through-composition on small color-quantized grids.<n>Recent instruction-first systems translate grids into concise natural-language or DSL rules executed in generate-execute-select loops.
  arXiv  Detail & Related papers  (2025-11-11T19:06:41Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models [65.96186414865747]
  Text-to-Image (T2I) diffusion models face a trade-off between inference speed and image quality.<n>We introduce the first Time-independent Unified TiUE for the student model UNet architecture.<n>Using a one-pass scheme, TiUE shares encoder features across multiple decoder time steps, enabling parallel sampling.
  arXiv  Detail & Related papers  (2025-05-28T04:23:22Z)
• Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval [15.503629941274621]
  This study proposes a Representation Discrepancy Bridging (RDB) method for the Remote Image-Text Retrieval (RSITR) task.<n>Experiments on RSICD and RSITMD datasets show that the proposed RDB method achieves a 6%-11% improvement in mR metrics.
  arXiv  Detail & Related papers  (2025-05-22T14:59:30Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• A Lightweight Sparse Focus Transformer for Remote Sensing Image Change Captioning [11.93705794906543]
  This paper proposes a Sparse Focus Transformer (SFT) for the remote sensing image change captioning (RSICC) task. The proposed SFT network can reduce the parameter number and computational complexity by incorporating a sparse attention mechanism.
  arXiv  Detail & Related papers  (2024-05-10T16:56:53Z)
• ResiDualGAN: Resize-Residual DualGAN for Cross-Domain Remote Sensing Images Semantic Segmentation [15.177834801688979]
  The performance of a semantic segmentation model for remote sensing (RS) images pretrained on an annotated dataset would greatly decrease when testing on another unannotated dataset because of the domain gap. Adversarial generative methods, e.g., DualGAN, are utilized for unpaired image-to-image translation to minimize the pixel-level domain gap. In this paper, ResiDualGAN is proposed for RS images translation, where a resizer module is used for addressing the scale discrepancy of RS datasets.
  arXiv  Detail & Related papers  (2022-01-27T13:56:54Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.