CR-JEPA: Cross-Modal Joint-Embedding Predictive Learning for Remote Sensing Image Retrieval

• URL: http://arxiv.org/abs/2606.00706v2
• Date: Sat, 06 Jun 2026 17:41:51 GMT
• Title: CR-JEPA: Cross-Modal Joint-Embedding Predictive Learning for Remote Sensing Image Retrieval
• Abstract summary: Cross-modal remote sensing image retrieval aims to retrieve semantically related scenes across heterogeneous sensing modalities.<n>We propose CR-JEPA, a Cross-modal Retrieval Joint-Embedding Predictive Architecture for dual-modality remote sensing retrieval.
• Score: 17.886004315444357
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-modal remote sensing image retrieval aims to retrieve semantically related scenes across heterogeneous sensing modalities. This remains challenging because paired observations may differ substantially in imaging physics, spatial resolution, spectral configuration, and visual appearance. Moreover, a single retrieval projection trained with one objective may be insufficient to jointly support cross-modal semantic alignment and same-modal neighbourhood preservation. We propose CR-JEPA, a Cross-modal Retrieval Joint-Embedding Predictive Architecture for dual-modality remote sensing retrieval. The model uses modality-specific stems, a shared transformer trunk, and JEPA-style predictive objectives to estimate masked latent target features within and across modalities. Inspired by LeJEPA, we apply Sketched Isotropic Gaussian Regularization to raw retrieval projections to stabilize embeddings and mitigate collapse. CR-JEPA further employs a decoupled-head design with a unified retrieval head for same-modal retrieval and a cross-modal retrieval head for cross-modal search. We evaluate CR-JEPA on BEN-14K, CBRSIR_VS, and DSRSID. On BEN-14K, CR-JEPA improves S1 to S2 retrieval from 61.23% to 75.82% and S2 to S1 retrieval from 63.73% to 75.40% over X-JEPA, while also achieving competitive same-modal retrieval with fewer parameters.

Related papers

• HQ-JEPA: Hybrid Quantum Joint-Embedding Predictive Architecture for Cross-Modal Remote Sensing Representation Learning [17.65267962254814]
  HQ-JEPA is a hybrid quantum-classical joint-embedding predictive architecture for cross-modal remote sensing representation learning.<n>Results show that HQ-JEPA achieves competitive and often superior performance over strong self-supervised and remote sensing foundation-model baselines.
  arXiv  Detail & Related papers  (2026-05-29T09:37:35Z)
• Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration [3.0222726254970174]
  Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning.<n>This paper presents a hybrid belief-reinforcement learning framework to address this gap.<n>Results show 10.8% higher cumulative reward and 38% faster convergence over baselines.
  arXiv  Detail & Related papers  (2026-03-04T00:00:34Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• MOS: Mitigating Optical-SAR Modality Gap for Cross-Modal Ship Re-Identification [7.7794453452329]
  Cross-modal ship re-identification (ReID) between optical and synthetic aperture radar (SAR) imagery has emerged as a critical yet underexplored task in maritime intelligence and surveillance.<n>We propose MOS, a novel framework designed to mitigate the optical-SAR modality gap and achieve modality-consistent feature learning for optical-SAR cross-modal ship ReID.
  arXiv  Detail & Related papers  (2025-12-03T03:23:19Z)
• iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning [80.44805667907612]
  iEBAKER is an innovative strategy to filter weakly correlated sample pairs.<n>We introduce an alternative Sort After Reversed Retrieval (SAR) strategy.<n>We incorporate a Keyword Explicit Reasoning (KER) module to facilitate the beneficial impact of subtle key concept distinctions.
  arXiv  Detail & Related papers  (2025-04-08T03:40:19Z)
• REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval [11.105541122596003]
  REJEPA (Retrieval with Joint-Embedding Predictive Architecture) is an innovative self-supervised framework designed for unimodal RS-CBIR.<n>It uses spatially distributed context token encoding to forecast abstract representations of target tokens.<n>It achieves a reduction in computational complexity of 40-60% when compared to pixel-reconstruction baselines.
  arXiv  Detail & Related papers  (2025-04-04T04:59:10Z)
• GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learning [51.677086019209554]
  We propose a Generalized Structural Sparse to capture powerful relationships across modalities for pair-wise similarity learning. The distance metric delicately encapsulates two formats of diagonal and block-diagonal terms. Experiments on cross-modal and two extra uni-modal retrieval tasks have validated its superiority and flexibility.
  arXiv  Detail & Related papers  (2024-10-20T03:45:50Z)
• Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation [50.433911327489554]
  The goal of referring remote sensing image segmentation (RRSIS) is to generate a pixel-level mask of the target object identified by the referring expression.<n>To address the aforementioned challenges, a novel RRSIS framework is proposed, termed the cross-modal bidirectional interaction model (CroBIM)<n>To further forster the research of RRSIS, we also construct RISBench, a new large-scale benchmark dataset comprising 52,472 image-language-label triplets.
  arXiv  Detail & Related papers  (2024-10-11T08:28:04Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification [102.89434996930387]
  VI-ReID aims to match cross-modality pedestrian images, breaking through the limitation of single-modality person ReID in dark environment. Existing works manually design various two-stream architectures to separately learn modality-specific and modality-sharable representations. We propose a novel method, named Cross-Modality Neural Architecture Search (CM-NAS)
  arXiv  Detail & Related papers  (2021-01-21T07:07:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.