Rethinking Electro-Optical Vision Foundation Models for Remote Sensing Retrieval: A Controlled Comparison with Generalist VFM

• URL: http://arxiv.org/abs/2605.02283v1
• Date: Mon, 04 May 2026 07:18:33 GMT
• Title: Rethinking Electro-Optical Vision Foundation Models for Remote Sensing Retrieval: A Controlled Comparison with Generalist VFM
• Abstract summary: Vision foundation models have attracted significant attention for their ability to leverage large-scale unlabeled visual data.<n>Recent electro-optical vision foundation models aim to learn domain-specific representations from remote sensing imagery.<n>It remains unclear whether they are more effective than strong generalist vision foundation models under retrieval-based evaluation.
• Score: 7.734759516415116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision foundation models have attracted significant attention for their ability to leverage large-scale unlabeled visual data. This advantage is particularly important in remote sensing, where data acquisition is costly and annotation often requires expert knowledge. Recent electro-optical vision foundation models aim to learn domain-specific representations from remote sensing imagery, but it remains unclear whether they are more effective than strong generalist vision foundation models under retrieval-based evaluation. In this study, we conduct a controlled comparison between representative EO-specific and generalist vision foundation models for remote sensing image retrieval. Using the same datasets, retrieval protocol, and evaluation metric, we evaluate both in-domain performance and cross-scene generalization. Our results show that strong generalist vision foundation models are competitive with, and in some cases outperform, existing EO-specific models. Moreover, EO-specific models often suffer from substantial degradation under cross-scene evaluation, while generalist models show more stable transfer. These findings suggest that EO pretraining alone does not guarantee stronger retrieval-oriented remote sensing representations. We discuss the limitations of current EO-specific pretraining strategies and highlight the need for future EO vision foundation models to better exploit the physical, spatial, spectral, and geographic characteristics of remote sensing imagery.

Related papers

• Cross-modal learning for SAR target recognition using optical vision foundation models [10.266600680575166]
  We investigate how vision foundation models trained on optical data can provide class level supervision for SAR classification.<n>A SAR model is then trained to classify SAR images while aligning its embeddings to the corresponding EO class prototype.<n>We evaluate our approach on the UNICORNv2 dataset, an EO and SAR dataset of civilian vehicles with heavily speckled images.
  arXiv  Detail & Related papers  (2026-09-07T16:54:30Z)
• Large Vision-Language Models Get Lost in Attention [51.851592109135716]
  We propose a unified framework grounded in information theory and geometry to quantify the geometric and entropic nature of residual updates.<n>Applying this unified framework reveals a fundamental functional decoupling: Attention acts as a subspace-language operator focused on reconfiguration, whereas FFNs serve as subspace-language operators driving semantic innovation.
  arXiv  Detail & Related papers  (2026-05-07T04:45:52Z)
• Benchmarking Vision Foundation Models for Domain-Generalizable Face Anti-Spoofing [2.099922236065961]
  Face Anti-Spoofing (FAS) remains challenging due to the requirement for robust domain generalization across unseen environments.<n>This paper revisits the potential of vision-only foundation models to establish a highly efficient and robust baseline for FAS.<n>Our comprehensive analysis reveals that self-supervised vision models, particularly DINOv2 with Registers, significantly suppress attention artifacts and capture critical, fine-grained spoofing cues.
  arXiv  Detail & Related papers  (2026-04-21T08:05:21Z)
• Enhancing Cross-View UAV Geolocalization via LVLM-Driven Relational Modeling [31.36539752384395]
  Cross-view UAV geolocalization aims to identify the exact spatial coordinates of drone-captured imagery by aligning it with extensive, geo-referenced satellite databases.<n>We introduce a novel, plug-and-play ranking architecture designed to explicitly perform joint relational modeling for improved UAV-to-satellite image matching.
  arXiv  Detail & Related papers  (2026-03-09T07:57:29Z)
• The View From Space: Navigating Instrumentation Differences with EOFMs [0.0]
  Earth Observation Foundation Models (EOFMs) have exploded in prevalence as tools for processing the massive volumes of remotely sensed and other earth observation data, and for delivering impact on the many essential earth monitoring tasks.<n>An emerging trend posits using the outputs of pre-trained models as 'embeddings' which summarize high dimensional data to be used for generic tasks such as similarity search and content-specific queries.<n>Most EOFM models are trained only on single modalities of data and then applied or benchmarked by matching bands across different modalities.<n>It is not clear from existing work what impact diverse sensor architectures have on the internal representations
  arXiv  Detail & Related papers  (2025-10-01T00:53:45Z)
• GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning [47.13305707860122]
  We propose a novel post training framework that incorporates task aware rewards to enable effective adaptation of reasoning based RL models to diverse Earth Observation tasks.<n>This training strategy enhances reasoning capabilities for remote sensing images, stabilizes optimization, and improves robustness.
  arXiv  Detail & Related papers  (2025-09-29T16:48:54Z)
• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms [91.19304518033144]
  We aim to align vision models with human aesthetic standards in a retrieval system. We propose a preference-based reinforcement learning method that fine-tunes the vision models to better align the vision models with human aesthetics.
  arXiv  Detail & Related papers  (2024-06-13T17:59:20Z)
• VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models [57.43276586087863]
  Large Vision-Language Models (LVLMs) suffer from hallucination issues, wherein the models generate plausible-sounding but factually incorrect outputs. Existing benchmarks are often limited in scope, focusing mainly on object hallucinations. We introduce a multi-dimensional benchmark covering objects, attributes, and relations, with challenging images selected based on associative biases.
  arXiv  Detail & Related papers  (2024-04-22T04:49:22Z)
• Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations [58.442103936918805]
  We show that Attention Mask Consistency produces superior visual grounding results than previous methods. AMC is effective, easy to implement, and is general as it can be adopted by any vision-language model.
  arXiv  Detail & Related papers  (2022-06-30T17:55:12Z)
• INFOrmation Prioritization through EmPOWERment in Visual Model-Based RL [90.06845886194235]
  We propose a modified objective for model-based reinforcement learning (RL) We integrate a term inspired by variational empowerment into a state-space model based on mutual information. We evaluate the approach on a suite of vision-based robot control tasks with natural video backgrounds.
  arXiv  Detail & Related papers  (2022-04-18T23:09:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.