RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation

• URL: http://arxiv.org/abs/2607.24810v1
• Date: Mon, 13 Jul 2026 08:02:57 GMT
• Title: RRS-10K: A Multitask Vision-Language Model Benchmark for Rare Remote Sensing Image Interpretation
• Abstract summary: Vision-language models (VLMs) have achieved strong performance on general remote sensing tasks.<n>Existing benchmarks are dominated by common urban and rural imagery.<n>We present RRS-10K, a benchmark for rare remote sensing image interpretation.
• Score: 19.526908696306506
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) have achieved strong performance on general remote sensing tasks. However, their capability for rare scenes remains insufficiently understood, because existing benchmarks are dominated by common urban and rural imagery. To address this gap, we present RRS-10K, a benchmark for rare remote sensing image interpretation. RRS-10K contains 10,738 military-related remote sensing images and corresponding multiple format question-answer pairs for comprehensive evaluation. All of the images are collected from first-hand sources and organized into three capability dimensions, six sub-dimensions, and 20 leaf tasks, covering perception, reasoning, and robustness. To improve the quality of multiple-choice questions, we introduce a similarity-based distractor filtering strategy (SDFS) during benchmark construction. We further evaluate 52 representative models and show that current VLMs achieve only moderate zero-shot performance on rare remote sensing image interpretation, with clear weaknesses in visual grounding, referring segmentation, and complex semantic reasoning tasks. RRS-10K enables systematic analysis of failure modes in long-tail remote sensing interpretation and provides guidance for developing more reliable remote sensing VLMs.

Related papers

• More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe [52.210953881548996]
  We show that a vision-language model can achieve competitive or state-of-the-art performance at challenging remote sensing benchmarks.<n>Our model uses a single language policy that can either answer directly in text or invoke a localization tool for segmentation and grounding.<n>Our approach achieves competitive results across a broad set of benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks.
  arXiv  Detail & Related papers  (2026-07-17T13:25:44Z)
• Beyond Visual Fidelity: Benchmarking Super-Resolution Models for Large-Scale Remote Sensing Imagery via Downstream Task Integration [61.02313993032298]
  Super-resolution (SR) techniques have made major advances in reconstructing high-resolution images from low-resolution inputs.<n>GeoSR-Bench is the first SR benchmark that directly connects improved image resolution from SR models with downstream Earth monitoring tasks.<n>We conduct experiments with 270 settings, covering 2 cross-platform SR tasks, 9 SR models, 3 downstream task models, and 5 downstream tasks for each SR task.
  arXiv  Detail & Related papers  (2026-05-01T00:44:46Z)
• Observe Less, Understand More: Cost-aware Cross-scale Observation for Remote Sensing Understanding [49.97682794425118]
  High-resolution (HR) imagery provides critical local details at much higher acquisition cost and limited coverage.<n>This motivates a cross-scale sensing strategy that selectively acquires HR imagery from LR-based global perception.<n>We present GL-10M, a large-scale benchmark of 10 million spatially aligned multi-resolution images.
  arXiv  Detail & Related papers  (2026-04-13T13:00:29Z)
• FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensing [21.38912956638889]
  MF-RSVLM is a multi-Feature Fusion Remote Sensing Vision--Language Model.<n>It learns multi-scale visual representations and combines global context with local details.<n>It achieves state-of-the-art or highly competitive performance across remote sensing classification, image captioning, and VQA tasks.
  arXiv  Detail & Related papers  (2025-12-30T06:48:07Z)
• A Benchmark for Ultra-High-Resolution Remote Sensing MLLMs [15.631314307574705]
  Multimodal large language models (MLLMs) demonstrate strong perception and reasoning performance on existing remote sensing (RS) benchmarks.<n>We show that text-only LLMs can perform competitively with multimodal vision-language models on RS reasoning tasks without access to images.<n>We introduce RSHR-Bench, a super-high-resolution benchmark for RS visual understanding and reasoning.
  arXiv  Detail & Related papers  (2025-12-19T08:07:51Z)
• Co-Training Vision Language Models for Remote Sensing Multi-task Learning [68.15604397741753]
  Vision language models (VLMs) have achieved promising results in RS image understanding, grounding, and ultra-high-resolution (UHR) image reasoning.<n>We present RSCoVLM, a simple yet flexible VLM baseline for RS MTL.<n>We propose a unified dynamic-resolution strategy to address the diverse image scales inherent in RS imagery.
  arXiv  Detail & Related papers  (2025-11-26T10:55:07Z)
• Efficient Few-Shot Learning in Remote Sensing: Fusing Vision and Vision-Language Models [0.33985395340995606]
  This work investigates the combination of vision models and Vision Language Models to enhance image analysis in remote sensing.<n>The integration of YOLO with VLMs such as LLaVA, ChatGPT, and Gemini aims to achieve more accurate and contextually aware image interpretation.<n>The findings show an average MAE improvement of 48.46% across models in the accuracy of aircraft detection and counting.
  arXiv  Detail & Related papers  (2025-10-15T18:19:48Z)
• SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization [57.484274282231226]
  We propose SVQA-R1, the first framework to extend R1-style training to spatial VQA.<n>In particular, we introduce Spatial-GRPO, a novel group-wise RL strategy that constructs view-consistent rewards by perturbing spatial relations between objects.<n>Our model, SVQA-R1, not only dramatically improved accuracy on spatial VQA benchmarks but also exhibits interpretable reasoning paths even without using supervised fine-tuning data.
  arXiv  Detail & Related papers  (2025-06-02T06:58:43Z)
• SARLANG-1M: A Benchmark for Vision-Language Modeling in SAR Image Understanding [20.314150537672198]
  Vision-Language Models (VLMs) have demonstrated remarkable success in RGB image understanding, offering powerful open-vocabulary interpretation and flexible language interaction.<n>We introduce SARLANG-1M, a large-scale benchmark tailored for multimodal SAR image understanding, with a primary focus on integrating SAR with textual modality.<n>It features hierarchical resolutions (ranging from 0.1 to 25 meters), fine-grained semantic descriptions (including both concise and detailed captions), diverse remote sensing categories, and multi-task question-answering pairs spanning seven applications and 1,012 question types.
  arXiv  Detail & Related papers  (2025-04-04T08:09:53Z)
• RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts [17.76606110070648]
  We propose RSUniVLM, a unified, end-to-end RS VLM for comprehensive vision understanding across multiple granularity.<n> RSUniVLM performs effectively in multi-image analysis, with instances of change detection and change captioning.<n>We also construct a large-scale RS instruction-following dataset based on a variety of existing datasets in both RS and general domain.
  arXiv  Detail & Related papers  (2024-12-07T15:11:21Z)
• RS-Mamba for Large Remote Sensing Image Dense Prediction [58.12667617617306]
  We propose the Remote Sensing Mamba (RSM) for dense prediction tasks in large VHR remote sensing images. RSM is specifically designed to capture the global context of remote sensing images with linear complexity. Our model achieves better efficiency and accuracy than transformer-based models on large remote sensing images.
  arXiv  Detail & Related papers  (2024-04-03T12:06:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.