Towards Realistic Open-Vocabulary Remote Sensing Segmentation: Benchmark and Baseline

• URL: http://arxiv.org/abs/2604.15652v1
• Date: Fri, 17 Apr 2026 02:49:46 GMT
• Title: Towards Realistic Open-Vocabulary Remote Sensing Segmentation: Benchmark and Baseline
• Abstract summary: Open-vocabulary remote sensing image segmentation (OVRSIS) remains underexplored due to fragmented datasets, limited training diversity, and the lack of evaluation benchmarks.<n>We propose textitOVRSISBenchV2, a large-scale and application-oriented benchmark for OVRSIS.<n>Our results highlight the importance of realistic benchmark design and the effectiveness of perturbation-based transfer for OVRSIS.
• Score: 52.65099689153431
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-vocabulary remote sensing image segmentation (OVRSIS) remains underexplored due to fragmented datasets, limited training diversity, and the lack of evaluation benchmarks that reflect realistic geospatial application demands. Our previous \textit{OVRSISBenchV1} established an initial cross-dataset evaluation protocol, but its limited scope is insufficient for assessing realistic open-world generalization. To address this issue, we propose \textit{OVRSISBenchV2}, a large-scale and application-oriented benchmark for OVRSIS. We first construct \textbf{OVRSIS95K}, a balanced dataset of about 95K image--mask pairs covering 35 common semantic categories across diverse remote sensing scenes. Built upon OVRSIS95K and 10 downstream datasets, OVRSISBenchV2 contains 170K images and 128 categories, substantially expanding scene diversity, semantic coverage, and evaluation difficulty. Beyond standard open-vocabulary segmentation, it further includes downstream protocols for building extraction, road extraction, and flood detection, thereby better reflecting realistic geospatial application demands and complex deployment scenarios. We also propose \textbf{Pi-Seg}, a baseline for OVRSIS. Pi-Seg improves transferability through a \textbf{positive-incentive noise} mechanism, where learnable and semantically guided perturbations broaden the visual-text feature space during training. Extensive experiments on OVRSISBenchV1, OVRSISBenchV2, and downstream tasks show that Pi-Seg delivers strong and consistent results, particularly on the more challenging OVRSISBenchV2 benchmark. Our results highlight both the importance of realistic benchmark design and the effectiveness of perturbation-based transfer for OVRSIS. The code and datasets are available at \href{https://github.com/LiBingyu01/RSKT-Seg/tree/Pi-Seg}{LiBingyu01/RSKT-Seg/tree/Pi-Seg}.

Related papers

• ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation [12.428190609914594]
  Training-free open-vocabulary remote sensing segmentation (OVRSS) has emerged as a promising paradigm for achieving category-agnostic semantic understanding in remote sensing imagery.<n>We propose ConInfer, a context-aware inference framework for OVRSS that performs joint prediction across multiple spatial units.<n>Our method significantly enhances segmentation consistency, robustness, and generalization in complex remote sensing environments.
  arXiv  Detail & Related papers  (2026-03-31T05:12:02Z)
• EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition [54.55914886780534]
  Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion.<n>We introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR.<n>EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios.
  arXiv  Detail & Related papers  (2026-02-13T13:25:05Z)
• SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images [49.52402091341301]
  Current models can parse simple, single-target commands but fail when presented with complex geospatial scenarios.<n>We present LaSeRS, the first large-scale dataset built for comprehensive training and evaluation.<n>We also propose SegEarth-R2, an MLLM architecture designed for comprehensive language-guided segmentation in RS.
  arXiv  Detail & Related papers  (2025-12-23T03:10:17Z)
• Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing [55.291219073365546]
  Open-Vocabulary Remote Sensing Image (OVRSIS) is an emerging task that adapts Open-Vocabulary (OVS) to the remote sensing (RS) domain.<n>textbfRSKT-Seg is a novel open-vocabulary segmentation framework tailored for remote sensing.<n> RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation.
  arXiv  Detail & Related papers  (2025-09-15T15:24:49Z)
• Semantic Localization Guiding Segment Anything Model For Reference Remote Sensing Image Segmentation [12.67400143793047]
  We propose a framework named textitprompt-generated semantic localization guiding Segment Anything Model(PSLG-SAM)<n>PSLG-SAM decomposes the Reference Remote Sensing Image (RRSIS) task into two stages: coarse localization and fine segmentation.<n> Notably, the second stage can be train-free, significantly reducing the annotation data burden for the RRSIS task.
  arXiv  Detail & Related papers  (2025-06-12T09:04:07Z)
• A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmark [8.707197692292292]
  We introduce NWPU-Refer, the largest and most diverse RRSIS dataset to date, comprising 15,003 high-resolution images (1024-2048px) spanning 30+ countries with 49,745 annotated targets.<n>We also propose the Multi-scale Referring Network (MRSNet), a novel framework tailored for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2025-06-04T05:26:51Z)
• SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model [61.97017867656831]
  We introduce a new task, ie, geospatial pixel reasoning, which allows implicit querying and reasoning and generates the mask of the target region.<n>We construct and release the first large-scale benchmark dataset called EarthReason, which comprises 5,434 manually annotated image masks with over 30,000 implicit question-answer pairs.<n>SegEarth-R1 achieves state-of-the-art performance on both reasoning and referring segmentation tasks, significantly outperforming traditional and LLM-based segmentation methods.
  arXiv  Detail & Related papers  (2025-04-13T16:36:47Z)
• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.