Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks

• URL: http://arxiv.org/abs/2606.10819v1
• Date: Tue, 09 Jun 2026 13:01:51 GMT
• Title: Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks
• Abstract summary: Earth-OneVision is a 2B RS-MLLM that unifies six sensor modalities and cross-sensor fusion across 9 task categories.<n>With only 2B parameters, Earth-OneVision achieves competitive or state-of-the-art results across extensive benchmarks.
• Score: 20.969678030412407
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RS-MLLMs enable natural-language understanding and spatial reasoning over earth observation imagery. However, existing models support only a narrow range of sensor types and tasks, yielding a fragmented view of the earth and leaving cross-modal geoscientific knowledge largely unexploited. This work presents Earth-OneVision, a 2B RS-MLLM that unifies six sensor modalities (i.e., optical, SAR, infrared, multispectral, temporal, and video) and cross-sensor fusion across 9 task categories within a single autoregressive framework. Three dedicated mechanisms address three bottlenecks. Full-Granularity Vision-Language Alignment (FGVLA) aligns multi-level visual features with the multi-dimensional language space. Spatial-Linguistic Isomorphic Serialization (SLIS) unifies heterogeneous spatial outputs as autoregressive tokens. Progressive Cross-Modality Adaptation (PCMA) decomposes the compound domain gap into sequential stages, tackling the viewpoint and imaging physics gaps in turn. To support joint training, MMRS-OneVision is constructed with ~34M QA pairs spanning all six sensor modalities and cross-sensor fusion across 9 task categories, substantially exceeding existing RS multimodal instruction datasets. With only 2B parameters, Earth-OneVision achieves competitive or state-of-the-art results across extensive benchmarks, consistently matching or outperforming 4B-72B RS-MLLMs. It achieves 87.52% P@0.5 on the OPT-RSVG testset for optical visual grounding and 80.68% on the SAR VQA benchmark SARLANG-Bench, exceeding 7B models by over 7%. It further achieves 75.74% recall on the BigEarthNet-MS testset for multispectral classification, and 81.94% MCQ accuracy on EarthMind-Bench for cross-modality reasoning.

Related papers

• Talk2Sensors: 3D Visual Grounding in Autonomous Driving via Sensor-Adaptive Physical Cue Matching [66.91228384649921]
  We introduce Talk2Sensors, the first multi-sensor 3D visual grounding dataset built upon camera, LiDAR, and 4D radar.<n>We propose TSFormer, a unified Transformer-based framework for language-guided 3D visual grounding in autonomous driving.
  arXiv  Detail & Related papers  (2026-08-05T07:58:35Z)
• SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts [9.401672754315614]
  SpaR3DMoE is an end-to-end framework that enables adaptive spatial reasoning by MLLMs with geometryaware capabilities from only sparse RGB inputs.
  arXiv  Detail & Related papers  (2026-07-07T09:27:38Z)
• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• Are VLMs Lost Between Sky and Space? LinkS$^2$Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligence [107.79496500634691]
  LinkS$2$Bench links 1,022 minutes of dynamic UAV footage with high-resolution satellite imagery covering over 200 km$2$.<n>Through an LMM-assisted pipeline and rigorous human annotation, we constructed 17.9k high-quality question-answer pairs.<n> Evaluations reveal a substantial gap compared to human baselines, identifying accurate cross-view dynamic alignment as the critical bottleneck.
  arXiv  Detail & Related papers  (2026-04-02T13:22:57Z)
• DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++ [1.1666234644810893]
  We propose DualSwinFusionSeg, a multimodal segmentation architecture that separates modality-specific feature extraction and performs multi-scale cross-modal fusion.<n>The model employs two parallel Swin Transformer V2 encoders to independently process RGB and auxiliary geophysical inputs, producing hierarchical feature representations.<n>Experiments on the MMLSv2 dataset from the PBVS 2026 Mars-LS Challenge show that modality-specific encoders and simple concatenation-based fusion improve segmentation accuracy under limited training data.
  arXiv  Detail & Related papers  (2026-03-14T21:42:14Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM [103.7537991413311]
  Earth Observation (EO) data analysis is vital for monitoring environmental and human dynamics.<n>Recent Multimodal Large Language Models (MLLMs) show potential in EO understanding but remain restricted to single-sensor inputs.<n>We propose EarthMind, a unified vision-language framework that handles both single- and cross-sensor inputs.
  arXiv  Detail & Related papers  (2025-06-02T13:36:05Z)
• RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation [24.48561340129571]
  RingMoE is a unified RS foundation model with 14.7 billion parameters, pre-trained on 400 million multi-modal RS images from nine satellites.<n>It has been deployed and trialed in multiple sectors, including emergency response, land management, marine sciences, and urban planning.
  arXiv  Detail & Related papers  (2025-04-04T04:47:54Z)
• Referring Remote Sensing Image Segmentation via Bidirectional Alignment Guided Joint Prediction [7.8862197224709805]
  ours is a novel framework designed to bridge the vision-language gap, enhance multi-scale feature interaction, and improve fine-grained object differentiation.<n>Experiments on the benchmark datasets RefSegRS and RRSIS-D demonstrate that ours achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-02-12T15:21:18Z)
• MLMT-CNN for Object Detection and Segmentation in Multi-layer and Multi-spectral Images [4.2623421577291225]
  We present a multi-task deep learning framework that exploits the dependencies between image bands to produce 3D AR localisation. Our framework achieves an average of 0.72 IoU (segmentation) and 0.90 F1 score (detection) across all modalities.
  arXiv  Detail & Related papers  (2024-07-19T17:21:53Z)
• MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection [53.03687787922032]
  Mamba-based models with superior long-range modeling and linear efficiency have garnered substantial attention.<n>This study pioneers the application of Mamba to multi-class unsupervised anomaly detection, presenting MambaAD.<n>The proposed LSS module, integrating parallel cascaded (Hybrid State Space) HSS blocks and multi- kernel convolutions operations, effectively captures both long-range and local information.
  arXiv  Detail & Related papers  (2024-04-09T18:28:55Z)
• MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection [89.26380781863665]
  Fusing LiDAR and camera information is essential for achieving accurate and reliable 3D object detection in autonomous driving systems. Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images into 3D space for fusion. We propose a novel framework that focuses on the multi-scale progressive interaction of the multi-granularity LiDAR and camera features.
  arXiv  Detail & Related papers  (2022-09-07T12:29:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.