TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation

• URL: http://arxiv.org/abs/2506.06281v1
• Date: Fri, 06 Jun 2025 17:59:50 GMT
• Title: TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation
• Authors: Muhammad Sohail Danish, Muhammad Akhtar Munir, Syed Roshaan Ali Shah, Muhammad Haris Khan, Rao Muhammad Anwer, Jorma Laaksonen, Fahad Shahbaz Khan, Salman Khan,
• Abstract summary: We introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery.<n>Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism.<n>TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench.
• Score: 65.74990259650984
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern Earth observation (EO) increasingly leverages deep learning to harness the scale and diversity of satellite imagery across sensors and regions. While recent foundation models have demonstrated promising generalization across EO tasks, many remain limited by the scale, geographical coverage, and spectral diversity of their training data, factors critical for learning globally transferable representations. In this work, we introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery, combined with large spatial tiles and land-cover aware sampling to enrich spatial and semantic coverage. By treating sensing modalities as natural augmentations in our self-supervised approach, we unify radar and optical inputs via modality-specific patch embeddings and adaptive cross-attention fusion. Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism that incorporates class-frequency-aware regularization to address long-tailed distributions in land cover.TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench. Our code and pretrained models are publicly available at: https://github.com/mbzuai-oryx/TerraFM .

Related papers

• Towards Scalable and Generalizable Earth Observation Data Mining via Foundation Model Composition [0.0]
  We investigate whether foundation models pretrained on remote sensing and general vision datasets can be effectively combined to improve performance.<n>The results show that feature-level ensembling of smaller pretrained models can match or exceed the performance of much larger models.<n>The study highlights the potential of applying knowledge distillation to transfer the strengths of ensembles into more compact models.
  arXiv  Detail & Related papers  (2025-06-25T07:02:42Z)
• EarthMind: Towards Multi-Granular and Multi-Sensor Earth Observation with Large Multimodal Models [96.18182289276649]
  We present EarthMind, a novel vision-language framework for multi-granular and multi-sensor Earth Observation (EO) data understanding.<n>EarthMind features two core components: (1) Spatial Attention Prompting (SAP), which reallocates attention within the LLM to enhance pixel-level understanding; and (2) Cross-modal Fusion, which aligns heterogeneous modalities into a shared space.<n>To facilitate multi-sensor fusion evaluation, we propose EarthMind-Bench, a comprehensive benchmark with over 2,000 human-annotated multi-sensor image-question pairs.
  arXiv  Detail & Related papers  (2025-06-02T13:36:05Z)
• A Sensor Agnostic Domain Generalization Framework for Leveraging Geospatial Foundation Models: Enhancing Semantic Segmentation viaSynergistic Pseudo-Labeling and Generative Learning [5.299218284699214]
  High-performance segmentation models are challenged by annotation scarcity and variability across sensors, illumination, and geography.<n>This paper introduces a domain generalization approach to leveraging emerging geospatial foundation models by combining soft-alignment pseudo-labeling with source-to-target generative pre-training.<n> Experiments with hyperspectral and multispectral remote sensing datasets confirm our method's effectiveness in enhancing adaptability and segmentation.
  arXiv  Detail & Related papers  (2025-05-02T19:52:02Z)
• TerraMind: Large-Scale Generative Multimodality for Earth Observation [3.5472166810202457]
  We present TerraMind, the first any-to-any generative, multimodal foundation model for Earth observation.<n>Unlike other multimodal models, TerraMind is pretrained on dual-scale representations combining both token-level and pixel-level data.
  arXiv  Detail & Related papers  (2025-04-15T13:17:39Z)
• Efficient Self-Supervised Learning for Earth Observation via Dynamic Dataset Curation [67.23953699167274]
  Self-supervised learning (SSL) has enabled the development of vision foundation models for Earth Observation (EO)<n>In EO, this challenge is amplified by the redundancy and heavy-tailed distributions common in satellite imagery.<n>We propose a dynamic dataset pruning strategy designed to improve SSL pre-training by maximizing dataset diversity and balance.
  arXiv  Detail & Related papers  (2025-04-09T15:13:26Z)
• No Location Left Behind: Measuring and Improving the Fairness of Implicit Representations for Earth Data [13.412573082645096]
  Implicit neural representations (INRs) exhibit growing promise in addressing Earth representation challenges.<n>Existing methods disproportionately prioritize global average performance.<n>We introduce FAIR-Earth: a first-of-its-kind dataset to examine and challenge inequities in Earth representations.
  arXiv  Detail & Related papers  (2025-02-05T16:51:13Z)
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification [57.945437355714155]
  Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions.<n>Existing approaches focus on single-source domain generalization to unseen target domains.<n>We propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data.
  arXiv  Detail & Related papers  (2024-12-05T06:15:08Z)
• Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentation [7.265569559979736]
  High-quality annotated remote sensing images are often isolated and distributed across institutions.<n>The issue of remote sensing data islands poses challenges for fully utilizing isolated datasets to train a global model.<n>We propose a novel Geographic heterogeneity-aware Federated learning (GeoFed) framework to bridge data islands in RSS.<n>Our framework consists of three modules, including the Global Insight Enhancement (GIE) module, the Essential Feature Mining (EFM) module and the Local-Global Balance (LoGo) module.
  arXiv  Detail & Related papers  (2024-04-14T15:58:35Z)
• Federated Multi-Agent Mapping for Planetary Exploration [0.4143603294943439]
  We propose a federated multi-agent mapping approach that jointly trains a global map model across agents without transmitting raw data.<n>Our method leverages implicit neural mapping to generate parsimonious, adaptable representations, reducing data transmission by up to 93.8% compared to raw maps.<n>We demonstrate the efficacy of our approach on Martian terrains and glacier datasets, achieving downstream path planning F1 scores as high as 0.95 while outperforming on map reconstruction losses.
  arXiv  Detail & Related papers  (2024-04-02T20:32:32Z)
• Recognize Any Regions [55.76437190434433]
  RegionSpot integrates position-aware localization knowledge from a localization foundation model with semantic information from a ViL model.<n>Experiments in open-world object recognition show that our RegionSpot achieves significant performance gain over prior alternatives.
  arXiv  Detail & Related papers  (2023-11-02T16:31:49Z)
• Activation Regression for Continuous Domain Generalization with Applications to Crop Classification [48.795866501365694]
  Geographic variance in satellite imagery impacts the ability of machine learning models to generalise to new regions. We model geographic generalisation in medium resolution Landsat-8 satellite imagery as a continuous domain adaptation problem. We develop a dataset spatially distributed across the entire continental United States.
  arXiv  Detail & Related papers  (2022-04-14T15:41:39Z)
• Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification [61.44538721707377]
  We present Embedding Earth a self-supervised contrastive pre-training method for leveraging the large availability of satellite imagery. We observe significant improvements up to 25% absolute mIoU when pre-trained with our proposed method. We find that learnt features can generalize between disparate regions opening up the possibility of using the proposed pre-training scheme.
  arXiv  Detail & Related papers  (2022-03-11T16:14:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.