Towards a Unified Copernicus Foundation Model for Earth Vision

• URL: http://arxiv.org/abs/2503.11849v2
• Date: Sat, 29 Mar 2025 20:01:44 GMT
• Title: Towards a Unified Copernicus Foundation Model for Earth Vision
• Authors: Yi Wang, Zhitong Xiong, Chenying Liu, Adam J. Stewart, Thomas Dujardin, Nikolaos Ioannis Bountos, Angelos Zavras, Franziska Gerken, Ioannis Papoutsis, Laura Leal-Taixé, Xiao Xiang Zhu,
• Abstract summary: We take a step towards next-generation Earth observation foundation models with three key components.<n>Copernicus-Pretrain is a massive-scale pretraining dataset that integrates 18.7M aligned images from all major Copernicus Sentinel missions.<n>Copernicus-FM is a unified foundation model capable of processing any spectral or non-spectral sensor modality.
• Score: 39.500074980218926
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advances in Earth observation (EO) foundation models have unlocked the potential of big satellite data to learn generic representations from space, benefiting a wide range of downstream applications crucial to our planet. However, most existing efforts remain limited to fixed spectral sensors, focus solely on the Earth's surface, and overlook valuable metadata beyond imagery. In this work, we take a step towards next-generation EO foundation models with three key components: 1) Copernicus-Pretrain, a massive-scale pretraining dataset that integrates 18.7M aligned images from all major Copernicus Sentinel missions, spanning from the Earth's surface to its atmosphere; 2) Copernicus-FM, a unified foundation model capable of processing any spectral or non-spectral sensor modality using extended dynamic hypernetworks and flexible metadata encoding; and 3) Copernicus-Bench, a systematic evaluation benchmark with 15 hierarchical downstream tasks ranging from preprocessing to specialized applications for each Sentinel mission. Our dataset, model, and benchmark greatly improve the scalability, versatility, and multimodal adaptability of EO foundation models, while also creating new opportunities to connect EO, weather, and climate research. Codes, datasets and models are available at https://github.com/zhu-xlab/Copernicus-FM.

Related papers

• Efficient Self-Supervised Learning for Earth Observation via Dynamic Dataset Curation [67.23953699167274]
  Self-supervised learning (SSL) has enabled the development of vision foundation models for Earth Observation (EO) In EO, this challenge is amplified by the redundancy and heavy-tailed distributions common in satellite imagery. We propose a dynamic dataset pruning strategy designed to improve SSL pre-training by maximizing dataset diversity and balance.
  arXiv  Detail & Related papers  (2025-04-09T15:13:26Z)
• Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection [2.8220015774219567]
  DEFLECT is a novel strategy for adapting GFMs to multispectral satellite imagery with very few additional parameters.<n>It achieves on-par or higher accuracy with 5-10$times$ fewer parameters for classification and segmentation tasks.
  arXiv  Detail & Related papers  (2025-03-12T15:53:58Z)
• SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation [81.36747103102459]
  Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications.<n>Current state-of-the-art methods focus on training innovative architectural designs on confined datasets.<n>We investigate the impact of scaling up EHPS towards a family of generalist foundation models.
  arXiv  Detail & Related papers  (2025-01-16T18:59:46Z)
• A Foundation Model for the Solar Dynamics Observatory [2.63089646549647]
  SDO-FM is a foundation model using data from NASA's Solar Dynamics Observatory (SDO) spacecraft. This paper marks release of our pretrained models and embedding datasets, available to the community on Hugging Face and sdofm.org.
  arXiv  Detail & Related papers  (2024-10-03T14:36:32Z)
• AI Foundation Model for Heliophysics: Applications, Design, and Implementation [1.2851259989174175]
  Foundation models (FMs) are pre-trained on a large-scale datasets. This paper provides our perspective on the criteria for designing an FM for heliophysics. We believe that this is the first study to design an FM in the domain of heliophysics.
  arXiv  Detail & Related papers  (2024-09-30T15:48:28Z)
• SpectralEarth: Training Hyperspectral Foundation Models at Scale [47.93167977587301]
  We introduce SpectralEarth, a large-scale multi-temporal dataset designed to pretrain hyperspectral foundation models. We pretrain a series of foundation models on SpectralEarth using state-of-the-art self-supervised learning (SSL) algorithms. We construct four downstream datasets for land-cover and crop-type mapping, providing benchmarks for model evaluation.
  arXiv  Detail & Related papers  (2024-08-15T22:55:59Z)
• Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation [48.66623377464203]
  Our novel approach introduces the Dynamic One-For-All (DOFA) model, leveraging the concept of neural plasticity in brain science. This dynamic hypernetwork, adjusting to different wavelengths, enables a single versatile Transformer jointly trained on data from five sensors to excel across 12 distinct Earth observation tasks.
  arXiv  Detail & Related papers  (2024-03-22T17:11:47Z)
• LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment [59.320414108383055]
  We present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation. We propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses.
  arXiv  Detail & Related papers  (2024-02-27T03:08:44Z)
• SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery [35.550999964460466]
  We present SkySense, a generic billion-scale model, pre-trained on a curated multi-modal Remote Sensing dataset with 21.5 million temporal sequences. To our best knowledge, SkySense is the largest Multi-Modal to date, whose modules can be flexibly combined or used individually to accommodate various tasks.
  arXiv  Detail & Related papers  (2023-12-15T09:57:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.