EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision

• URL: http://arxiv.org/abs/2501.08111v1
• Date: Tue, 14 Jan 2025 13:42:22 GMT
• Title: EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision
• Authors: Diego Velazquez, Pau Rodriguez López, Sergio Alonso, Josep M. Gonfaus, Jordi Gonzalez, Gerardo Richarte, Javier Marin, Yoshua Bengio, Alexandre Lacoste,
• Abstract summary: This paper presents a dataset specifically designed for self-supervision on remote sensing data, intended to enhance deep learning applications on Earth monitoring tasks. The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic. Accompanying the dataset is EarthMAE, a tailored Masked Autoencoder developed to tackle the distinct challenges of remote sensing data.
• Score: 72.84868704100595
• License:
• Abstract: This paper presents EarthView, a comprehensive dataset specifically designed for self-supervision on remote sensing data, intended to enhance deep learning applications on Earth monitoring tasks. The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic. Our dataset provides a wide spectrum of image data with varying resolutions, harnessed from different sensors and organized coherently into an accessible HuggingFace dataset in parquet format. This data spans five years, from 2017 to 2022. Accompanying the dataset, we introduce EarthMAE, a tailored Masked Autoencoder, developed to tackle the distinct challenges of remote sensing data. Trained in a self-supervised fashion, EarthMAE effectively processes different data modalities such as hyperspectral, multispectral, topographical data, segmentation maps, and temporal structure. This model helps us show that pre-training on Satellogic data improves performance on downstream tasks. While there is still a gap to fill in MAE for heterogeneous data, we regard this innovative combination of an expansive, diverse dataset and a versatile model adapted for self-supervised learning as a stride forward in deep learning for Earth monitoring.

Related papers

• AnySat: An Earth Observation Model for Any Resolutions, Scales, and Modalities [5.767156832161819]
  We propose AnySat, a multimodal model based on joint embedding predictive architecture (JEPA) and resolution-adaptive spatial encoders. To demonstrate the advantages of this unified approach, we compile GeoPlex, a collection of $5$ multimodal datasets. We then train a single powerful model on these diverse datasets simultaneously.
  arXiv  Detail & Related papers  (2024-12-18T18:11:53Z)
• M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data [1.4053129774629076]
  M3LEO is a multi-modal, multi-label Earth observation dataset. It spans approximately 17M 4x4 km data chips from six diverse geographic regions.
  arXiv  Detail & Related papers  (2024-06-06T16:30:41Z)
• UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction [93.77809355002591]
  We introduce UniTraj, a comprehensive framework that unifies various datasets, models, and evaluation criteria. We conduct extensive experiments and find that model performance significantly drops when transferred to other datasets. We provide insights into dataset characteristics to explain these findings.
  arXiv  Detail & Related papers  (2024-03-22T10:36:50Z)
• Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery [78.43828998065071]
  Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks. Such pre-training techniques have also been explored recently in the remote sensing domain due to the availability of large amount of unlabelled data. In this paper, we re-visit transformers pre-training and leverage multi-scale information that is effectively utilized with multiple modalities.
  arXiv  Detail & Related papers  (2024-03-08T16:18:04Z)
• Ben-ge: Extending BigEarthNet with Geographical and Environmental Data [1.1377027568901037]
  We present the ben-ge dataset, which supplements the BigEarthNet-MM dataset by compiling freely and globally available geographical and environmental data. Based on this dataset, we showcase the value of combining different data modalities for the downstream tasks of patch-based land-use/land-cover classification and land-use/land-cover segmentation.
  arXiv  Detail & Related papers  (2023-07-04T14:17:54Z)
• EarthNets: Empowering AI in Earth Observation [24.160463837610074]
  Earth observation (EO) aims at monitoring the state of planet Earth using remote sensing data. This paper presents a comprehensive review of more than 500 publicly published datasets. We propose to measure, rank, and select datasets to build a new benchmark for model evaluation.
  arXiv  Detail & Related papers  (2022-10-10T18:09:35Z)
• EOD: The IEEE GRSS Earth Observation Database [21.824996070545616]
  In the era of deep learning, annotated datasets have become a crucial asset to the remote sensing community. EOD is an interactive online platform for cataloguing different types of datasets leveraging remote sensing imagery.
  arXiv  Detail & Related papers  (2022-09-26T07:44:41Z)
• Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding [137.3719377780593]
  A new design (named Detection Hub) is dataset-aware and category-aligned. It mitigates the dataset inconsistency and provides coherent guidance for the detector to learn across multiple datasets. The categories across datasets are semantically aligned into a unified space by replacing one-hot category representations with word embedding.
  arXiv  Detail & Related papers  (2022-06-07T17:59:44Z)
• REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter [52.117388513480435]
  We present a new dataset named regrad to sustain the modeling of relationships among objects and grasps. Our dataset is collected in both forms of 2D images and 3D point clouds. Users are free to import their own object models for the generation of as many data as they want.
  arXiv  Detail & Related papers  (2021-04-29T05:31:21Z)
• Sketch and Scale: Geo-distributed tSNE and UMAP [75.44887265789056]
  Running machine learning analytics over geographically distributed datasets is a rapidly arising problem. We introduce a novel framework: Sketch and Scale (SnS) It leverages a Count Sketch data structure to compress the data on the edge nodes, aggregates the reduced size sketches on the master node, and runs vanilla tSNE or UMAP on the summary. We show this technique to be fully parallel, scale linearly in time, logarithmically in memory, and communication, making it possible to analyze datasets with many millions, potentially billions of data points, spread across several data centers around the globe.
  arXiv  Detail & Related papers  (2020-11-11T22:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.